synthesizer-0.0.3: doc/Prologue.txt
This is a collection of modules for synthesizing and processing audio signals.
It allows generation of effects, instruments and
even music using the Haskore package.
It can write raw audio data to files,
convert them to common audio formats or
play them using external commands from the Sox package.
A signal is modeled by a list of values.
E.g. @[Double]@ represents a mono signal,
@[(Double, Double)]@ stores a stereo signal.
Since a list is lazy, it can be infinitely long,
and it also supports feedback.
(The drawback is, that its implementation is very slow.
I'm working on that issue.)
We are using the NumericPrelude type class hierarchy
which is cleaner than the one of Haskell 98
and provides us with type classes for vector spaces.
This allows us to formulate many algorithms for mono, stereo and multi-channel signals at once.
The drawback is that the vector space type class has multiple type parameters.
This type extension is availabe in GHC and Hugs and maybe other compilers.
It may hurt you, because type inference fails sometimes,
resulting in strange type errors.
(To be precise: GHC suggests type constraints intended for fixing the problem,
but if you copy them to your program, they won't fix the problem,
because the constraint refers to local variables
that you have no access to at the signature.
In this case you have to use 'asTypeOf' or similar self-written helpers.)
There must also be information about how fast sample values are emitted.
This is specified by the sample rate.
44100 Hz means that 44100 sample values are emitted per second.
This information must be stored along with the sample values.
This is where things become complicated.
In the very basic modules in the "Synthesizer.Plain" directory,
there is no notion of sample rate.
You have to base all computations on the number of samples.
This is unintuitive and disallows easy adaption to different audio devices
(CD, DAT, ...).
But it is very simple and can be re-used in the higher level modules.
Let's continue with the sample rate issue.
Sounds of different sources may differ in their sampling rate
(and also with respect to its amplitude and the unit of the values).
Sampled sounds have 44100 Hz on a compact disk,
48000 Hz or 32000 Hz on DAT recorders.
We want to respect different sampling rates and volumes,
we want to let signals in different formats coexist nicely,
and we want to let the user choose when to do which conversion
(called /resampling/)
in order to bring them together.
In fact this view generalises the concept of note, control, and audio rates,
which is found in some software synthesizers,
like CSound and SuperCollider.
If signals of different rate are fed to a signal processor
in such a software synthesizer,
all signals are converted to the highest rate among the inputs.
Then the processor runs at this rate.
The conversion is usually done by \"constant\" interpolation,
in order to minimize recomputation of internal parameters.
However the handling of different signal rates must be built into every processor,
and may even reduce the computation speed.
Consider an exponential envelope which is computed at control rate
and an amplifier which applies this envelope to an audio signal.
The amplifier has to upsample the exponential envelope before applying it to the signal.
But the generation of the exponential is very simple,
one multiplication per sample,
and the amplifier is very simple, too,
again only one multiplication per sample.
So, is there a need for trouble of the resampling?
Does it really accelerates computation?
Many other envelope generators like straight lines, sines, oscillators,
are comparably simple.
However there are some processors like filters,
which need some recomputation when a control parameter changes.
Our approach is this one:
We try to avoid resampling and compute all signals at the same rate,
if no speed loss must be expected.
If a speed loss is to be expected,
we can interpolate the internal parameters of the processor explicitly.
This way we can also specify an interpolation method.
Alternatively we can move the interpolation into the processor
but let the user specify an interpolation method.
(Currently it can be used only manually for the low-level routines in "Synthesizer.Plain"
and there is no support for that mechanism in the high level variants.)
Additional to the treatment of sampling rates,
we also want to separate amplitude information from the signal.
The separated amplitude serves two purposes:
(1) The amplitude can be equipped with a physical unit,
whereas this information is omitted for the samples.
Since I can hardly imagine that it is sensible to mix samples
with different physical units,
it would be only wasted time to always check
if all physical values of a sequence have the same unit.
(2) The amplitude can be a floating point number,
but the samples can be fixed point numbers.
This is interesting for hardware digital signal processors
or other low-level applications.
With this method we can separate the overall dynamics from the samples.
Let's elaborate on the physical units now.
With them we can work with values from the real world immediately
and we have additional (dynamic) safety by unit checks.
Of course I prefer static safety.
E.g. I want to avoid
to accidentally call a function with conflicting parameters.
However, I see no way for both applying the unit checks statically
and let the user enter physical quantities.
Phantom types or unit vectors stored in a type do not seem to help here.
We have two solutions:
(1) Store units in a data structure and check them dynamically.
This is imported from NumericPreludes's "Number.Physical".
Units can be fetched from the user.
The API of signal processing functions is generic enough
to cover both values without units and values with units.
Debugging of unit errors is cumbersome.
(2) Store physical dimensions in types
either using Buckwalter's dimensional package
or using NumericPreludes's "Number.DimensionTerm".
Here we use the latter one.
This is the most useful if user interaction is not needed.
If data is fetched from an audio file
the dimensions are statically fixed.
There are still several alternatives
of how to handle the sample rates
(that can be equipped with physical dimensions).
(1) Stick to simple lists as data and
pass additional information directly to the functions.
E.g. mixing several signals is easy
since only one sampleRate is given
which applies to all signals.
But it leads to the problem
that subsequent function calls must receive the same value.
This cannot be guaranteed and is thus a source of error.
E.g. the mistake
@play (44100*hertz) (osciSine (22050*hertz) (440*hertz))@
can't be detected.
In this approach the signal data structure is very simple,
the values may be passed to multiple functions,
the combinations are simply done by function application,
a supervisor is not necessary,
consistency checks can hardly be performed.
This approach is certainly the most basic one,
on which others, more safer ones, can sit on top.
It is implemented in "Synthesizer.Plain" with numbers without units.
(2) Equip signals with sample rate and amplitude.
Processors without input need the sample rate as explicit parameter.
If there is more than one signal as input,
then there must be additional checks.
The error in
@
mix (osciSine (22050*hertz) (440*hertz))
(osciSine (44100*hertz) (330*hertz))
@
can be detected at runtime.
However the sample rate has to be specified for both input signals,
although it is obvious, that both signals have to share the sample rate.
In this approach the data structure is more complex,
the values may be passed to multiple functions
but consistency checks can be performed
and a supervisor is still not necessary.
This strategy is implemented in the "Synthesizer.Physical" modules.
(3) We still like to hide the sample rate where possible.
All processors should work as good as possible at each rate.
Here we provide the sample rate to each processor.
The result of a processor is not just a list of samples
but it is a function, which computes the list of samples
depending on the sample rate.
Sample rate is fixed not until it comes to the rendering of a sound,
e.g. for playing or writing of a file.
@play (44100*hertz) (osciSine (440*hertz))@
Returning a function instead of computed data
has the disadvantage that multiply used data cannot be shared.
For these situations we need a @share@ function.
Combinator functions similar to @($)@ are used
to plug sample rate dependent output from one processor
into plain signal parameters.
With this approach, the type signature tells
which signals share the sample rate.
Infinitely many signals can be handled.
Types for time and volume can be chosen quite freely.
Supervision is not necessary.
This strategy is implemented in the "Synthesizer.Inference.Reader" modules,
where we hide the sample rate in a 'Control.Monad.Reader.Reader'.
There is also "Synthesizer.SampleRateContext"
which exposes the sample rate.
It is more convenient to implement and to call,
but I think it is more unsafe,
because you can mix sample rates from different sources accidentally.
The same is available for numbers with dimension terms in types.
See "Synthesizer.Dimensional".
/In most cases this will be the method of choice!/
Maybe I'm going to wrap this in a Reader monad\/applicative functor.
It also requires that Haddock supports comments in parameters of type constructors.
(4) I have tried more sophisticated approaches
in order to handle not only the sample rates but also the amplitudes.
However I feel that I wanted more than I actually needed.
I do no longer maintain these approaches but explain them for completeness.
The most convenient solution for handling sample rates and amplitudes
is certainly an inference system like Haskell's type system.
If some input and output signals of a processor
must have the same sampling rate,
then the concrete rate must only be known for one of these signals.
If no participating signal has a fixed rate, this is an error.
The dependencies of sampling rates become very large by this system.
The direction can be from inputs to outputs and vice versa,
not to mention loops.
This approach needs a lot of management,
e.g. a supervisor which runs the network,
but it is very convenient and safe.
However, sometimes you have to fiddle with monads.
Unfortunately it is restricted to finitely many monads
and the types for time and volume are restricted.
Thus this concept does not scale to physical units expressed in types.
This strategy is implemented in the modules under "Synthesizer.Inference.Monad".
(5) We try to work-around the restrictions
using a function based approach.
Since the parameters are functions,
sharing cannot take place.
There is no way to spread sample rate from one consumer to another one.
E.g. If there is
@
let y = f x;
z = g x
@
and it is known that @f@ and @g@ maintain the sample rate,
and the sample rate of @z@ is known - how to infer the sample rate of @y@?
This approach was dropped quickly and
exists for historical reasons in "Synthesizer.Inference.Func".
(6) There is a very cool approach,
which implements the equation solver of the monadic approach
by lazy evaluation and Peano numbers.
This poses no restriction on types
and works for infinitely many equations as well.
The drawbacks are difficult application
(you cannot simply apply a function to a signal,
but you must compose functions in a special way),
and slow solution of the equation system
(quadratic time although in principle
only run-time around linear time is necessary,
it's similar to topological sort).
However it's as slow as the explicit solver using monads in "Synthesizer.Inference.Monad".
This strategy is tested in the modules under "InferenceFix".
An interface to the music composition library Haskore
can be found in "Haskore.Interface.Signal.Write".
Example songs based on this interface
are stored in the directory "Music".
The module "Presentation" in the @dafx@ package contains functions
for demonstrating synthesizer functions in GHCi
and "DAFx" contains some examples based on them.
Just hit @make dafx@ in a shell in order to compile the modules
and enter the interactive GHC with all modules loaded.