Building an Audiovisual Synth #6: New Audio Engine with FAUST
Rebuilding the audio DSP with FAUST and adding a drum engine.
In the previous iteration, I was using fundsp for creating audio graphs. While that was working fine for a proof of concept, it wasn’t quite fitting for my purposes.
Relating Shaders and DSP Graphs
There are two things I really like about writing GLSL shaders:
You can create really complex visuals in a single file.
You can define a number of uniforms and use them across many files.
This makes it really easy to add new shaders and extend the system. For the audio part, I was looking for something similar, and this is where FAUST comes in.
Writing DSP Code in Faust
Being a functional programming language, it is very easy to define DSP graphs in FAUST. It has a nice syntax and comes with a huge standard library.
import(”stdfaust.lib”); // Imports standard library
freq = hslider(”Freq”, 440, 40, 2000, 1); // Input parameter in range
gate = button(”Gate”); // Input Trigger
osc = os.osc(freq); // Sine Oscillator with freq
env = en.ar(0.01, 0.2, gate); // Attack Release Envelope
process = (osc * env) <: _,_; // Main Output in StereoThe code above generates a simple sine wave in stereo with a frequency input and a trigger parameter. You can use the FAUST Online IDE to run and listen to it.
Compiling FAUST to Rust
While defining DSP graphs in FAUST is quite nice, what makes it even more powerful is its compiler, which can target many different platforms (like Rust, C++, Web Assembly, VCV Rack, and Daisy, just to name a few).
It structures the compiled code into a DSP and an Architecture part:
In my case with Rust, it created the raw DSP code in Rust, alongside some basic Faust-related DSP Traits and functions.
Then it added an architecture in Jack, which is used to play back the DSP code on an audio stream.
The good thing about separating these is that I could replace the Jack part with cpal (which I am using to play audio), and it worked with the same DSP code.
Updating Parameters to Modify Sounds
The key aspect of my synthesizer is having shared parameters across the graphic and audio engines, meaning that changing one will be directly visible and audible at the same time.
To achieve that, I had already implemented a triple buffer to share state between a coordinator thread and the audio thread. Using a triple buffer is crucial since every activity in the audio loop has to be non-blocking. This will ensure a new block of samples is delivered upon request by the audio buffer.
Building a Drum Engine
Having the groundwork done, I started to implement my first audio engine. An engine, in this context, is a system that has a fixed number of inputs.
For now, I am restricting myself to using four normalized float values in the range of 0.0 to 1.0 and four boolean or binary values that are either on or off. This constraint helps me to avoid parameter fatigue and from building overly complex stuff. It is also directly mapped to the physical hardware.
So the first engine I built was a drum engine. It has four gates that trigger kick, snare, hi-hat, and clap sounds. Each of them has one CV (float) input to control their characteristics. There were many things to tweak and fine-tune along the way.
Connecting Everything to Prototype Number Two
Having all that done, I also consolidated every part into one software prototype to test if it can be used together.
It seemed to work, and now I have something that can:
Take in keyboard, MIDI, and hardware input (on a Pi)
To control shared state (reflected on an OLED screen on a Pi)
That affects shaders, which can switch on runtime
And affects audio generation, which can also switch on runtime
While having one codebase that I can compile to different targets
Next Steps
With every part I add, there are more things I could optimize. In particular, the UI on the OLED needs some logic to use the device without MIDI or a keyboard. Speaking of the device, this needs a more permanent circuit and a proper case. Those two parts are what I think I will focus on next.