Algorithmic Waveform Synthesizer

Algorithmic Waveform or AW Synth is kind of a code-it-yourself synthesizer. That may sound complex, but in the simplest case you can just use one of the built-in waveform generators and adjust amplitude and pitch envelope. On the other hand you could implement a fully custom synth using FM and other synthesis methods.

Here you can listen to three sound effects that use just square wave plus amplitude and pitch envelope, and at the end some FM synthesized laser gun shooting:

I have also made a sound effect editor, AW Patch Editor, that works both in FemtoIDE and in a web browser. The zip file below contains the AW Patch Editor plus AWSynthSource and SimpleTuneAW classes together with example main.cpp and sound effects.

Source code: AWSynth.zip (40.7 KB)

You can also get it from GitHub: https://github.com/jpfli/awsynth

Or try the editor online: https://jpfli.github.io/awsynth

The core idea of the AW Synth is similar to bytebeat: You define a callback function that is evaluated for each time frame and the returned values describe the waveform. The most notable difference is the additional phase input variable p. Unlike the t variable, itā€™s increase rate isnā€™t fixed, but depends on the note being played, which allows you to generate sounds at the desired pitch. Also, instead of unsigned 8-bit value, the callback function returns a signed 8-bit value (actually 32-bit, but clipped to range -128ā€¦127).

AWSynth and SimpleTuneAW basic usage example:

#include <Pokitto.h>
#include "AWSynthSource.h"
#include "SimpleTuneAW.h"

int main() {
    using Pokitto::Core;
    Core::begin();
    
    using AWSynth = Audio::AWSynthSource;
    using AWPatch = Audio::AWPatch;
    
    // Create a patch
    auto arp_patch = AWPatch([](std::uint32_t t, std::uint32_t p)->std::int32_t {  // Lambda callback function
            return AWSynth::sqr(p);         // Square wave generator
        })
        .volume(80).step(6).release(12)     // Volume 80%, step duration 6*8.33ms, release 12*step
        .amplitudes(AWPatch::Envelope(100,100,100,100).loop(32,4))                  // Length 4 steps, then loop (jump) to end
        .semitones(AWPatch::Envelope(   0, 12,  4,  7).smooth(false).loop(0,4));    // Use discrete pitches to create an arpeggio
    
    AWSynth& snd = AWSynth::play<0>(arp_patch, 52);     // Play arp_patch on channel 0, midikey 52 (E-3)
    
    // If the amplitude envelope doesn't loop, release is automatically triggered at the end. Otherwise manually release it:
    //snd.release();
    
    // Create a SimpleTuneAW tune
    constexpr auto tune = SIMPLE_TUNE_AW(A-3,A-3,G-3,E-4,E-4,D-4,D-4,A-3,A-3).tempo(120*8);    // Tempo 120, use 8th notes

    Audio::playTuneAW<1>(tune).patch(arp_patch);        // Play the tune on channel 1 using arp_patch
    
    while(Core::isRunning()) {
        if(!Core::update()) {
            continue;
        }
    }
}

Amplitude and Pitch Envelope

There are no ADRS envelopes. Instead, AW Synth uses arrays of max 32 values. Each value specifies the amplitude (0 to 100 %) or relative pitch (-128 to +127 semitones) at the corresponding step. Step duration is defined with the AWPatch parameter step in 8.33 ms increments. This technique allows you to create quite versatile envelopes and even arpeggios or simple sequences.

auto patch1 = Audio::AWPatch([](std::uint32_t t, std::uint32_t p)->std::int32_t { return AWSynth::sin(p); })
    .step(5)        // Step duration 5*8.33ms
    // Start from 100, decay to 80 and sustain the note until released
    .amplitudes(Audio::AWPatch::Envelope(100,98,96,94,92,90,88,86,84,82,80).smooth(true).loop(10,11))
    // Pitch slide down and vibrato
    .semitones(Audio::AWPatch::Envelope(12,10, 8, 6, 4, 2, 0, 0, 0, 0, 0, 0, 0, 1, 0, -1).smooth(true).loop(12,16));

smooth option lets you choose whether the values are discrete (=false) or linearly
interpolated (=true, default) from one step to next.

loop option takes two parameters: loop start index and length of the envelope. For example loop(12,16) means that after index 15 the envelope loops back to index 12. Looping can be disabled by setting length parameter to 0 or >32.

If the end of the envelope is reached, the last value is sustained, and in case of amplitude envelope, release is automatically triggered. You can trigger release earlier by setting loop start parameter to 32, for example loop(32,11).

Waveform Generators

AW Synth has built-in functions that take input phase variable p and turn it into one of the basic waveforms.

// Waveform generators turn input variable 'p' (phase, 256 is full period) into a waveform.
// Output is in the range -128..128

// Sine wave, triangluar wave and square wave generators
static constexpr std::int32_t sin(std::uint32_t p) { std::int32_t o = ((p&127) * (-p&127)) >> 5; return p&128 ? -o : o; }
static constexpr std::int32_t tri(std::uint32_t p) { std::int32_t o = (2*(p+64)) & 255; return (p+64)&128 ? 128-o : o-128; }
static constexpr std::int32_t sqr(std::uint32_t p) { return (p&128) ? -128 : 128; }

// Pulse wave generator with variable pulse width 'w'
static constexpr std::int32_t sqr(std::uint32_t p, std::uint32_t w) { return (p&255) < w ? 128 : -128; }

// Sawtooth or modified triangle wave generator. Optional template parameter 'XPeak' defines the x location (0 to 256) of the peak.
// XPeak values 0 (default) and 256 give a sawtooth wave. XPeak=128 gives a symmetric triangle wave, similar to 'tri()'.
template<signed XPeak=0>
static constexpr std::int32_t saw(std::uint32_t p) {
    static_assert(XPeak>=0 && XPeak<=256);
    constexpr std::int32_t RATE_A = XPeak>1 ? ((1<<20)/XPeak) : (1<<20);
    constexpr std::int32_t RATE_B = XPeak<255 ? ((1<<20)/(256-XPeak)) : (1<<20);
    std::int32_t o = (p+XPeak/2) & 255;
    return ((o<XPeak ? o*RATE_A : (256-o)*RATE_B) >> 12) - 128;
}

// Noise generator
static inline std::int32_t noise() {
    static std::uint32_t state = 0x5DEECE66;
    state ^= state << 17;
    state ^= state >> 13;
    state ^= state << 5;
    return ((state>>24) & 255) - 128;
}

Full period is 256. When e.g. note A-4 (440 Hz) is played, p increases by 256 every 1/440 seconds or 2.273 milliseconds. In the callback function, p can be multiplied and divided. sin(p/2) generates a tone one octave lower. You can even write something like sin(p + sin(2*p)) for basic two operator FM, where one oscillator is modulated by another.

Ramp Generator and LFO

In addition to the waveform generators, there is ramp and lfo function. Ramp generator is used to create amplitude attack and decay envelopes and pitch slides. LFO is a higher resolution, custom period length sine wave generator and it can be used for vibrato and tremolo effects. Usually, though, it is easier to use the built-in amplitude and pitch envelopes.

// Envelope generator takes input variable 't' (ticks) and template parameter 'T' (duration).
// Returned value is 0 when t<=0, 1<<14 when t>=T and increases linearly when 0<t<T
template<signed T>
static constexpr std::int32_t ramp(std::int32_t t) {
    static_assert(T>0 && T<=(1<<30));
    return t<=0 ? 0 : (t>=T ? (1<<14) : (t * ((1<<(14+16))/T) >> 16));
}

// LFO is a sine wave generator that takes input variable 't' (ticks) and template parameter 'T' (full period).
// Output is in the range -1<<14..1<<14 for greater resolution
template<signed T>
static constexpr std::int32_t lfo(std::uint32_t t) {
    t = t*(((1<<(14+16))+T-1)/T) >> 16;
    std::uint32_t o = ((t&8191)*(-t&8191)) >> 10;
    return t&8192 ? -o : o;
}

SineBeat and Parametric ByteBeat

Naturally AW Synth can also do bytebeat. and the following examples demonstrate two possible ways it can be used. First example uses bytebeat equation as phase input for sine generator. This way you get the same bytebeat tune, but with smoother, almost sine waveform instead of the typical sawtooth.

auto sinebeat_patch = AWPatch([](std::uint32_t t, std::uint32_t p)->std::int32_t {
        std::uint32_t b = t*((t>>9|t>>13)&25&t>>6); // Evaluate bytebeat equation
        return AWSynth::sin(b);                     // Use as phase input for sine wave generator
    })
    .amplitudes(AWPatch::Envelope(0,100).loop(1,2));

Second example uses a callback function with a variable passed as user data to interactively change the bytebeat equation while it is playing. Different values of parameter p1 give you different tunes that still have some similar characteristics. This could be used for example as game background music that changes when there are more enemies around or when health bar gets low.

std::uint32_t parambeat_p1 = 1;                 // Variable passed to the callback function as user data

auto parambeat_patch = AWPatch([](std::uint32_t t, std::uint32_t p, void* data)->std::int32_t { // Lambda callback function with user data
        std::uint32_t& p1 = *reinterpret_cast<std::uint32_t*>(data);
        std::int32_t o = ((p1*t)>>4)|(t>>5)|t;  // Evaluate bytebeat equation using current value of p1
        return (o&255) - 128;                   // Bytebeat result must be truncated to 8-bits and converted to signed value
    }, parambeat_p1)
    .amplitudes(AWPatch::Envelope(0,100).loop(1,2));

4 Likes

This sounds very good for a generated music. Ability to variate it based on the enemy count is great!

1 Like

After reading @tuxinator2009ā€™s FemtoIDE Custom Editor Tutorial, I started wondering if it would be possible to make a sound editor for FemtoIDEā€¦ and yes, it is. This is an editor to create AW Synth patches.

How to get started:

  • Download the zip file in the first post (or here)
  • Copy the contents of the zip file into your project (excluding main.cpp and sounds folder if you donā€™t want examples)
  • You should now have at least AWPatchEditor and scripts folders and AWSynthSource.h in the project folder
  • Start FemtoIDE and open your project
  • You can create a new empty mysfx.awpatch file or click one of the example .awpatch files to open the editor

In FemtoIDEā€™s Scripts menu you should see Convert AW Patches option. It searches recursively all folders in the project and converts all .awpatch files into .h files. The path is added to the name of the sound inside the .h file. Then you can use your sound effect in the code like this:

#include "AWSynthSource.h"
#include "sounds/mysfx.h"

void playMySfx() {
    Audio::AWSynthSource::play<0>(sounds_mysfx, 60); // play note C4 using mysfx patch
}

The editor currently has four waveforms. I might add more if I can figure out how to use Web Audio API AudioWorklet.

Some of you might notice similarities with Pico-8 sound editor. It has much more features and is like a simple tracker, but if you know how to use sound editor in Pico-8, then using AW Patch Editor should be easy. Also, tutorials such as this video on making various Super Mario sounds can be helpful (only first half is relevant, after that it gets too advanced):

12 Likes

Awesome work!

1 Like

This is great!

1 Like

Great addition!

1 Like

This will definitely be the way I will do sfx for a game next time :slight_smile:

1 Like

Added some features to the editor:

  • 3 new waveforms: pulse, soft sawtooth, noise
  • Waveform view, which is used to set waveform for each step
  • 4 effects that can be applied to pitch and amplitude envelopes for each step (replaces smooth option):
    • step: constant value, no slide
    • attack: ramp from 0 to value
    • decay: ramp from value to 0
    • slide: slide from previous to current value
  • Buttons to load from and save to a file when editor is started in a web browser

With the new features, I think itā€™s now possible to do most of the things you could do in Pico-8 sfx editor. I have updated the zip file in the first post. Included are drum beat, explosions and other example sound effects.

You can also get it from GitHub: https://github.com/jpfli/awsynth

Or try the editor online: https://jpfli.github.io/awsynth

AWSynthSource class

I had to make some changes to the AWSynthSource class too. One small but significant change is the t parameter of the callback function. It is no longer increased by one each time frame. Instead it now counts 1/256th steps, so the increase rate depends on the selected step length. This makes it possible to synchronize things with envelope steps, for example to play different waveform each step. If step length is 4, then the increase rate is 7680/second (120/4 steps/second times 256 => 7680), which is roughly equivalent to the old t with 8000 Hz sample rate.

Amplitude and pitch envelope effects can be set like this:

constexpr auto tune = Audio::AWPatch([](std::uint32_t t, std::uint32_t p)->std::int32_t { return Audio::AWSynthSource::tri(p); })
    .volume(80).step(4).release(0).glide(0)
    .semitones(Audio::AWPatch::Envelope(-2,0,0,0,2,2,4,4,5,5,5,5)
        .effects(2,0,0,0,3,0,3,0,1,0,0,2)   // 0=step, 1=attack, 2=decay, 3=slide
        .loop(32,12))
    .amplitudes(Audio::AWPatch::Envelope(31,31,31,31,31,31,31,31,31,31,31,31)
        .effects(1,0,0,2,1,2,1,2,1,0,0,2)   // 0=step, 1=attack, 2=decay, 3=slide
        .loop(32,12));

Also note that maximum amplitude is now 31 instead of 100. This is because amplitude is stored as signed 8-bit value and 2 bits are reserved for the effect type.

5 Likes

Excellent, I need to get time to try this! Anyone feels like a Game Jam would be a good idea? :slight_smile:

4 Likes

Petition to embedded in FemtoIde release?

3 Likes

That is a good idea! This could come built-in with FemtoIDE.

2 Likes

The online sfx editor is really great! Very easy to make experiments.
Few comments:

  • Selecting a waveform or an effect needs a double click. I often mistakenly thought that one click is enough.
  • there could be more help texts explaining the different functions. Or a separate help page.
  • few times the audio was lost in Chrome. The Edge browser seemed to work better.
  • the saved file looked a bit hard to read. Few line feeds could make it clearer.

Good work!

1 Like

Just for clarification, single click does select a waveform/effect, after which you can paint individual steps in the waveform/envelope view.

Double-click is used to automatically paint all the steps.

Iā€™ll add some help pop-ups.

Maybe itā€™s because Iā€™m using ScriptPocessor for direct audio processing, which is deprecated and has been replaced by AudioWorklet.

I tried AudioWorklet, but havenā€™t had success with it. I always get a Cross-Origin Resource Sharing (CORS) error. Which, I guess, is because the audio processing callback function needs to be loaded from a separate .js file.

Does anyone have ideas how to solve the CORS issue?

The file is not meant to be read by humans, but there seems to be some parameters you can give to the JSON.stringify function to change how the output looks, so it should be easy thing to do.

Thanks for the feedback.

Oh, you can select a waveform for each note in the sound? Cool!

Yes, it can be used e.g. to create drum beats and tunes with more than one instrument. You can also create interesting sounds by rapidly alternating two waveforms. Check out the included drumbeat and explosion patches for examples.

Does this happen when you try to open your editor in a browser using a ā€œfile://ā€ url? Or online?

Iā€™m using ā€œfile://ā€ url. I havenā€™t tried online. For it to be useful with FemtoIDE, I need to make it work from local filesystem.

Iā€™m testing with this noise generator example: https://github.com/pioug/audio-worklet

Yeah, a file: url isnā€™t going to work. One workaround would be to host an http server in the project script and use that to load your editor. I havenā€™t tried it, but it should be pretty simple.

Thanks, I had no idea something like that was even possible. The ScriptProcessor works good enough for now, but Iā€™ll see if I manage to make that server thing work, so I can switch to AudioWorklet.

That actually works! I managed to create a simple bytebeat generator using AudioWorklet.

Here is the javascript file that starts a local http server (goes in ā€˜scriptsā€™ folder):

CustomEditor.js
//!APP-HOOK:addMenu
//!MENU-ENTRY:Custom Editor

const fs = require('fs').promises;
const http = require("http");
      
if(!APP.customEditorInstalled()){
  APP.log("Adding custom editor");
  
  // list of file extensions this view can edit
  const extensions = ["CE"];
  
  // path to the editor directory
  const dirname = `${DATA.projectPath}/editor`;
  // the html to load, file path will be concatenated after the "?"
  const prefix = 'http://localhost:8080/editor.html?';
  
  // add extensions for binary files here
  Object.assign(encoding, {
    "CE":null
  });
  
  class CustomEditorView {
    
    // gets called when the tab is activated
    attach(){
      if( this.DOM.src != prefix + this.buffer.path )
        this.DOM.src = prefix + this.buffer.path;
      this.DOM.contentWindow.readFile = this.readFile;
      this.DOM.contentWindow.saveFile = this.saveFile;
      
      // create http server
      this.server = http.createServer((req, res) => { // http request handler
        const len = req.url.indexOf('?');
        let filename = dirname + (len < 0 ? req.url : req.url.slice(0, len));
        filename = filename + (filename.endsWith("/") ? "index.html" : "");
        fs.readFile(filename)
          .then(contents => {
            res.setHeader("Content-Type", filename.endsWith(".css") ? "text/css" : (filename.endsWith(".js") ? "text/javascript" : "text/html"));
            res.writeHead(200);
            res.end(contents);
          })
          .catch(err => {
            return;
          });
      });
      
      // start listening http requests on port 8080
      this.server.listen(8080, 'localhost', () => {
          APP.log('Server is running on http://localhost:8080');
        });
    }
    
    detach(){
      APP.log("Closing server");
      this.server.close();
    }
    
    // file was renamed, update iframe
    onRenameBuffer( buffer ){
      if( buffer == this.buffer ){
        this.DOM.src = prefix + this.buffer.path;
      }
    }
    
    readFile(filePath, mode)
    {
      return window.require("fs").readFileSync(filePath, mode);
    }
    
    saveFile(filePath, data)
    {
      window.require("fs").writeFileSync(filePath, data);
    }
    
    constructor( frame, buffer ){
      this.buffer = buffer;
      this.DOM = DOC.create( frame, "iframe", {
        className:"CustomEditorView",
        src: prefix + buffer.path,
        style:{
          border: "0px none",
          width: "100%",
          height: "100%",
          margin: 0,
          position: "absolute"
        },
        load:function(){
        }
      });
    }
  }
  
  APP.add(new class CustomEditor{
    
    customEditorInstalled(){ return true; }
    
    pollViewForBuffer( buffer, vf ){
      if( extensions.indexOf(buffer.type) != -1 && vf.priority < 2 ){
        vf.view = CustomEditorView;
        vf.priority = 2;
      }
    }
    
  }());
  
}

And here are the accompanying files for the bytebeat generator ā€œeditorā€ (these go in ā€˜editorā€™ folder)

editor.html
<!DOCTYPE html>
<html>
<head>
  <script>
    function startBytebeatGenerator() {
      const context = new AudioContext();
      context.audioWorklet.addModule('./bytebeat-generator.js')
        .then(() => {
          const bytebeatGenerator = new AudioWorkletNode(context, 'BytebeatGenerator');
          bytebeatGenerator.connect(context.destination);
        })
        .catch(err => {
            console.log(err);
            return;
        });
    }
  </script>
</head>
<body>
  <button onclick="startBytebeatGenerator()">Start Bytebeat Generator</button>
</body>
</html>
bytebeat-generator.js
class BytebeatGenerator extends AudioWorkletProcessor {
  constructor(options) {
    super(options);
    this.t = 0;
    this.rate = 8000/sampleRate;
  }
  
  process(inputs, outputs, parameters) {
    const output = outputs[0];
    const amplitude = parameters.amplitude;
    for (let channel = 0; channel < output.length; ++channel) {
      const outputChannel = output[channel];
      for (let i = 0; i < outputChannel.length; ++i) {
        const t = this.t | 0;
        this.t += this.rate;
        const o = ((t)>>4) | (t>>5) | t;
        outputChannel[i] = ((o&255) - 128) / 256;
      }
    }

    return true;
  }
}

registerProcessor('BytebeatGenerator', BytebeatGenerator);
7 Likes