[Wiki]Everything there is to know about LibAudio

What Is It?

As the name says, LibAudio is a new library for playing multi-channel sound on the Pokitto. It has been merged into PokittoLib, so feel free to check it out.

But why?

Pokitto::Display got its overhaul, now it’s Pokitto::Sound’s turn.
While Pokitto::Sound’s API was designed with compatibility in mind, LibAudio aims to increase performance, minimize flash and ram use, and to be extensible.

Specs

  • Hand-written assembly IRQ handler and buffer mixer
  • 1kb audio buffer (vs 8kb for Pokitto::Sound)
  • Multiple configurable channels

How do I get started?

Eventually, Pokitto::Sound would use LibAudio as a backend. For now, you can use LibAudio directly.

In My_settings.h:

// Not strictly necessary.
#define PROJ_AUD_FREQ 8000 // Only 'cause it's used in main.cpp
// #define PROJ_ENABLE_SOUND 0 // Nah 
// #define PROJ_ENABLE_SFX // don't need that
// #define PROJ_ENABLE_SD_MUSIC // nope

In main.cpp:

#include <Pokitto.h>
#include <LibAudio>

#include "Boop.h" // wav converted to C++

// Configure a LibAudio sink with 4 channels at 8khz
Audio::Sink<4, PROJ_AUD_FREQ> audio;

constexpr auto tune = SIMPLE_TUNE(C#/4, D/4, E/2).tempo(170);

void init(){
  // stream music from the SD card
  auto music = Audio::play("music.raw"); // streams are on channel 0 by default
  if(music) music->setLoop(false);

  // Play sound effect on channel 1
  Audio::play(Boop);

  // Play sound effect on channel 2
  Audio::play<2>(Boop);

  // Another stream on channel 3
  Audio::play<3>("ambience.raw");

  // play bytebeat on channel 3
  // Audio::play<3>([](uint32_t t)->uint8_t{ return (t>>4)|(t>>5)|t; });
  // ... on second thought, better leave that off.

  // create an note using one of the oscillators
  auto note = Audio::Note(random(0, 40))
                    .wave(1 + rand()%3)
                    .duration(1000)
                    .volume(random(127, 255));
  note.play(); // Put the synth on on channel 0, play note on oscillator 0
  // note.play<1>(2); // or Synth on channel 1, note on oscillator 2 of 3 (0, 1, 2)

  // play a simple tune on channel 0
  Audio::play(tune);
}

As you can see, you can mix multiple sound effects and streams together, so long as your game maintains the minimum framerate to keep the buffer full. For 8khz audio, you don’t want to go below 16 FPS (8000 / 512).


Sources

LibAudio comes with a few different sources, things that produce sound.
This is a description of each.

RAWFile

Raw, unsigned 8-bit audio data, streamed from the SD card using the File API.
You can give it a file path like this:
Audio::play("music/song.raw");

Or you can give it an already open file and tell it how many bytes it should play:

File music;
music.openRO("music/song.raw");
Audio::play(music, music.size());

SFX8

Plays raw, unsigned 8-bit audio data that is stored in flash.
Since it takes up so much space, it’s good for short sound effects like gunshots and explosions, but not for music or dialog.

You can convert your audio files using FemtoIDE:

  • Right-cick on a WAV or MP3 file in the file list
  • Click on “Convert to C”
  • Include the generated file in your code:
#include "boop.h"
Audio::play(boop);

ByteBeatSource

If you really, really want to, you can create procedural masterpieces using bytebeat.
Just give it a function that returns a sample for a given time t:

Audio::play([](uint32_t t)->uint8_t{ 
    return (t>>4)|(t>>5)|t; 
});

Keep in mind that ByteBeat does not mix well with other audio channels.


SynthSource

PokittoLib comes with a set of oscillators that can play all sorts of procedural sound effects.
Use the editor to create your sound effects, then play them using:

setOSC(&osc1, /* settings from editor here */);
Audio::play(osc1)

If you prefer a more verbose but easier-to-read style, you can use Audio::Note:

constexpr auto note = Audio::Note()
    .noteNumber(40) // Note 0 is B0
    .wave(WSQUARE)
    .attack(0)
    /* etc */
    ;
note.play();

SimpleTune

Not really a source, as it uses SynthSource as a back end.
This is a LibAudio extension for playing… tunes.
Single-channel. Single-instrument. No-frills. Nothing-fancy.
Easy-to-write. Easy-to-read. Quick-and-Simple. Tunes.

Want to play a fancy orchestra piece with violins, cellos, drums, organs, horns, that annoying cowbell, a 12-neck electric guitar, bagpipes, a theremin and a chorus of 50 kazoos?
This ain’t for that. There will be something else for that.

This is for the little victory jingle that plays when Mario reaches the castle. The ditty you hear when he picks up a 1UP mushroom. The “bop-ba-ding!” when he slams his head on a solid brick platform and gets a penny for his troubles.

It uses the PokittoLib’s Synth oscillators, so you can customize the sound and get quite a bit of variation. Notes are played first to last, until the end, at which point it stops. Automatic looping would be too fancy for SimpleTune. Each note takes up 2 bytes (one for the note, one for the duration).

constexpr auto tune = SIMPLE_TUNE(
    B0, // lowest possible note
    C#1, // notes are case-sensitive
    X,   // X marks a Pause
    D-1, // dash is for formatting, same as D1
    D#1,

    E3, // third octave
    E4, // go up one octave
    E,  // same note. Octave is optional, the default is 4.
    
    G#    , // whole note
    A-4/1 , // also a whole note
    A#4*1 , // another whole note
    B-4/2 , // half note
    C-4   , // explosive
    X/2   , // half pause
    D * 2 , // double note
    A / 4   // quarter note
);

Audio::play(tune);

By default, a tune plays at 120 BPM. You can change that like this:

constexpr auto tune = SIMPLE_TUNE(...).tempo(240);

You can also change the tempo in run-time like this:

auto& source = Audio::play(tune);
source.tempo(240);

The source also lets you customize other settings:

source.note().wave(WSAW);

Under the hood

If you want to extend/customize LibAudio, there are two concepts to be aware of:

  • Sinks: This is the thing that gets audio data and actually plays it. Audio::Sink will give you the appropriate sink for the platform you’re compiling for. Right now there is only a PokittoHWSink, but a PokittoSimSink is planned.
  • Sources: get data from somewhere and provide data to the sink they’re connected to, 512 bytes at a time. They should be platform-independent whenever possible.

Sources “provide data” in two different ways:
If they’re on channel zero, they’re expected to copy data into the given buffer, preferably using MemOps::copy.
Any other channel has to mix their data in using one of the Audio::mix functions:

namespace Audio {
// Mix the src buffer into the dst buffer
void mix(void* dst, const void* src, std::size_t count);
// Mix two individual samples
u8 mix(u8 a, u8 b);
}

For a simple example, have a look at the ByteBeatSource.

For an even simpler example, here is a source that lowers the volume of the channels below it (so if it’s on channel 2, it will lower channels 0, 1 but not 4):

void beJustABitQuieter(uint32_t lastChannel){
    Audio::connect(
        lastChannel,
        nullptr,
        +[](uint8_t *buffer, void *){
            for(int i=0; i<512; ++i) 
                buffer[i] >>= 1;
        }
    );
}

Note that sound effects are low-latency by default and insert themselves into the buffer that is currently being played. The function above would not affect that part of the audio.

5 Likes

If @HomineLudens and @Vampirics ‘Steam Driller’ is anything to go by, this sounds great.

I have a couple of questions:

// Not strictly necessary.
// If this isn't enabled the player can't change the volume.
#define PROJ_ENABLE_SOUND 1 
#define PROJ_AUD_FREQ 8000 // Only 'cause it's used in main.cpp

Do you mean they cannot change the sound prior to starting the game or within the game. The sound controls that are shown on startup are part of the bootloader aren’t they?

One thing to note: it is currently incompatible with PokittoCookie, unless you use it with:
#define PROJ_ENABLE_SOUND 0

If I set it to zero, the player cannot change the volume level but the cookies and sound work?

Will this be fixed or is the approach incompatible?

Are any of the following settings valid?

#define PROJ_HIGH_RAM HIGH_RAM_OFF
#define PROJ_ENABLE_SFX
#define PROJ_ENABLE_SD_MUSIC
#define PROJ_STREAM_LOOP 1
#define PROJ_SDFS_STREAMING
#define PROJ_FILE_STREAMING
#define PROJ_ENABLE_SYNTH 0

If you don’t enable sound you’ll still see the volume controls, but the component that regulates volume never gets initialized by the PokittoLib. It’ll just play at maximum, IIRC.

Correct, for now. A fix has already been made (as you can see in Steam Driller), it just hasn’t been pushed upstream yet.

  • PROJ_HIGH_RAM - Currently ignored, but I’ll add support for it.
  • PROJ_ENABLE_SFX - Ignored, unnecessary
  • PROJ_ENABLE_SD_MUSIC - Ignored, unnecessary
  • PROJ_STREAM_LOOP - Ignored, use setLoop(true|false)
  • PROJ_SDFS_STREAMING - Valid, used by the File lib
  • PROJ_FILE_STREAMING - Valid, used by the File lib
  • PROJ_ENABLE_SYNTH - Ignored, unnecessary
1 Like

Awesome … that clarifies a few things.

I’ll wait for the Steam Driller PR as my game uses Cookies.

1 Like

Post contents moved to first post.

6 Likes

Did someone say cowbells?

3 Likes

Explosive cowbell!

4 Likes

Is it possible to play a .raw file from memory (not the SD card)? When you convert the .wav file, what format is it in?

That’s what the sound effect source does, it reads a raw file from memory (generally flash memory) and plays it. Both are simply raw 8-bit unsigned audio data.