The journey of implementing voice chat via C++

Dev diaries
1

lately i’ve retaken up trying to create a multiplayer game, an FPS at that. and what FPS would be complete without voice chat, to banter with or trash talk your teammates and opponents alike?

though of course, an issue exists: the defold engine doesn’t have any way to get microphone input, nor does it have any way to support contunuous-stream audio.
at first i tried to use an existing extension: defold mic, however it cannot play back, and outputs in compressed wav which must manually replace a sound object’s audio, which results in unacceptable audio quality because doing so isn’t buffered, or fast enough.

so, i needed to make my own system. now, i’ve messed around with miniauidio before, which is a pretty great low-level barebones audio library, so i thought to myself to just shove that into the engine and call it a day. naturally, it took me a bit to translate a standalone executable test into an engine extention, but i did manage to get it to work, at first simply routing audio from the machine’s microphone to speakers.

once i had the basic groundwork down, i then built a pretty simple lua API to push and pull raw PCM data, and once that was done, i added the networking layer. in total, there’s 3 audio buffers in the whole system to help combat packet issues and timing desyncs. i like to think it’s fairly robust, though it is definetly not perfect. plus the API implementation itself is a bit crude, but it’s functional.

diagram of how the audio layers work
diagram of how the audio layers work1920×1080 91.3 KB

the audio itself is 8 bit 8000hz mono, which i’m aware is low quality, but this is to make things fast and easy on the server, and also for stylistic reasons as well. demonstrating it is the wonderful song from money island, a pirate i was meant to be

9 Likes
2

Nice work! Curious how you handled latency with the three buffers and the 8-bit format.

4

i kinda didn’t. the buffers are largely independent, so whatever is read is read. if the timing is off, chunks are either skipped or repeated. it’s all quite crude, really. but, it keeps the system pretty fast and simple to tweak.
the 8 bit audio is mostly because i don’t want to bother having to deal with endianess, so that saves both processing and coding time, as well as network bandwith.
at a worst case, the system will add roughly .2 seconds of latency, since there’s the first ring buffer, which runs at 100hz, which may be behind 1 tick, so 2/100ths, plus the network layer, which is looked at every frame, and will keep a 3 frame sample buffer, so 3/60ths, plus the networking layer can add up to 2 frames of latency (2/60ths), and finally the playback ring buffer keeps a few more samples in the buffer, at the same 100hz, for another 2/100ths of a second. worst case. there’s probably some flaws in my math but exact timing isn’t terribly important.

1 Like