Some questions about RNBO integration to a custom game/app.
Moin!
I've been reading over RNBO and developing some intuition for how it can be used effectively for a real-time application which uses miniaudio as its audio engine (but SoLoud is another option). I'm happy to provide more context if that helps the discussion, but here are my initial questions that hopefully some folks might have some input on:
When exporting to C++ and targeting a custom application and audio library:
If a rnbo patch (device?) is the source of a sound (footsteps, creature sfx), is it correct to say that I would instantiate the device for each entity associated with it?
All rnbo processing works on de-interleaved data correct?
My game loads all audio data using a custom format where all buffers are loaded as a single chunk of data, with a sample index that gives names to regions of the audio bank. It works with uncompressed and compressed audio, but I think using compressed audio wouldn't be possible here with RNBO because I can only create buffer references with uncompressed sample arrays as buffer inputs to a patch. Is this correct?
Is it possible to extend RNBO with custom objects? I can create a max external representing the spatialization system in the game and I can prototype an audio graph this way with my rnbo patches and so on, but that will always be a completely separate construction from the way the game works. (For context, I'm using steamaudio for spatialization).
Thanks for any feedback and helping me get a broader understanding of RNBO. :D
Hi Photex,
That's cool, please keep us in the loop as you work on this project, would be great to see how it comes together. I'm happy to give some answers, and point to some resources just in case you haven't seen them yet.
I've done some hacking with miniaudio, have not tried out SoLoud, but yes, you should be able to work with miniaudio and RNBO for this. To your particular questions:
- You would not necessarily need to create an instance of your RNBO device for each entity associated with it, I think this depends on your design. For example, perhaps you've designed a RNBO patch which emits the background music for your game, you might just have one of these. I could imagine a creature SFX device which is instantiated once for each creature, but I could also imagine some sort of polyphonic RNBO patcher that emits the sound for all of the creatures. This is up to you and your design.
- You can pass deinterleaved buffers to RNBO's process() method, as demonstrated in this guide, but process()is templatized in such a way that it should handle a variety of input and output formats. Check out the C++ API Reference for CoreObject::process(). And yes, you'll need to pass an array of decoded audio data to setExternalData(), as described in this guide. You should be able to point at your large single chunk of data and pass smaller chunks to RNBO.
- In terms of making RNBO objects -- RNBO is not extensible in this way yet, but this is being explored for a future version of RNBO
Best of luck, can't wait to see the game.
Hi Alex! Thanks.
This gives me some food for thought about approaches I might take. For example, rather than generate N individual sources for some category of sounds, maybe I should explore encoding/mixing them as an ambisonic bus which steamaudio can use.
The main tradeoff then might be that I might lose some of the scene based filtering that steamaudio is capable of.
Plenty to experiment with! :) As is often the problem with such flexible systems is determining the constraints to use as a scaffold.