RNBO with Node for Max?
Hi
Does RNBO support Node for Max?
For example, if I'd create a patch that uses Node.js to obtain audio buffers, could I export that functionality with RNBO?
/Björn
to my knowledge javascript and node.js are supported within RNBO. Here is an article on how to use buffers in RNBO: https://rnbo.cycling74.com/learn/using-buffers maybe that can give you a starting point.
Hi Björn.
The [node.script] object itself is not available within RNBO and therefore not part of the export.
How much of the code and logic from your [node.script] can be reused will depend on the export target / host you are aiming for. If we assume that you'll be exporting to use your RNBO patch in a web browser (WASM) there might be some overlap to reuse some of the code in case it performs HTTP requests to obtain audio files etc.
Further details on how load buffer data into a RNBO device in the web context can be found here:
https://rnbo.cycling74.com/learn/buffers-and-file-dependencies
Florian
Thanks, Jan and Florian, for the pointers.
Node.js would actually be generating the audio buffers, with bindings to a Rust implementation.
/Björn
Hi Björn,
interesting! In that case I guess I don't fully grasp the context of the project yet. My understanding from your messages is that the filling of the audio buffers is basically something the host (in your case Max via [node.scrip]) provides.
Guess one option would be to check whether that could actually be done within RNBO itself. If not you might have a few options - the project you linked to is a Node implementation of the WebAudio API:
If you are intending to export RNBO to use your patch within a web browser the WebAudio API is basically available and you could run your code to generate the AudioBuffers and then load them into the RNBO Device.
If there is something in the Rust bindings that is not available natively within the browser you could in theory set up a thin API server layer that runs the code of your [node.script] to generate the audio data upon request and return it via an API call.
If you are intending to use RNBO within a different target export environment (eg. Raspberry Pi, Max external etc) there might be other options / ways to consider.
Hope that is helpful?
Florian
As an almost complete aside: are there any thoughts about making the RNBO patcher scriptable?
Hi Florian,
Currently I don't have a project in Max but am rather exploring, before doing anything hands-on, if Max + RNBO could be good platforms implementing what I'd like to try out.
It is correctly grasped, that filling the audio buffers with Max via [node.script], is what I'm wondering if is possible via an RNBO export. Furthermore, I'm mainly thinking about environments other than the web browser, as the project motivating the exploration is already implemented with the WebAudio API at synth.is. Obtaining the sounds rendered there in other contexts, such as in a VST, Max External, (native) mobile app, Raspberry Pi or Bela would be interesting.
An API layer for rendering the sounds could be an option, but preferably the sounds would be rendered in a self contained application - requiring e.g. a VST user to setup and run an API server would be a more complex user experience (and I may not have the resources to maintain a centralised service : )
/Björn
Hi Björn,
I think a good starting point would be to read up on how to fill buffers within RNBO itself. Here a few links:
Doing things within RNBO would mean that you can export the functionality to multiple targets without having to add a lot of complexity on your host application.
-florian
Thanks for the links Florian.
Those approaches seem to require the API server layer you mentioned, which leads to the question: Is it possible to perform network (API or even (web) socket) calls with RNBO to fetch data into buffers? Would it then be possible to fill the buffers incrementally, while playing, for realtime-ish performance?
/Björn
Hi Björn,
now I'm slightly confused. The links I shared would allow you to fill buffer objects within the RNBO patch itself so the logic would also be part of the exported RNBO patch regardless of the platform.
If you eg look at the [bufferop~] helpfile in RNBO you will see how a buffer can be filled using an expression language, the Max apply syntax etc. However, I can't really judge if and how that would map to your current implementation of generating buffer data with the WebAudio API .
To answer your questions - yes you can fill a buffer incrementally and at runtime from within RNBO. To learn more please refer to the Filling Buffers section in the docs and the [bufferop~] helpfile.
Here also a simple example of filling a buffer based on a parameter change
Wrt networking capabilities - RNBO does currently not have any networking objects available.
-florian
Thanks Florian,
Re-implementing the audio buffer generating logic from my project in RNBO - which is based on neural network activations for audio buffer sources, which are fed into web audio signal patches - might not be feasible, for a first iteration at least. That's why Node for Max seems appealing and motivated the original question.
Those NNs can actually be represented as a single function, and if it can easily be mapped to the expression language, then that part could be feasible, but the other part consists of a web audio patch, which is automatically generated (evolved), and I guess programmatically declaring patches is beyond the scope of Max / RNBO ? - initially I've considered Max / RNBO as consumers of that chain of audio buffer generation.
/Björn