Accessing recorded buffer in RNBO Web Export device

Matthew Barile

Howdy,

I've been struggling trying to get my js code to interface correctly with my RNBO web export patch. My goal is to be able to record a sound into a buffer~ or data object (whichever is the right tool for the job) and then have js call the correct functions to access that DataBuffer, convert and download it into a WAV file.

Using the web interface (on Chrome) from my RNBO Web Export patches, I've successfully been able to use record~ to write audio into both objects (data and buffer~), and then play those back in via the web interface (using peek~ for the data object and a bang to a groove~ object associated with the buffer~). But I'm running into issues I believe with calling the right functions in JS in order to parse the data in the data or buffer~ objects and ultimately download it. Does anyone have any experience making this work?

Matt

Florian Demmer

Hi Matt,

do you have some stripped down code to share?

Looking at the reference it seems like using Device.releaseDataBuffer and operating on the returned DataBufferinstance could get you access to the data.

DataBuffer has a getAsAudioBuffer function that should return a WebAudio audiobuffer or you can access the raw binary data using the buffer property. Not sure what your encoding requirements are but using the former you should be able to find guidance on how to encode a WebAudio AudioBuffer as eg. a .wav file or use libraries that provide that functionality. I haven't looked too much in detail but a very rushed searched led to some results like this or this but I'm sure there is more.

Matthew Barile

Hi Florian, thank you so much for the suggestions. I tried working with Device.releaseDataBuffer but I might have missed the getAsAudioBuffer method... I might dig in a little more into that, but what I ended up doing as a workaround (and this may suffice as a long term solution for my needs regardless), is call the MediaCapture class to listen to my outs and take triggers from outports to know when to start and stop. This way, JS is handling all the recording and converting to WAV and I don't have to set this up in my RNBO patch. Here's what ended up working for me:

// Sets up the MediaRecorder to capture the RNBO device's output
    const setupMediaRecorder = (device, audioContext) => {
        const dest = audioContext.createMediaStreamDestination();
        device.node.connect(dest);
        
        mediaRecorder = new MediaRecorder(dest.stream);

        mediaRecorder.ondataavailable = (evt) => {
            recordedChunks.push(evt.data);
        };

        mediaRecorder.onstop = async () => {
            // Create a blob from the recorded chunks (likely in webm/opus format)
            const tempBlob = new Blob(recordedChunks, { type: 'audio/webm; codecs=opus' });
            
            // Decode the recorded blob back into raw audio data (an AudioBuffer)
            const arrayBuffer = await tempBlob.arrayBuffer();
            const audioBuffer = await audioContext.decodeAudioData(arrayBuffer);

            // Encode the raw AudioBuffer into a WAV file blob
            recordedBlob = encodeWAV(audioBuffer);
            
            downloadStatus.textContent = "Recording finished. Ready to download.";
            recordButton.classList.remove('recording');
            recordButton.textContent = "Record Audio";
            downloadButton.disabled = false;
            recordedChunks = [];
        };
    };

    // Listens for the message from the RNBO patch that tells us to stop recording
    const listenToOutports = (device) => {
        device.messageEvent.subscribe((ev) => {
            if (ev.tag === 'record_finished') {
                if (mediaRecorder && mediaRecorder.state === "recording") {
                    console.log("Received 'record_finished' trigger. Stopping recorder.");
                    mediaRecorder.stop();
                }
            }
        });
    };