Cephas Teom

Super excited about RNBO. Found it ridiculously simple to build a synth, build it the integrate into a web page. So many possibilities!

I've built a basic FM patch that sounds as you would expect when played in MAX. However, when I export it and included it in my webpage it sounds completely different - almost like the modulation is not happening and, instead we're hearing two tones.

Screenshot 2022-11-10 at 18.11.10.png

import { createDevice, MIDIEvent } from '@rnbo/js'

let rawPatcher, patcher;
let device, modi, harm;
let WAContext = window.AudioContext || window.webkitAudioContext;
let context = new WAContext();

window.addEventListener('click', () => context.resume() );

function playNote(pitch) {
    let midiChannel = 0;

    let noteOnMessage = [ 144 + midiChannel, pitch, 100];

    let noteOffMessage = [ 128 + midiChannel, pitch, 0 ];

    let midiPort = 0;
    let noteDurationMs = 250;

    let noteOnEvent = new MIDIEvent(context.currentTime, midiPort, noteOnMessage);
    let noteOffEvent = new MIDIEvent(context.currentTime * 1000 + noteDurationMs, midiPort, noteOffMessage);

    device.scheduleEvent(noteOnEvent);
    device.scheduleEvent(noteOffEvent);
}

const fm = async () => {
    rawPatcher = await fetch("/json/fm2.export.json");
    patcher = await rawPatcher.json();
    
    device = await createDevice({ context, patcher });
    modi = device.parametersById.get("modi");
    harm = device.parametersById.get("harm");

    device.node.connect(context.destination);
};

fm()

window.addEventListener('click', () => {
    harm.value = 2
    modi.value = 1
    playNote(60)
});

In Max, the same synth with a harmonicity of 2 and a mod index of 1 create an open, concordant tone. The web export sounds like a bell, as if you're hearing both the carrier and the modulator and they are not multiples of each other.

This may not be a bug - there may be something obvious that I'm doing wrong - but it's odd that the same synth behaves differently when compiled.

I've built a basic FM patch that sounds as you would expect when played in MAX. However, when I export it and included it in my webpage it sounds completely different - almost like the modulation is not happening and, instead we're hearing two tones.

Here's my RNBO patch:

Hi there!
Super excited about RNBO. Found it ridiculously simple to build a synth, build it the integrate into a web page. 

rnbo-fm-patch-export-to-web-results-in-a-different-sounding-synth

Hi there. Would you mind copying the patcher content compressed? (Select all the objects and then select copy compressed from the edit menu). 

This way it’s easier to reproduce without having to duplicate the patch from your screenshot.

Hi there. Would you mind copying the patcher  content compressed? (Select all the objects and then select copy compressed from the edit menu). 

This way it’s easier to reproduce without having to duplicate the patch from your screenshot.  

One debugging step would be to check the values of all of the parameters using for example

console.log(device.parametersById.get("harm").value);

for each of your parameters, just to check that they are all set correctly.

One debugging step would be to check the values of all of the parameters using for exampleconsole.log(device.parametersById.get("harm").value);
for each of your parameters, just to check that they are all set correctly.  

Florian - alas, I'm just trying out RNBO and haven't bought a license yet. So, it doesn't let me copy the code. If I do, I'll paste it here...

Is the synth polyphonic? Maybe there is a param that is not initialized correctly in some voices?

It is polyphonic yes. I've rebuild it as simple as possible to illustrate the issue - sorry I can't export more easily:

Screenshot 2022-11-10 at 20.44.48.png

Screenshot 2022-11-10 at 20.44.00.png

But when exported to web, same values sound completely different - unharmonious, despite the modulator being at the same frequency as the carrier!

One thing I notice is that you have an [out~ 0] (which I am surprised compiles). Does it change anything if you use [out~ 1] and [out~ 2] instead?

Changed and exported. Still the same issue. I wonder if the harm and modi inputs in JS are getting scaled somehow?

Even simpler - have removed any parameter inputs and just left the internal calculation:

Screenshot 2022-11-10 at 21.08.30.png

import { createDevice, MIDIEvent } from '@rnbo/js'

let device;
let WAContext = window.AudioContext || window.webkitAudioContext;
let context = new WAContext();

window.addEventListener('click', () => context.resume() );

function playNote(pitch) {
    let midiChannel = 0;

    let noteOnMessage = [ 144 + midiChannel, pitch, 100];

    let noteOffMessage = [ 128 + midiChannel, pitch, 0 ];

    let midiPort = 0;
    let noteDurationMs = 250;

    let noteOnEvent = new MIDIEvent(context.currentTime, midiPort, noteOnMessage);
    let noteOffEvent = new MIDIEvent(context.currentTime * 1000 + noteDurationMs, midiPort, noteOffMessage);

    device.scheduleEvent(noteOnEvent);
    device.scheduleEvent(noteOffEvent);
}

const fmSimple = async () => {
    const rawPatcher = await fetch("/json/fm-simple.export.json");
    const patcher = await rawPatcher.json();
    device = await createDevice({ context, patcher });
    device.node.connect(context.destination);
};

fmSimple()

window.addEventListener('click', () => {
    playNote(48)
});

It's as if the math being applied to the signal is incorrect.

import { createDevice, MIDIEvent } from '@rnbo/js'

let device;
let WAContext = window.AudioContext || window.webkitAudioContext;
let context = new WAContext();

window.addEventListener('click', () => context.resume() );

function playNote(pitch) {
	let midiChannel = 0;

	let noteOnMessage = [ 144 + midiChannel, pitch, 100];

	let noteOffMessage = [ 128 + midiChannel, pitch, 0 ];

	let midiPort = 0;
	let noteDurationMs = 250;

	let noteOnEvent = new MIDIEvent(context.currentTime, midiPort, noteOnMessage);
	let noteOffEvent = new MIDIEvent(context.currentTime * 1000 + noteDurationMs, midiPort, noteOffMessage);

	device.scheduleEvent(noteOnEvent);
	device.scheduleEvent(noteOffEvent);
}

const fmSimple = async () => {
    const rawPatcher = await fetch("/json/fm-simple.export.json");
    const patcher = await rawPatcher.json();
    device = await createDevice({ context, patcher });
    device.node.connect(context.destination);
};

fmSimple()

window.addEventListener('click', () => {
    playNote(48)
});

Just experimented with the Synth Building Blocks package and it behaves in the same way:

https://rnbo.cycling74.com/explore/synth-building-blocks-intro

Here's the exported FM Synth JSON file - no changes made to the patch:

ssb-fm.export.json

import { createDevice, MIDIEvent } from '@rnbo/js'

let device;
let WAContext = window.AudioContext || window.webkitAudioContext;
let context = new WAContext();

window.addEventListener('click', () => context.resume() );

function playNote(pitch) {
    let midiChannel = 0;

    let noteOnMessage = [ 144 + midiChannel, pitch, 100];

    let noteOffMessage = [ 128 + midiChannel, pitch, 0 ];

    let midiPort = 0;
    let noteDurationMs = 250;

    let noteOnEvent = new MIDIEvent(context.currentTime, midiPort, noteOnMessage);
    let noteOffEvent = new MIDIEvent(context.currentTime * 1000 + noteDurationMs, midiPort, noteOffMessage);

    device.scheduleEvent(noteOnEvent);
    device.scheduleEvent(noteOffEvent);
}

const fmSimple = async () => {
    const rawPatcher = await fetch("/json/ssb-fm.export.json");
    const patcher = await rawPatcher.json();
    device = await createDevice({ context, patcher });
    device.node.connect(context.destination);
};

fmSimple()

window.addEventListener('click', () => {
    device.parametersById.get("ratio").value = 3;
    device.parametersById.get("index").value = 5;
    playNote(48)
});

Again, odd behaviour when changing the values. For example, if you change the modulation index value, the pitch changes, rather than just the timbre.

Just experimented with the Synth Building Blocks package and it behaves in the same way:
https://rnbo.cycling74.com/explore/synth-building-blocks-intro 

import { createDevice, MIDIEvent } from '@rnbo/js'

let device;
let WAContext = window.AudioContext || window.webkitAudioContext;
let context = new WAContext();

window.addEventListener('click', () => context.resume() );

function playNote(pitch) {
	let midiChannel = 0;

	let noteOnMessage = [ 144 + midiChannel, pitch, 100];

	let noteOffMessage = [ 128 + midiChannel, pitch, 0 ];

	let midiPort = 0;
	let noteDurationMs = 250;

	let noteOnEvent = new MIDIEvent(context.currentTime, midiPort, noteOnMessage);
	let noteOffEvent = new MIDIEvent(context.currentTime * 1000 + noteDurationMs, midiPort, noteOffMessage);

	device.scheduleEvent(noteOnEvent);
	device.scheduleEvent(noteOffEvent);
}

const fmSimple = async () => {
    const rawPatcher = await fetch("/json/ssb-fm.export.json");
    const patcher = await rawPatcher.json();
    device = await createDevice({ context, patcher });
    device.node.connect(context.destination);
};

fmSimple()

window.addEventListener('click', () => {
    device.parametersById.get("ratio").value = 3;
    device.parametersById.get("index").value = 5;
    playNote(48)
});

if you look at the json, you can see it looks like you have some kind of per voice parameters setup

it'd be interesting to see the source code to see how this is actually being used...

have you tried exporting as max externals/vst... do you get a similar behaviour?

it would be useful to know if this is solely web export related issue or a more general code generation rnbo issue.

(it'd also be easier to 'debug' if it exists in the c++ code export)

(Ive been focusing on monophonic rnbo suff at the moment, so haven't been looking at much poly c++ code... though I did see one oddity early on when I tried the poly code export)

if you look at the json, you can see it looks like you have some kind of per voice parameters setup 

"paramId": "poly/1/index",
 "paramId": "poly/2/index",
 "paramId": "poly/3/index",
 "paramId": "poly/4/index",

as well as ..
"paramId": "index",
"paramId": "poly/index"



have you tried exporting as max externals/vst... do you get a similar behaviour?
it would be useful to know if this is solely web export related issue or a more general code generation rnbo issue.
(it'd also be easier to 'debug' if it exists in the c++ code export) 


(Ive been focusing on monophonic rnbo suff at the moment, so haven't been looking at much poly c++ code... though I did see one oddity early on when I tried the poly code export)

Still looking into this but as an aside - I assume you are exporting with 

enabled either on the [rnbo~] object or in the export configuration?

Still looking into this but as an aside - I assume you are exporting with per-voice parameters enabled either on the [rnbo~] object or in the export configuration?

Yes, that is correct. I've done it with and without to the same effect. Going to have a go exporting some even simpler patches with isolated operators and see if I can find the one that's at fault.

So, it's not to do with the polyphony or any of the parameter inputs. Here's a patch with a constant carrier frequency, harmonicity and mod index.

Screenshot 2022-11-11 at 20.09.02.png

max.mp3

wasm.mp3

I just recreated the patch in max/rnbo, and in Max, mine sounds exactly like the second example (which. you label web ) .. not the first.

I also exported the c++ code, and ran against my template/host , and it sounded identical.

so, from what I hear, your first example in max is the issue, not the second.

edit for those that want to try the rnbo patch... 


so, from what I hear, your first example in max is the issue, not the second.


edit for those that want to try the rnbo patch... 


Thank you for sharing this with us. I think there's at least a few things here that are worth greater investigation. 

In my local testing related to your minimal example with audio recording, a Max (not RNBO version) sounds like your first audio recording. A RNBO version running in Max sounds like your second audio recording (I believe the same as @technobear). I haven't had a chance to dig into the RNBO in Max vs RNBO as wasm in web browser yet, but there are some interesting things to note here, which may also relate to the RNBO in Max vs RNBO as wasm in browser.

First, there are possibilities of differing sounds with this type of dsp based on where the phase of the two oscillators are with respect to one another. The way in which that might work out might be dependent on a variety of factors like vector size, the ways in which various parameters might update differing parts of the graph (e.g. if multiple parameters are meant to be set simultaneously, but perhaps happen in two different vectors for some reason), the way in which phase is initialized, and or drift across differing oscillators.

Second, the cycle~ oscillator in gen and RNBO uses a combination of floating point and fixed point arithmetic in a different way than Max, and there could be aspects of this implementation that relates to these issues, especially when multiplying the output of these oscillators with very large numbers as you are doing in your patch. I've isolated a very simple example this is difference between RNBO and Max below. When initially turning on dsp there is minimal difference which noticeably grows over time when scaling with a large factor like 6600 or more like your example does. It could be that this is really pushing some of the boundaries of RNBO cycle's 32bit fixed point phase increment (as opposed to Max cycle's double precision floating point phase increment).

I can do some deeper investigations, but likely won't get much further until sometime later this week. In the meantime, perhaps it gives you and others some additional starting points to scrutinize. I would suggest further analysis by elimination, paying close attention phase, math precision and what results you are expecting. There of course could be other issues at play here, and only now that RNBO is released some very subtle issues for us to improve may come to light. Thank you for helping us track them down.

image.png

RNBO-fm-bug.zip

Thank you for sharing this with us. I think there's at least a few things here that are worth greater investigation.  
In my local testing related to your minimal example with audio recording, a Max (not RNBO version) sounds like your first audio recording. A RNBO version running in Max sounds like your second audio recording (I believe the same as @technobear). I haven't had a chance to dig into the RNBO in Max vs RNBO as wasm in web browser yet, but there are some interesting things to note here, which may also relate to the RNBO in Max vs RNBO as wasm in browser. 

First, there are possibilities of differing sounds with this type of dsp based on where the phase of the two oscillators are with respect to one another. The way in which that might work out might be dependent on a variety of factors like vector size, the ways in which various parameters might update differing parts of the graph (e.g. if multiple parameters are meant to be set simultaneously, but perhaps happen in two different vectors for some reason), the way in which phase is initialized, and or drift across differing oscillators. 

I can do some deeper investigations, but likely won't get much further until sometime later this week. In the meantime, perhaps it gives you and others some additional starting points to scrutinize. I would suggest further analysis by elimination, paying close attention phase, math precision and what results you are expecting. There of course could be other issues at play here, and only now that RNBO is released some very subtle issues for us to improve may come to light. Thank you for helping us track them down. 

ah, indeed... I assumed the OP was testing the rnbo patch, not a 'similar' patch written as max.

(thats comparing apple's n' oranges in my mind)

yes, if I use Max/MSP objects, I hear the first example.

but, if its Max RNBO or C++ RNBO then I hear the second example.

so it not web related at all (which was my suspicion) , rather RNBO vs Max.

in this case, as @Joshua says, either 32bit vs 64bit , 

or perhaps order of evaluation of initialisation or processing... this could cause phase differences.

(thats comparing apple's n' oranges in my mind)


yes, if I use Max/MSP objects, I hear the first example.
but, if its Max RNBO or C++ RNBO then I hear the second example.

so it not web related at all (which was my suspicion) , rather  RNBO vs Max.

in this case, as @Joshua says, either 32bit vs 64bit , 
or perhaps order of evaluation of initialisation or processing... this could cause phase differences.


Thanks so much for your considered replies here. Interesting that I only hear the unwanted outcome once I've exported the RNBO patch. Sounds fine when I'm using a RNBO patch within Max - which points to some additional environmental factors that could be affecting the signal processing.

It also makes sense that the multiplication by large values inherent in FM synthesis would expose small phase artefacts. I'll carry on investigating based on the suggestions here, and if I have any further insight, I'll share it.

Once again, thanks so much for your contributions and time.

Thanks so much for your considered replies here. Interesting that I only hear the unwanted outcome once I've exported the RNBO patch. Sounds fine when I'm using a RNBO patch within Max - which points to some additional environmental factors that could be affecting the signal processing. 

just a quick note that after further debugging we identified an issue with modulating cycle~ that led to the discrepancies you are experiencing between the Web / WASM Export and running the patch inside rnbo~ in Max or within other exports targets.

We have a fix for this that will be released with the next update to RNBO.

Thanks again for your report and helping us to track this down.

just a quick note that after further debugging we identified an issue with modulating cycle~ that led to the discrepancies you are experiencing between the Web / WASM Export and running the patch inside rnbo~ in Max or within other exports targets.

We have a fix for this that will be released with the next update to RNBO.
Thanks again for your report and helping us to track this down.

That's great news. I'll look forward to the update. I've been enjoying RNBO in the meantime. Thanks for such an excellent piece of technology.

If you haven't already, it may be worth checking the other oscillators in the same context. I swapped out cycle for 'saw', 'tri', and 'pulse'. Saw appeared to work as expected, whilst tri and pulse exhibited similar behaviour to cycle.

just a quick note that we have released RNBO v1.0.2 which includes a fix for this. If you still encounter issues, please don't hesitate to get in touch.

An equally quick note to say that it is working fabulously for the cycle~ and saw~ oscillators. But, frequency modulating tri~ or pulse~ encounters the same issue. Not something I do too often so no stress from here. But thought I'd let you know.

This is a little bit of a different issue, but yields similar results. The tri~ and rect~ objects in RNBO do not accept negative frequencies and should behave the same regardless of WASM or native C++ output. In Max these objects simply use the absolute value of the input frequency and we can do the same in an update. In the meantime, please use an abs operator in front of the frequency input and you should get results more to your expectations.

RNBO FM patch export to Web results in a different sounding synth