Husserl tutorial series(11). Applying gen to MIDI and real-world cases
This simpler tutorial moves from gen~ and JavaScript to gen. It compares event processing to audio processing (with a more limited abstract discussion than in tutorials 1 and 8, which introduced gen~ and JavaScript). It then considers some real-world design issues. All tutorials in this series:
Designing a good LFO in gen~ Codebox: https://cycling74.com/forums/gen~-codebox-tutorial-oscillators-part-one
Resampling: when Average is Better: https://cycling74.com/forums/gen~-codebox-tutorial-oscillators-part-2
Wavetables and Wavesets: https://cycling74.com/forums/gen~-codebox-tutorial-oscillators-part-3
Anti-Aliasing Oscillators: https://cycling74.com/forums/husserl-tutorial-series-part-4-anti-aliasing-oscillators
Implementing Multiphony in Max: https://cycling74.com/forums/implementing-multiphony-in-max
Envelope Followers, Limiters, and Compressors: https://cycling74.com/forums/husserl-tutorial-series-part-6-envelope-followers-limiting-and-compression
Repeating ADSR Envelope in gen~: https://cycling74.com/forums/husserl-tutorials-part-7-repeating-adsr-envelope-in-gen~
JavaScript: the Oddest Programming Language: https://cycling74.com/forums/husserl-tutorial-series-javascript-part-one
JavaScript for the UI, and JSUI:<a href="https://cycling74.com/forums/husserl-tutorial-9-javascript-for-the-ui-and-jsui"> https://cycling74.com/forums/husserl-tutorial-9-javascript-for-the-ui-and-jsui
Programming pattrstorage with JavaScript: https://cycling74.com/forums/husserl-tutorial-series-programming-pattrstorage-with-javascript
Applying gen to MIDI and real-world cases. https://cycling74.com/forums/husserl-tutorial-series-11-applying-gen-to-midi-and-real-world-cases
Custom Voice Allocation. https://cycling74.com/forums/husserl-tutorial-series-12-custom-voice-allocation
gen versus gen~: circuit versus event-based simulators
Electrical and electronic simulation were the precursors of the Max modeling method. Electrical simulation, now almost universally performed with SPICE, is the most important tool for the design analog electrical circuits. It works on a 'SPICE DECK' that contains a set of low-level functional models of individual components, such as resitors, capacitors, inductors, and transistors. The models are in the form of equations, which are evaluated all at once for the entire circuit, producing a new set of values that are fed into the same model. As in Max, SPICE simulation can be in the time or the frequency domain. In Max, time-domina processing is more common, but it uses the same modeling system, just as SPICE does, for frequency-domain calculations, available via the pfft~ objects (the pfft~ objects work like a poly~ object, but instead of providing multiple instances that work concurrently over time, each instance processes a frequency band in pfft~).
Circuit simulators have historically been limited to networks componing less than 300 transistors, and as computers have improved in performance, the model quality has improved rather than the capacity. For larger circuits, such as now very common in the digital world, engineers instead use 'event-based' simulators. In the most abstract form, event-based simulators work on functional models of entire digital integrated circuits. In more detailed modeling, event-based simulators work on abstract models of digital 'gates' which are essentially binary in nature, although better simulators provide a range of 'unknown' states (such as unknown but high impedance, high or low but impedance unknown, etc). However abstract or detailed the model, the simulators contain a 'scheduler,' like Max does, with a 'timewheel' for upcoming events. The timewheel may have nanosecond resolution or model a digital system's clock rate, but in all cases, events are 'queued' for processing in 'timeslots' by a 'scheduler.'
Max's scheduler works in exactly the same way. The significant differences between the two modeling types, like for gen~ and abstract objects, are design size, speed, and the ability to work in the frequency domain. Like SPICE, gen~ has lower capacity, strictly limited by the system clock speed; it provides higher resolution at a lower rate; and it can work in the frequency domain. This is mostly because the entire circuit is be evaluated every tick, whereas for event-based simulation, the model size can be as large as the computer can support, and the events which it generates do not need to be completed within a strict time window.
When Cycling74 added gen, it added the ability to compile objects that work on the Max scheduler. From the above, it is clear that this ability is most suitable for complex designs and large functions, because that's what event-based simulators do best. For simpler tasks, the built-in objects in Max already provide a plethora of abilities, but when there are a lot of equations to process, or a lot of variables to evaluate, then custom code in gen provides a better solution.
gen versus primary objects in Max
Here I'll illustrate with an example from Husserl. To reduce audio-rate processing, and parameter range adjustment for the control dials is performed at the top level. In most cases, it's a single addition and/or multiplication, so gen is superfluous for it. But for parameters that control timing, such as LFOs and envelope ADR settings, more complex maths is required.
Inside the subpatch below the envelopes are currently separate gen objects for each of the envelopes (the subpatches between the multislider and numboxes simply splits the multislider output and sets the numbox values, but the subpatch below the numboxes is much more complicated).
The reason for the large network is to stop propagation of unnecessary events. If an output isn't needed, the gen object still creates an event, so the [sel] objects on the gen outputs filter out unneeded events. All the events issued from this patch go to MIDI, as well as gen~, and there are two panel controls for each parameter: the sliders and the number boxes.
Note: An easy way to process all the values at once is to set the 'hot' attribute of the gen object. However that creates output of all values each time any one of its inputs change, for a total of 45 MIDI messages each time the displayed channel is changed, when there should be none.
So the network in the above subpatch sends a bang to the gen object's first input when any one value changes. That means the sample rate and multi inputs do not generate events when they change, which was the first step in reducing the number of duplicate events. Also, the multislider sends new values for all three ADR values, so changing it caused nine events with 'hot' enabled. This network only creates three events for each envelopes, reducing the number of unnecessary events by an order of magnitude.
Now as for the gen object itself:
A primary problem with precalculating and caching timing parameters is that the values change when the sample rate changes. Upon receiving any input, the gen objects therefore check whether the sample rate changed and update the values for the ADR parameters in all 128 preset slots via a simple but enormously powerful loop statement.
Also, to reduce calculations after initializing, the 128 possible ADR values are recalculated and stored in a Data() array. Additionally, I have to restore the panel values when the displayed channel changes. For that, instead of performing the complex envelope duration calculation in reverse, I cache the values of the dials directly in the programs buffer too. When the channel changes, JavaScript then looks up the cached values in the programs buffer and sets the dials to those values.
Finally, I had an enormous amount of events when the system started up, so when the sample rate message is received, it doesn't do anything. A semaphore, as previously described, skips the big loop to update values instead of triggering it, because the duration values have already been stored in the buffer~ and restored from a file on startup.
env1(val, idx, chan){ // envelope without hold
Buffer programs("programs");
Data curv(128);
programs.poke(val, idx, chan);
return curv.peek(val);
}
env2(val, idx, hold, chan){ // envelope with hold
Buffer programs("programs");
Data curv(128);
programs.poke(val, idx, chan);
if (hold >0) return curv.peek(127);
else return curv.peek(val);
}
Buffer programs("programs");
Data curv(128);
History srate, dinit;
multi = in6;
chan = -1;
x, y = 0;
if(in5 != srate){
srate = in5; //cache the srate value
for (i = 0; i < 128; i +=1 ){ // calculate all 128 envelope timing values
curv.poke(1 / (srate * pow(1.12202, i * .7874 -70)), i);
}
if (dinit > 0){ // update the entire buffer if srate changes
for (k = 0; k <128; k += 1){
y = k * 160;
for (i = 0; i < 16; i += 1 ){
x = programs.peek(135 + multi, y); //e1a
x = curv.peek(x);
programs.poke(x, 2 + multi, y);
x = programs.peek(136 + multi, y); //e1d
x = curv.peek(x);
programs.poke(x, 3 + multi, y);
x = programs.peek(137 + multi, y); //e1r
x = curv.peek(x);
programs.poke(x, 70 + multi, y);
}
}
}
if (dinit == 0){ // filter out unwanted outputs
dinit = 1;
out1 = -1;
out2 = -1;
out3 = -1;
}else { // set ADR values
chan = programs.peek(0, 0);
out1 = env1(in1 , 135 +multi, chan);
out2 = env1(in2 , 136 +multi, chan);
out3 = env2(in3 , 137 +multi, in4, chan);
}Here's the subpatch if you want to look inside it yourself.
Using the Hot attribute
The 'hot' attribute eliminates the needed for large networks, but all inputs cause events to be generated. For the LFO timing calculations, I currently am using the 'hot' attribute because one output changes if any of its five inputs change.
Then I had a problem that changing the display channel retriggered sending the values, which depending on the order of events could cause the prior channel's values to appear in the current channel. So the caching of the panel values is in primary objects outside the gen! object. Inside the gen objects the code is rather simple, but the timing calculations use pow() and divisions, so I wanted to avoid putting them in the gen~ audio-rate processing, and the CPU-intensive activities thus only occur on panel changes:
lfoFreq(in1){
out1 = pow(in1 * .05, 1.5) +.01;
return int(out1 *100) *.01;
}
//mode = in1;
//bpm = in2;
//beats = in3;
//bars = in4;
//dial = in5;
if (in2 >0)
out1 = (in1 * in4) / (in3 * 60);
else
out1 = lfoFreq(in5);In this case I let the final calculation of the LFO frequency (multiplication by 1/samplerate) happen in gen~, after the huge design I made for myself with the envelopes above. Eventually I really should recode this to include that multiply operation too, in which case it will also need to update all the cached lfo frequency values in buffer~ upon change of the sample rate.
Note. gen also provides 'parameter watching' and low-frequency gen~-style operation. While these options certainly have their uses, I don't use them in Husserl3 because they add to CPU usage.
Dynamics Attack, Release, and Threshold
The last cases of timing-related parameters are for the two dynamics blocks, one for the channels and voice parameters, and one for the output. The channel and voice parameters also need to be updated when the sample rate changes, but currently the pokes of the panel default values are outside the gen object as in the example above.
But the gen code is more similar to the envelope example:
mstoincr(in1, srate){
return 1 / (in1 * .001 * srate);
}
Buffer programs("programs");
History srate, dinit;
x, y = 0;
if (in5 != srate){
srate = in5;
if (dinit > 0){
for (k = 0; k <128; k += 1){
y = k * 160;
for (i = 0; i <16; i +=1){
x = programs.peek(152, y); // limatt
programs.poke(mstoincr(x, srate), 107, y);
x = programs.peek(153, i); // limrel
programs.poke(mstoincr(x, srate), 108, y);
}
}
}
}
if (dinit == 0){
dinit = 1;
out1 = -1000;
out2 = -1000;
out3 = -1000;
} else {
out1 = mstoincr(in1, srate); // limatt
out2 = mstoincr(in2, srate); // limrel
out3 = dbtoa(in3); //limthresh
out4 = dbtoa(in4); //limlvl
}By contrast, the global dynamics doesn't need to be cached across 16 channels, because it's one set of controls that change per multi, so they are simply create normal gen~ param values, making the 'hot' attribute useful again:
And it's internal code is extremely simple:
But it would still require a lot of Max objects to do the math, and I'd already written the equations for the channel/voice dynamics, so this was purely a case of being lazy after doing all the MIDI and panel changing code, which was an immense amount of work.
Gen for MIDI
I introduced the use of massive selector statements in tutorial 5. In Husserl2 I put all the MIDI objects in a subpatch so they could be disabled with control if not needed to reduce CPU, but in Husserl3 I assumed people would always be using MIDI so I put the objects where they were needed, which really simplified the top-level network. For MIDI in, there is a currently a simple input to JavaScript:
But this is misleading in appearance, because the JavaScript just updates the panel display if the MIDI controller is for the currently displayed channel. The gen object sets the actual MIDI data in the programs buffer~ directly, Subsequently, if the displayed channel changes to one receiving MIDI data, the display values are fetched from the buffer~, so this method assured low latency while enabling MIDI changes to be displayed for all 16 channels. As mentioned in tutorial 9, I ran out of controller CC numbers and had to bit pack some fields, so the gen object also performs some bit decoding; and when values are received for parameters that influence each other, such as the LFO frequency shown above, the gen object fetches the other current values from the buffer and recalculates them. I already showed example of both of those requirements, so for the MIDI I/o code listings I truncate them to show the primary structure:
lfoFreq(bpm, mode, beats, bars, freq){
x = 0;
if (mode >0){
return (bpm * bars) / (beats * 60);
}
else {
x = pow(freq * .05, 1.5) +.01;
return int(x *100) *.01;
}
}
Buffer programs("programs");
Data envCurv(128);
History dinit, sust, hold, srate,
l1mode, l2mode, l1beats, l2beats, l1bars, l2bars,
l1f, l2f, rpnlo(-1), rpnhi(-1), bsenslo, bsenshi(2);
if(change(in4) != 0 || dinit==0){ //envelope curve
for (i = 0; i < 128; i +=1 ){
envCurv.poke(1 / (srate * pow(1.12202, i * .7874 -70)), i);
}
dinit = 1;
}
ccin = in2;
x = 0;
val = -999;
val = selector(in2,
min(in1, 0, 100), // 1 modwheel
in1, // 2 e1a,
in1, // 3 e1d,
min( in1 *.01, 1), // 4 e1s,
//...and so on...
in1 // 127 vel2y
);
idx = in2;
chan = in3 -1;
multi = in5;
if(in2==2){ // caching origiinal param values as well as timing
programs.poke(in1, 135 +multi, chan); // e1a
val = envCurv.peek(in1);
// and so on...
}
if (val > -999){
programs.poke(val, idx +multi, chan);
out3 = in3; //chan
out2 = ccin;
out1 = in1;
}else{
out1 = -999;
}MIDI output was simpler, because I did not need to clip out-of range values. When the panel controls send values to gen, their buffer indices are calculated as described in tutorial 9, and in parallel, the indices and parameter values are sent to gen object for scaling to MIDI output ranges:
On the right you can also see the MIDI port selector design which is almost identical to the example from MaxHelp, except the menus also have an option 'none' for disabling MIDI. Before the MIDI input you can also see logic to disable output while program changes are occurring. Inside the gen object, the same process for MIDI in pretty well happens in reverse:
Buffer programs("programs");
History dinit,
l1mode, l2mode, l1beats, l2beats, l1bars, l2bars,
rpnlo(-1), rpnhi(-1);
x = 0;
if (in3 == 0 ) {
out1 = -1;
} else {
out3 = in4;
out2 = selector(in1,
in2, // 1 modwheel
programs.peek(135, in4), // 2 e1a,
programs.peek(136, in4), // 3 e1d,
in2 *100, // 4 e1s,
//.. etc
in2 // 127 vel2y
);
out1 = in1;
}I had done all the above in primary-level Max objects before, and it's much easier to maintain in gen.
That's it for this tutorial. The next and last tutorial for Husserl3 will be describing the gen voice allocator, which does much more than the simple voice allocator objects provided as primary objects.
Happy patching )