Husserl tutorial series (5). Implementing Multiphony in Max

This is part of a series of tutorials on the Husserl3 design. All tutorials in this series:

1. Designing a good LFO in gen~ Codebox: https://cycling74.com/forums/gen~-codebox-tutorial-oscillators-part-one

2. Resampling: when Average is Better: https://cycling74.com/forums/gen~-codebox-tutorial-oscillators-part-2

3. Wavetables and Wavesets: https://cycling74.com/forums/gen~-codebox-tutorial-oscillators-part-3

4. Anti-Aliasing Oscillators: https://cycling74.com/forums/husserl-tutorial-series-part-4-anti-aliasing-oscillators

5. Implementing Multiphony in Max: https://cycling74.com/forums/implementing-multiphony-in-max

6. Envelope Followers, Limiters, and Compressors: https://cycling74.com/forums/husserl-tutorial-series-part-6-envelope-followers-limiting-and-compression

7. Repeating ADSR Envelope in gen~: https://cycling74.com/forums/husserl-tutorials-part-7-repeating-adsr-envelope-in-gen~

8. JavaScript: the Oddest Programming Language: https://cycling74.com/forums/husserl-tutorial-series-javascript-part-one

9. JavaScript for the UI, and JSUI:<a href="https://cycling74.com/forums/husserl-tutorial-9-javascript-for-the-ui-and-jsui"> https://cycling74.com/forums/husserl-tutorial-9-javascript-for-the-ui-and-jsui

10. Programming pattrstorage with JavaScript: https://cycling74.com/forums/husserl-tutorial-series-programming-pattrstorage-with-javascript

11. Applying gen to MIDI and real-world cases. https://cycling74.com/forums/husserl-tutorial-series-11-applying-gen-to-midi-and-real-world-cases

12. Custom Voice Allocation. https://cycling74.com/forums/husserl-tutorial-series-12-custom-voice-allocation

Implementing Multiphony in Max

This time it only took me a couple of hours to convert a polyphonic design into a multiphonic one. This tutorial describes how any number of voices can play different sounds set from any number of channels, with different parameters for each channel. I'll be describing how I'm running 16 MIDI channels with 128 parameters each on any of 32 poly~ voices, but the same technique generalizes to MC.gen~ with any number of voices and any number of parameters, within any available computational limitations. Also this tutorial contains a multiphonic voice allocator patch for MIDI and kslider notes.

• This successful multiphonic design was after three failed attempts, because there are serious performance issues in implementing multiphony with so many channels and parameters. It also took a lot of work on the prior attempts, but this time I got it working so fast I am still in shock how easy it was. So here's a tutorial on how you could easily enhance your designs the same way.

While the technique is easy, this is an advanced tutorial that requires at least some acquaintance with Max buffers, pattrstorage, zl objects, gen codebox, Max polyphony, and javascript.

Preparation
If you're wanting multiphony, really gen~ is the best option. It can do everything the msp~ audio processing objects can, and it is much faster. So first I got a polyphonic design working in a poly~ object in gen~., with all its params stored in pattrstorage.

That meant setting scripting names for all the panel controls, and using the same names for params in gen~, which was the majority of the effort. Using the same monicker for panel obects' scripting names and gen~ Params means for example you can send pattrstorage a 'dump' command and wire its output into gen~ to set Param values.

When I started on the naming thing, I didn't realize how much difference it would really make. Now I can use the same list of param names for javascript panel control, buffer peek/pokes, and midi i/o too. So it was worth the effort.

• So I've a standard naming convention of 3 characters: The first character is for the function ('e' for envelopes, 'f' for filters, etc), the second is a number for when I want more than one ('f1,' 'f2,' etc), and the third is for the function parameter ('a' for attack, 'd' for decay, 'q' for resonance, etc). Sometimes I need another character or two, and sometimes I get confused whether 'p' is pitch' or 'poles', but on the whole it's greatly reduced the time it takes me to design and debug, especially...the time I spend typing. I tried omitting the number when I thought Id never want more than one, but I very quickly learned that was a mistake, not only because Id later want to add a second one, but more importantly when I needed to start the name with two letters for the function instead of one, which got very confusing. That said, it takes a lot of effort to rename things, so I still have some names without numbers in them kicking around.

The Buffer~ object and CPU monitoring
To convert the polyphonic design to multiphony, first I created a buffer~ called 'programs' with 16 channels, and one sample for each param value, as below. When the patch is closed, the freebang message saves the buffer's state, which is then loaded again when the patch is reopened. This snip also shows how I measure for CPU peaks. When the CPU peak is over 90%, I reduce the number of playing channels automatically.

The same buffer can hold multiphonic presets, but it's advisable to arrange them as sequences within the 16 channels, viz., "buffer~ programs -1 16 @samps 12800" for 100 multiphonic presets of 128 params. Max chokes with somewhere around more than 64 channels per buffer, I don't know the exact channel limit, but I tried 32,768 channels in one buffer, and Max was really unhappy about that.

Making the Parameter List
For MIDI I/O and sending parameters to gen~, first I sent a dump message to pattrstorage to print all the script names and used the zl object to concatenate them into a messagebox. I edited the messagebox to create a list of all the param names.

• After transferring the values to the 'programs' buffer, I guess the pattrstorage is redundant in some ways, but it has other advantages, so I'll be keeping it for at least a while. If Ableton Live ever supports parallel processing, I'd probably switch it to using a Live! device and put presets in Live. )

Storing the Parameters in the Buffer
So one can send name/value tuples around with send and receive objects, but each one adds a buffer delay and an additional message in the message queue, so I had painstakingly wired all my panel controls directly to gen~.

• It will become obvious that's not so necessary as it was. Originally I had thought Id need to send gen~ parameters to all active voices individually for each note they played, but with this buffer implementation, only the panel needs to be set to new values.

As I had hardwired it all, I dragged all the wires from the poly~ object input to an 'unpack' object's second input. Then here's the clever bit. I convert the param names into buffer index offsets with a 'zl sub' object, as shown below. I poke all the param name/value tuples into the buffer channel indices like this....this just pokes it into channel 1, I haven't made the panel control to select different channels yet.

• So you might wonder why there are zeroes in the message box. I use the same list to generate MIDI CC messages, and the zeroes are for CC messages that gen~ doesn't need, for example, MIDI CC 64 is a sustain pedal, and in my design I change the envelope release parameter for that, so gen~ doesn't need it. I don't need to filter out the zeroes from reaching the peek object because those messages go somewhere else, but passing the indices through the right outlet of a 'sel 0' object is the simplest way to remove them if you need to.

The poly~ object
The poly~ is very simple. It simply sends a thispoly message to the gen~ object on load. For simpler designs, the loadbang for thispoly~ can be inside the poly~ object, but if the gen~ code is more than about 500 lines it doesn't compile fast enough to receive the thispoly message before it fires. So now I send it a loadbang from the top-level patch.

• More recently I put the loadbang through a 2 second pipe to delay it enough so the poly~ object finishes compiling.

• You can use mc_channel instead of a thispoly~ object if you're using mc.gen~, but I had a lot of crashes when I make coding errors in ms.gen~, and I don't have any with poly~, and besides it not being embedded inside the patch, MC is just a wrapper for gen~, so it doesn't really make any difference.

The gen~ codebox
The next stage of the conversion is to convert the gen~ object to read the 'programs' buffer instead of using params. When I got all the Param messages set up to work properly, I simply changed them all to History variables by changing 'Param' to History' in codebox.

Then each gen~ voice peeks the buffer object for its current channel and puts the values into the instantly created history variables. There's a reason to make them static History variables rather than simple variables: by interleaving the buffer peeks for each voice on a simple clock counting from 1 to voiceCount, I was finally able to get the Buffer reads done for all 32 poly~ voices without causing a huge processor peak, and only adding a 0.6ms latency @48khz:

At first that may not look very efficient, but in fact Max processes audio I/O in chunks set by the DSP settings, so if it's set to 128, for example, all 128 buffer values for each voice are loaded into the cache at the same time. Programmatically, it would be cleaner if buffer peeks could load more than one value at a time, but as gen~ is compiled, the performance is quite good. the important thing is to keep all the peeks sequential. If you load the values from the buffers where you want them. rather than storing them as variables, the performance is terrible.

MIDI I/O
For MIDI output, I use a similar technique to the 'zl sub' example above with the output of the pattrstorage object. For MIDI input, a 'zl lookup' object uses the same list as 'zl sub' to create the name/value tuples, which may easily be sent to the panel objects with pattrhub if they are for the currently displayed channel, and which are simply poked into the 'programs' buffer to change playing voices.

Updating the panel to show different channels
So the big problem was how to display the panel controls for each channel. When I was first learning Max, I tried putting all the params for each channel in 'zl reg' objects each poly~ object. That REALLY didn't work. Then I tried making 16 separate panels for each MIDI channel. Updating them from pattrstorage drove me crazy. I tried doing dumps from pattrstorage for each channel and sending them as messages each time a voice switched to a new channel, and the coding for that drove me crazy too.

But I finally worked out to keep it entirely separate from pattrstorage, which doesn't even know there are separate channels, and simply saves and loads presets for whatever channel is currently displayed. When changing the panel controls for a different channel, I simply use a 'for' loop to peek the buffer and send 'set' messages to all the panel controls from JavaScript, using the same parameter list that stuffs index lookup values into the 'zl sub' object above. Piece of cake!

• The only fiddly bit is that buffers are indexed from 0 in gen~ and 1 in javaScript, but once you get that sorted out, it's pretty easy to move array lists between gen~ and javascript.

• The javascript just 'sets' the panel controls, because the channel parameters are already set in gen~ from the same buffer, so the panel controls don't need to send new values on channel display change. And the javascript is just updating the display for a different channel, so it's actually good that the javascript runs on the low-priority thread.

Voice Allocator
After I made a gen voice allocator, Cycling74 made a new voice allocator which may have fixed the problems with the original one. However, I still need to store the MIDI channel numbers with each note and send messages appropriately. So I amended my polyphonic voice allocator to support multiple channels in a matter of minutes. Here is the patch containing the voice allocator.

It also contains velocity shaping described at the end.

• A problem occurred that gen~ cannot tell a new note has been received on the same voice with the same pitch and velocity as the last note, so this patch also creates a 'gate signal from a counter that increments with each note on event, to make sure the gate on message is unique. The allocator does not send note-off messages to voices that are not already on, so the gate off messages simply have the gate set to zero.

The gen voice allocator resets to the number of voices set by a negative value on the first input, or a note when a positive value is received on the first input. That simply avoided making it 'hot.' It also turns notes off on the attached kslider object when the available voice count is exceeded. the 'lastv' signal is used to update the display indicators from the last played voice on the currently visible channel, in javascript.

Buffer pch(), vel(), age(), chn();    // voice state buffers
History numv(0), vcnt(32), lastv(32), gcnt(1);
x, aoff = 0; 
chan = in3;
if (in1 < 0){                 // RESETS 
    for (i=1; i<=32; i+=1){
        vel.poke(0, i); 
        pch.poke(0, i);
        age.poke(i, i);
        chn.poke(0, i); 
    }
    numv     = 0;
    vcnt     = neg(in1);
    lastv     = vcnt;
}else if (in2 >0) {        // NOTE ON
    for(i=1;i<=vcnt; i+=1){
        x = age.peek(i);
        if(x == vcnt){
            if(numv==vcnt){
                if(vel.peek(i)>0){
                    out4 = pch.peek(i); 
                    //kb overflow update
                }
            }else{
                numv +=1;
                out4= 0;
            }
            chn.poke(in3, i);
            age.poke(1, i);
            lastv = i;    // last voice on
            out7 = in2;    // velocity
            gcnt += 1;
            out6 = in3;
            out3 = gcnt;    //gate out 0-1
            out2 = in1;     //fc out, Hz
            out1 = i;
            pch.poke(in1,i); 
            vel.poke(out7,i);
        }else{
            age.poke(x+1,i);
        }
    }
    p0  = in1;
} else if (in2 ==0){                    //NOTE OFF
    for(i=1;i<=vcnt; i+=1){
        if(pch.peek(i) ==in1 && chn.peek==in2){
            aoff = age.peek(i);
            pch.poke(0, i);
            vel.poke(0, i);
            chn.poke(0, i);
            for(j=1;j<=vcnt; j+=1){
                x = age.peek(j);
                if(x>aoff && x<=numv){
                    age.poke(x-1, j);
                }
            }
            age.poke(numv, i);
            numv -=1;        
            out6 = in3;
            out4 = 0;     //kb fix
            out3 = 0;     //vel 0
            out2 = in1;     //fr off
            out1 = i;    //voice off
            break;
        }
    }
}
out5 = lastv;

Result
The result has no performance problem. On my 4Ghz 6700k i7, which admittedly is still a pretty good cpu, it only adds 2% to peek all 128 params into gen~ with it running at 48kHz with 256/64 dsp buffer sizes and 32 channels. It was interleaving the buffer reads across clock cycles for each poly~ voice that really made the difference, because otherwise that 2% becomes 64%. It just means a 0.6ms latency between param updates on channels, which actually is faster than the params are typically updated in gen~, which I'm told happens every 256 clock cycles, although I don't know if that's actually dependent on the DSP settings.

• Footnote: Generating only one message from multiple-output objects
  By the way, some Max objects store more than one value in pattrstorage. If sending them to gen~ you have to split them up, but what I do is use this little design trick to split them up and also provide numbox controls for each of their parameters. This is for the pictctrl object, which I use as an overlay of jsui to display stuff behind it.

• So I use scripting names of the number boxes for the name/value tuples rather than the pictrl object's value strings. The reason for the weird loopback and set messages is so that the left-hand number box for each of the pictctrl objects only sends one message after any change to any of the panel control objects in the cluster. I use that one message from the left-hand number box to trigger jsui processing from the XY values, and jsui is computationally expensive, so it took me a long time to figure out that asymmetric arrangement of 'set' message loopbacks to make sure that jsui processing only happens once. So maybe you find that helpful. That also works with function objects and others that store more than one value.

I hope you enjoyed the tutorial. Happy patching to you all )