Building a Synthesizer Editor with JavaScript, Part 3

After finishing up Part 2 of this series, I took a deep breath and got a good night's sleep. Then I rolled up my sleeves and got started on what I thought was going to be the straightforward task of incorporating System Exclusive communication into our editor-in-progress. I had good reasons to be optimistic: this whole project began because I had already made a quick-n-dirty sysex randomizer for the KORG minilogue xd, so I already had the basic framework. Right? Right?!!

In the end, it turned out to be a lot of additional work and that's great news for you, dear reader, because I'll have plenty to chat about in this, the final installment of "Let's Make a Synthesizer Editor". If you haven't read Parts 1 & 2, I highly encourage you to go back and start from the beginning -- we've covered a lot of ground already and I'll assume you've been paying attention!

What are System Exclusive Messages?

From the name alone, you can already tell that we're into elite MIDI territory: System Exclusive messages (or sysex for short) are MIDI messages with some device-specific format and meaning.

If you're a child of the 80s and 90s, you probably think of something like this when you think of sysex -- an unintelligible mess of tables filling up the last 20+ pages of the paper manual that came with your synth and why would you ever need something like that, anyway?

And indeed, that's the way most manufacturers still present this information, if they publish it at all anymore. Since sysex is mostly used by people developing editor software, or Max nerds, synth developers can save paper (and effort) by providing less beautiful documentation to those users who specifically ask for it. Or they forego the documentation entirely. But don't get me started.

System exclusive messages begin with a header byte (0xF0), then some value(s) identifying what kind of device the message is for, and then generally some values describing what kind of message is being sent (the instrument might know about 20 different kinds of sysex messages, for instance). Then come a bunch of data bytes -- how many and what they mean is determined, you guessed it, by the instrument. Some manufacturers like to finish up with a checksum -- that's a number derived from the bytes preceding it, generally according to some more or less poorly documented math -- which is used to insure the integrity of the data. And then there's a footer byte (0xF7).

Here's a pretty famous sysex message (as they go), which happens to be one of the simplest:

0xF0 (Header byte: here comes sysex!)
0x43 (Manufacturer ID)
0x20 (Substatus, in this case it means "some kind of a request")
0x00 (Data, in this case it means "voice edit buffer")
0xF7 (Footer, there goes sysex!)

That's how you request the voice currently being edited on a Yamaha DX7. And how did I know that? From this delightful bit of documentation:

If you look beyond the binary notation, it's all laid out fairly clearly, just as I wrote it above (including the handy hexadecimal $xx notation). You now know how to get 4 different kinds of voice data from that Yamaha DX7II you've been using as a controller keyboard (or a table).

But requesting the data is the easy part. What's more difficult, and what we're going to be looking at in some detail in this article, is how to parse the data you get back into a form you can use. And then, how to repackage whatever changes you've made to the data inside of Max into a valid sysex message to send back to the machine. It's analogous to knowing how to ask "where is the toilet?" in Japanese -- the answer isn't going to solve your problem unless you know enough of the language to understand it.

Luckily, KORG (which just happens to be Japanese) has provided us with all of the information we need to request, parse and transmit System Exclusive data from and to the minilogue xd. Here's the analogous documentation for the minilogue xd:

Check it out! It's not so different from the DX7 request message:

0xF0 (Header)
0x42 (Korg Manufacturer ID)
0x3G (minilogue xd)
0x00 (Data start)
.
.
.
0x51 (Data end)
0x10 (Current Program Data Dump Request)
0xF7 (Footer)

If you send this to your minilogue xd, like this (Max converts the hex numbers into decimal numbers inside of a box)...

...your minilogue xd will respond with a long message (1179 bytes long, to be precise). You can use the Max 'sysexin' object, or a MIDI Monitor application to see it arrive.

Here's what the documentation tells us about this message:

There's some stuff to unpack here, but one of the first things you might notice is that it's wrong. The data size of a 1179 byte message with 7 header bytes and one footer byte is 1171 bytes and not 384. This is a harmless typo, but the first rule of sysex documentation, as in nuclear disarmament, remains:

Trust, but verify.

There are a number of minor inaccuracies throughout the documentation, and finding them is just part of the work.

More importantly, the description refers to a conversion of "7bit" to "8bit" data, and this is where we'll begin.

System Exclusive data has the same 7-bit limitation of all MIDI data (the status bytes -- the ones which tell us that we're about to get Note data, or Controller data, or, yes, System Exclusive data -- use the 8th bit). A sysex message looks like: 0xF0 ... data ... 0xF7, and all of that data is restricted to 7-bit values. The block of data that KORG describes in Table 2 (PROGRAM PARAMETER), which is the data we are receiving, uses 8-bit bytes. Before we can interpret the 7-bit data packet we received from the instrument, we need to convert it back into 8-bit data.

KORG ASCII-arts the conversion method just above Table 1 (GLOBAL PARAMETER) in the docs. And because it's so charmingly unintuitive, here it is:

Clear as mud? It's trying to tell us that every 7 bytes of 8-bit data (56 bits total) will be converted into 8 bytes of 7-bit data by putting all of the high bits into the first byte of those 8 output bytes. Here's my own attempt at some ASCII art.

That means that:

byte 0: AAAA aaaa
byte 1: BBBB bbbb
byte 2: CCCC cccc
byte 3: DDDD dddd
byte 4: EEEE eeee
byte 5: FFFF ffff
byte 6: GGGG gggg

becomes for transmission via MIDI:

byte 0: 0GFE DCBA // bytes 6 - 0 above, 8th (leftmost) bits
byte 1: 0AAA aaaa // byte 0 from above, 7 remaining bits
byte 2: 0BBB bbbb // byte 1 from above, 7 remaining bits
byte 3: 0CCC cccc // etc.
byte 4: 0DDD dddd
byte 5: 0EEE eeee
byte 6: 0FFF ffff
byte 7: 0GGG gggg

and the reverse transformation has to be applied when receiving the data. The 56 bits remain intact, they've just been juggled a bit to fit the restrictions of the MIDI protocol. This technique, or some variation thereof, is used by a few manufacturers (Dave Smith/Sequential comes to mind).

If you've been paying attention during this series, you know what I'm going to say. This is a job for JavaScript if there ever was one. Let's do it.

Here's one way to perform that conversion (there are plenty of possible implementations):

function convert7to8bit(inputData) {
var convertedData = [];
var count = 0;
var highBits = 0;
for (var i = 0; i < inputData.length; i++) {
var pos = i % 8; // relative position in this group of 8 bytes
if (!pos) { // first byte
highBits = inputData[i];
}
else {
var highBit = highBits & (1 << (pos - 1));
highBit <<= (8 - pos); // shift it to the high bit
convertedData[count++] = inputData[i] | highBit;
}
}
return convertedData;
}

This determines the relative position of each incoming byte in its group of 8 bytes (a number from 0 - 7). For each byte position > 0, the correct bit is then grabbed from the byte in position 0 and put back as the high bit.

We call it like this:

function recv(b) {
if (b === 0xF0) { // new sysex
receiveBuffer = [b];
}
else if (b === 0xF7) { // end of sysex
receiveBuffer.push(b);
// slice off the 7-byte header and the 1-byte footer
var data = receiveBuffer.slice(7, receiveBuffer.length - 1);
var converted = convert7to8bit(data);
// do something with the converted buffer here...
var convertedBack = convert8to7bit(converted); // test
for (var i in data) {
if (data[i] !== convertedBack[i]) {
post("there is a mismatch at byte " + i + "\n");
}
}
receiveBuffer = [];
}
else if (b & 0x80) {
post("bad sysex byte, aborting receive\n");
receiveBuffer = [];
}
else if (receiveBuffer.length) { // data byte, append to buffer
receiveBuffer.push(b);
}
}

Sysex bytes arrive at the recv() function one at a time. When the header byte arrives, we initialize the global variable receiveBuffer and then fill that buffer with each arriving byte until the footer byte is read. At that point, we strip off the 7 header bytes and the footer byte and pass the data to convert7to8bit(), getting the converted 8-bit buffer back to work with.

In the code above, you can see that we then pass the converted buffer into the function convert8to7bit() and compare it to the original input. That's just for testing, to ensure that our conversion functions are working properly and symmetrically. We'll comment that out when we're sure everything is correct.

One of the simplest things we can do to ensure that we received the data we're expecting is to print the name of the patch. Examining the sysex spec (Table 2 in the implementation document), bytes 4-15 should contain ASCII data for the program name.

function printName(converted) {
var name = "";
for (var i = 4; i < 16; i++) {
name += String.fromCharCode(converted[i]);
}
post(name + "\n");
}

Does it work for you?

Integration

Alright, sysex reception is working. We have a valid data buffer. The next step is integration of the data in the buffer into our existing Model. For the most part, this is straightforward, but there are a few gotchas:

• Some of the data occupies more than 1 byte

• Some of the data occupies less than 1 byte, and shares the byte's other bits with other parameters

• Some of the data uses a different range than our Model

• Some of the data isn't currently represented in our Model

• Some of the data has a complex relationship to our Model

Let's start with the simplest case, an uncomplicated parameter which doesn't require any extra consideration:

{ name: "PORTAMENTO", cc: 5, min: 0, max: 127, syx: 17 },

Good old reliable portamento only needs a new object member, I'm calling it syx, to identify which byte is associated with this parameter in the sysex buffer. When we need to read the value of portamento from an incoming buffer, or when we want to modify the value before sending back out to the instrument, we'll be using byte 17 of the converted buffer.

But consider "VCO 1 LEVEL", which we used in Part 1. That's a 10-bit parameter, so its value won't fit into a single byte. Looking at the sysex spec, VCO 1 LEVEL is placed in bytes 54 and 55. So we can do something like this to describe that, adding a syxLen member:

{ name: "VCO 1 LEVEL", cc: 39, min: 0, max: 1023, syx: 54, syxLen: 2 },

In theory, we could accommodate any parameter length in this fashion. Practically, no single value occupies more than 2 bytes in the data buffer for the minilogue xd. Quick aside: the documentation claims that the high bits are in byte 54 and the low bits in byte 55, but the documentation is wrong -- it's the other way around.

The case of multiple parameters sharing a single byte doesn't come up much in the spec outside of some User Oscillator (Multi Engine) parameters and the Step Sequencer. In the interest of time and focus, the Step Sequencer isn't going to get much attention here (although I may come back to it for a Part 4 at some point, no promises!), so I'll skip over that case for now.

The problem of differing ranges is also rare, it only comes up three times. For example:

{ name: "MOD FX TYPE",
cc: 88, min: 0, max: 127, enum: [ ... ], syx: 89, syxOffset: 1 },

In the case of enumerated parameters, the enumeration index is stored in the sysex, rather than the CC value. The MOD FX TYPE enumerated value ranges from 0-4 in Max (representing "CHORUS", "ENSEMBLE", etc.), but 1-5 in the sysex buffer, an offset of 1. We can address this with the addition of syxOffset data member. When reading data out of the buffer, we need to subtract syxOffset (if present), and when writing to it we add syxOffset to the current (enumeration) value.

The case of data which has no representation in the Model can be solved simply by adding those parameters to the Model (in a special "sysex parameter" section). The new parameters are, for the most part, settings related to the Step Sequencer and Arpeggiator. In any case, these settings cannot be modified with a CC or NRPN message. The "OCTAVE" parameter, for instance, is sysex-only. You can find these parameters in minilogueXD_syx.js.

Finally, there's the case of complex mappings, which we'll address below.

This integrative work is all done, and if you take a quick peek at minilogueXD_cc.js and minilogueXD_nrpn.js, you'll see the sysex-related additions in the parameter arrays at the top of the files.

More Integration

How does this all fit together? When a sysex buffer from the instrument arrives, we first do all of the MIDI byte processing above (in the recv() function above), which yields a data buffer of valid 8-bit bytes. Then we can do something like this:

var syx = require("minilogueXD_syx");
var params = getAllParams();
for (var p in params) {
var param = params[p];
if (syx.getParameterValueFromInstrumentData(param)) {
param.setListenerValue();
}
}

For each parameter, getParameterValueFromInstrumentData() (a function we'll examine in a moment) is called and, if the parameter has a syx member —that is, if it is associated with a sysex value — the parameter's value member is updated with whatever was received from the instrument. Finally, we tell the parameter's listener that the value changed — just like when a CC or NRPN message arrives from the synth. This will update the View so that we can see the current instrument state.

And here's getParameterValueFromInstrumentData(), which is exported from the required file minilogueXD_syx.js:

function getParameterValueFromInstrumentData(param) {
var syx = param.syx;
if (syx >= 0) {
var val = DATAbytes[syx];
if (param.syxLen && param.syxLen > 1) { // can only be 2
val = val | (DATAbytes[syx + 1] << 8);
}
if (param.syxOffset) val -= param.syxOffset;
param.setValueFromSysex(val);
return true;
}
return false;
}

The syx member of our parameter model is already familiar. Note that we take the syxLen into account if it's present, placing the value of the second byte into the high 8 bits of the value. Finally, we apply syxOffset if it's present before setting the value using setValueFromSysex().

Even More Integration

OK, now we can address the final case listed above: a complex mapping from our Model to the sysex buffer. Do you remember way back in Part 1 -- I mentioned that a couple of the parameters serve double (or triple, or more) duty? For example, the "MOD FX SUB TYPE" parameter (that is, what kind of Chorus, Ensemble, etc. effect) is a single control in the interface and has a single CC parameter associated with it. But it is stored in five different places in the sysex data, depending on which MOD FX TYPE is currently selected!

We need a way to handle value changes coming from our interface which have a complex mapping to the sysex data. For that, I've extended the Model with a function called onChange(). The function will be called whenever the value changes (via the View or when setting the value from sysex), and is responsible for setting the actual value of the parameter, as well as handling as any special side effects. Here's the most minimal onChange(), which just sets the value:

onChange: function(newval) {
this.value = newval;
}

This is what will happen if a parameter doesn't have an onChange member, in fact.

And here's what we do for "MOD FX SUB TYPE":

onChange: function(newval) {
this.value = newval;
var param = getModFxTarget();
if (param) {
param.setValue(this.getValueForView());
}
}

In addition to setting the value, we get the parameter which corresponds to the current "MOD FX TYPE" value (that's what getModFxTarget() returns). For instance, the parameter "MOD FX ENSEMBLE", which is a sysex-only parameter. And then we set its value, too. This ensures that the state of those parameters remains in sync with changes we make at the View.

Now we have to take care of the reverse operation: sending the completed sysex data to the minilogue xd hardware. This is pretty straightforward, in fact: all we need to do is iterate all of the parameters in our model and, if they are associated with a sysex byte (or bytes), update the values in the sysex data buffer. Like this, in response to a 'push' message in Max:

function push() {
var params = getAllParams();
for (var p in params) {
var param = params[p];
if (syx.setInstrumentDataFromParameterValue(param)) {
;
}
}
var bytes = syx.getEditBuffer();
outlet(2, bytes);
}

After updating the values with setInstrumentDataFromParameterValue() (see below), we grab the edit buffer as an array of bytes and send it from the 'js' object's outlet. From there, it can be passed to the 'midiout' object and on to the hardware.

setInstrumentDataFromParameterValue() is the reverse operation of getParameterValueFromInstrumentData(), which we looked at above:

function setInstrumentDataFromParameterValue(param) {
var syx = param.syx;
if (syx >= 0) {
var val = param.getValueForSysex();
if (param.syxOffset) val += param.syxOffset;
DATAbytes[syx] = val & 0xFF;
if (param.syxLen && param.syxLen > 1) { // can only be 2
DATAbytes[syx + 1] = (val & 0xFF00) >> 8;
}
return true;
}
return false;
}

And getEditBuffer()? That gets us the bytes we need to send to the instrument to set the currently edited voice. The function slaps the sysex header (from the description in the documentation which we read above, called "CURRENT PROGRAM DATA DUMP") onto the 8-to-7-bit converted version of the internal data buffer, adds the sysex footer and returns the resulting Array object.

function getEditBuffer() {
var editBuffer = [ 0xF0, 0x42, 0x30, 0x00, 0x01, 0x51, 0x40 ];
editBuffer = editBuffer.concat(convert8to7bit(DATAbytes));
editBuffer.push(0xF7);
return editBuffer;
}

Save and Restore

Armed with all of that knowledge and code, it will be pretty easy to add a couple of last features to our software: I'd like to be able to save any presets I like to disk as a sysex file, so that I don't need to save them manually to the hardware memory — we could add sysex functionality to save our work to a particular memory location, but I'll leave that as an exercise for the passionate reader. Saving implies that we'll also need a way to read those files and send them to the minilogue xd if we want to use them later.

Max's JavaScript implementation gives us a simple way to work with files with the File object, but we need to use Max's 'opendialog' and 'savedialog' objects in order to open an OS file dialog. Like this:

Very straightforward, but note the finesse with strippath to ensure that the filename "sticks" between usages. The 'opendialog' case (in the 'p loadsyx' subpatcher in the main minilogueXD_Tutorial_Part3.maxpat patcher window) is even simpler.

'savesyx' is a message to 'js':

function savesyx(fname) {
if (fname) {
utils.writeSysex(fname, syx.getEditBuffer());
}
}

which calls into the exported writeSysex() function of the required minilogueXD_utils.js file with the edit buffer bytes as an additional argument:

exports.writeSysex = function(fname, bytes) {
var f = new File(fname, "write", "Midi");
if (f.isopen) {
f.writebytes(bytes);
f.eof = f.position;
f.close();
post("wrote sysex to " + fname + "\n");
return;
}
post("error writing sysex to " + fname + "\n");
}

Here, a new File object (f) is created with write permissions and type "Midi" (which ensures that the file being written has a .mid or .syx suffix). If it's open, we simply write the array of bytes to it. Then we set the eof (end of file) property to match the current file position and close the File object again. The eof technique is used to make sure that there are no extra bytes at the end of the file, if we are overwriting a file which already existed, and which was longer than the file currently being written.

Reading the sysex bytes back out of the file is similarly clean:

exports.readSysex = function(fname) {
var f = new File(fname, "read");
if (f.isopen) {
var a = f.readbytes(f.eof);
f.close();
if (a) {
post("read sysex from " + fname + "\n");
return a;
}
}
post("error reading sysex from " + fname + "\n");
return null;
}

This time, we create the File object with read permissions and, when it's open, read the bytes directly into a JavaScript Array object. readSysex() is called from the function 'loadsyx', a message to 'js':

function loadsyx(fname) {
if (fname) {
bytes = utils.readSysex(fname);
if (bytes) {
loadBytes(bytes);
outputBytes(bytes); // push to instrument, too
}
}
}

If bytes are returned from readSysex(), we first copy that byte array into our sysex data buffer and update our model (as if we had received a sysex block directly from the instrument). Then we output the bytes to the instrument.

We can now save our work to files on disk (and restore them later). Note that these files are not in the same format used by KORG's "minilogue xd Sound Librarian" software: hacking that file format is beyond the scope of this article. But for the motivated hacker, here's a hint: the program file saved by that software (with the .mnlgxdprog extension) is actually a zip archive and contains, in addition to a little bit of metadata, the 8-bit representation of the program data (1024 bytes).

If anyone takes up the challenge and adds .mnlgxdprog export (and import), I can't offer much more than some fame. But fame you will receive.

A Word About Testing

One very, very last thing: if you look near the end of minilogueXD_Tutorial_Part3.js, you might notice the function testreq(). This is a test function which I used to sanity check the sysex we're generating -- it helped me find and solve a number of problems along the way.

testreq() first gets the current edit buffer from the synth as a byte array (and updates all of the parameter values in our Model). Then it goes through every parameter and touches the value (that is, it re-sets the value to its current value using the setValue() function). Then it calls setInstrumentDataFromParameterValue() on every parameter (as in push(), above), updating the sysex data buffer from the current parameter values. And finally, it gets the updated sysex data buffer as a byte array and compares that to the original byte array we got from the synth. If they are identical, that's a pretty good indication that we haven't done anything wrong.

I mention it because, if you hadn't noticed, this stuff gets complicated quickly. Writing little tests, such as ensuring there aren't any parameter values which overflow the min/max values, will more than justify the additional effort the first time they lead you to discover a subtle problem in your code.

What's Still Missing? (redux)

Even with all of that work, our editor is still missing a few things which would be nice-to-haves in a future version. If I get around to adding them, I'll update the projects attached to this article, write another installment, and/or add a link to a Github repository or something like that:

• Global settings

• Step sequencer / motion sequencer

• User oscillator/effect (Multi Engine) settings

• User tunings

• .mnlgxdprog import/export

None of these are particularly challenging to model, although the Multi Engine settings would involve real-time changes to the UI which might be a burden. In any case, adding them was beyond the scope of our minimal viable patcher.

If you, dear reader, are inspired to add some of these yourself, please share your work so that we can all learn from it (or help solve any problems which arise).

What's Not To Like About JavaScript?

We've been able to accomplish an awful lot using the js object in Max, but it's not the answer to every problem in Max. In particular, the js object runs in the application's main (UI) thread, which means that you can't do audio-rate processing, nor audio-synchronized processing with any degree of accuracy. This is a totally adequate restriction for an editor, but might not be ideal if you want to put the editor into a sequencer (as a Max for Live device, for instance) and automate parameters.

If you want to use the js object to perform arbitrary processing outside of Max's UI thread, you might want to investigate Node for Max (new as of Max 8). Node for Max uses the most recent iteration of the ECMAScript -- JavaScript's real name -- language (ES6 -- 'js' uses the ES5 engine). To put it grandly (and vaguely), Node for Max is insanely powerful and permits you to use thousands of external libraries to accomplish whatever it is you want to do. But there's a catch: it doesn't have tight linkage to the Max patcher, you can't use it to perform patcher scripting, parameter listening and so on, so it wasn't appropriate for this task. It might be appropriate for many others, though.

A Final Deep Breath

We're pretty much done, I think it's safe to put your feet up and enjoy an appropriate beverage. We've gone over a significant portion of the most important concepts and code in this project over the course of this series. The finished-for-now version is included in the archive attached to this article, and is feature-complete (given our scope) and ready to use.

I encourage you to review the complete source code in the minilogueXD_Tutorial_Part3 folder, there are a lot of details which are worth taking a closer look at, starting with those parameters which have onChange() methods. In particular, you should familiarize yourself with the code paths which are exercised when data arrives (from View, from instrument CC/NRPN, from sysex) and is routed to its ultimate destination (the Model, the View, the instrument).

It would be worthwhile to quickly go through the general topics we've covered in this series, if for no other reason than to tickle the search engine algorithms. But we've covered a lot of ground, over nearly 40 pages in my text editor, and they say review is good for retention:

• How to read an instrument's MIDI specification

• Modelling of a data set in JavaScript (from simple to complex)

• Max's MIDI input and output objects

• Max support for NRPNs

• Laziness is a virtue

• Model-View-Controller concepts

• Using bitwise operators (<<, >>, |)

• JS Require

• Breaking complex projects into smaller units

• JS ParameterListener

• Attaching a View to a Model

• Generating Max objects using JS object scripting

• System Exclusive

• Never trust the spec

• 8- vs 7-bit data encoding

• Working with File objects in JS

• Testing is worthwhile

• A ton of programming techniques in JS which might inspire you in your JS (or other language) programming

The final lesson I'd like to impart is that building software (at least any software worth programming well), whether in Max, JS, C++ or whatever your weapon of choice, can and arguably should involve the kind of iterative process we've applied here. Using that method, we have gone from a simple value-setter qua randomizer to a full-blown editor in a pretty reasonable amount of time, tackling increasing, but manageable complexity with each iteration.

At each milestone, we had something we could use, and had a better idea of how to proceed for the next round. I already have a pretty good idea of how to start extending this project to add the "nice-to-haves" from above, and I'm sure many of you are already sharpening your hacking tools, too.

I hope this series has been helpful and would certainly appreciate any comments, corrections, complaints or criticism as they arise. I can't wait to see what you come up with.

Learn More: See all the articles in this series