Jeremy

After finishing up Part 2 of this series, I took a deep breath and got a good night's sleep. Then I rolled up my sleeves and got started on what I thought was going to be the straightforward task of incorporating System Exclusive communication into our editor-in-progress. I had good reasons to be optimistic: this whole project began because I had already made a quick-n-dirty sysex randomizer for the KORG minilogue xd, so I already had the basic framework. Right? Right?!!

In the end, it turned out to be a lot of additional work and that's great news for you, dear reader, because I'll have plenty to chat about in this, the final installment of "Let's Make a Synthesizer Editor". If you haven't read Parts 1 & 2, I highly encourage you to go back and start from the beginning -- we've covered a lot of ground already and I'll assume you've been paying attention!

3_minilogueXD_Tutorial_Part3.zip

From the name alone, you can already tell that we're into elite MIDI territory: System Exclusive messages (or 

 for short) are MIDI messages with some device-specific format and meaning.

If you're a child of the 80s and 90s, you probably think of something like this when you think of sysex -- an unintelligible mess of tables filling up the last 20+ pages of the paper manual that came with your synth and why would you ever need something like that, anyway?

image.png

And indeed, that's the way most manufacturers still present this information, if they publish it at all anymore. Since sysex is mostly used by people developing editor software, or Max nerds, synth developers can save paper (and effort) by providing less beautiful documentation to those users who specifically ask for it. Or they forego the documentation entirely. But don't get me started.

System exclusive messages begin with a header byte (0xF0), then some value(s) identifying what kind of device the message is for, and then generally some values describing what kind of message is being sent (the instrument might know about 20 different kinds of sysex messages, for instance). Then come a bunch of data bytes -- how many and what they mean is determined, you guessed it, by the instrument. Some manufacturers like to finish up with a checksum -- that's a number derived from the bytes preceding it, generally according to some more or less poorly documented math -- which is used to insure the integrity of the data. And then there's a footer byte (0xF7).

Here's a pretty famous sysex message (as they go), which happens to be one of the simplest:

0x20 (Substatus, in this case it means "some kind of a request")

0x00 (Data, in this case it means "voice edit buffer")

That's how you request the voice currently being edited on a Yamaha DX7. And how did I know that? From this delightful bit of documentation:

If you look beyond the binary notation, it's all laid out fairly clearly, just as I wrote it above (including the handy hexadecimal $xx notation). You now know how to get 4 different kinds of voice data from that Yamaha DX7II you've been using as a controller keyboard (or a table).

But requesting the data is the easy part. What's more difficult, and what we're going to be looking at in some detail in this article, is how to parse the data you get back into a form you can use. And then, how to repackage whatever changes you've made to the data inside of Max into a valid sysex message to send back to the machine. It's analogous to knowing how to ask "where is the toilet?" in Japanese -- the answer isn't going to solve your problem unless you know enough of the language to understand it.

Luckily, KORG (which just happens to be Japanese) has provided us with all of the information we need to request, parse and transmit System Exclusive data from and to the minilogue xd. Here's the analogous documentation for the minilogue xd:

Check it out! It's not so different from the DX7 request message:

If you send this to your minilogue xd, like this (Max converts the hex numbers into decimal numbers inside of a box)...

...your minilogue xd will respond with a long message (1179 bytes long, to be precise). You can use the Max 'sysexin' object, or a MIDI Monitor application to see it arrive.

Here's what the documentation tells us about this message:

There's some stuff to unpack here, but one of the first things you might notice is that 

. The data size of a 1179 byte message with 7 header bytes and one footer byte is 1171 bytes and not 384. This is a harmless typo, but the first rule of sysex documentation, as in nuclear disarmament, remains:

There are a number of minor inaccuracies throughout the documentation, and finding them is just part of the work.

More importantly, the description refers to a conversion of "7bit" to "8bit" data, and this is where we'll begin.

System Exclusive data has the same 7-bit limitation of all MIDI data (the 

 bytes -- the ones which tell us that we're about to get Note data, or Controller data, or, yes, System Exclusive data -- use the 8th bit). A sysex message looks like: 0xF0 ... data ... 0xF7, and all of that data is restricted to 7-bit values. The block of data that KORG describes in Table 2 (PROGRAM PARAMETER), which is the data we are receiving, uses 8-bit bytes. Before we can interpret the 7-bit data packet we received from the instrument, we need to convert it back into 8-bit data.

KORG ASCII-arts the conversion method just above Table 1 (GLOBAL PARAMETER) in the docs. And because it's so charmingly unintuitive, here it is:

Clear as mud? It's trying to tell us that every 7 bytes of 8-bit data (56 bits total) will be converted into 8 bytes of 7-bit data by putting all of the high bits into the first byte of those 8 output bytes. Here's my own attempt at some ASCII art.

byte 0: 0GFE DCBA // bytes 6 - 0 above, 8th (leftmost) bits

byte 1: 0AAA aaaa // byte 0 from above, 7 remaining bits

byte 2: 0BBB bbbb // byte 1 from above, 7 remaining bits

and the reverse transformation has to be applied when receiving the data. The 56 bits remain intact, they've just been juggled a bit to fit the restrictions of the MIDI protocol. This technique, or some variation thereof, is used by a few manufacturers (Dave Smith/Sequential comes to mind).

If you've been paying attention during this series, you know what I'm going to say. This is a job for JavaScript if there ever was one. Let's do it.

Here's one way to perform that conversion (there are plenty of possible implementations):

for (var i = 0; i < inputData.length; i++) {

var pos = i % 8; // relative position in this group of 8 bytes

var highBit = highBits & (1 << (pos - 1));

highBit <<= (8 - pos); // shift it to the high bit

convertedData[count++] = inputData[i] | highBit;

This determines the relative position of each incoming byte in its group of 8 bytes (a number from 0 - 7). For each byte position > 0, the correct bit is then grabbed from the byte in position 0 and put back as the high bit.

// slice off the 7-byte header and the 1-byte footer

var data = receiveBuffer.slice(7, receiveBuffer.length - 1);

// do something with the converted buffer here...

var convertedBack = convert8to7bit(converted); // test

post("there is a mismatch at byte " + i + "\n");

post("bad sysex byte, aborting receive\n");

else if (receiveBuffer.length) { // data byte, append to buffer

 function one at a time. When the header byte arrives, we initialize the global variable 

 and then fill that buffer with each arriving byte until the footer byte is read. At that point, we strip off the 7 header bytes and the footer byte and pass the data to 

, getting the converted 8-bit buffer back to work with.

In the code above, you can see that we then pass the converted buffer into the function 

 and compare it to the original input. That's just for testing, to ensure that our conversion functions are working properly and symmetrically. We'll comment that out when we're sure everything is correct.

One of the simplest things we can do to ensure that we received the data we're expecting is to print the name of the patch. Examining the sysex spec (Table 2 in the implementation document), bytes 4-15 should contain ASCII data for the program name.

name += String.fromCharCode(converted[i]);

Alright, sysex reception is working. We have a valid data buffer. The next step is integration of the data in the buffer into our existing Model. For the most part, this is straightforward, but there are a few gotchas:

Let's start with the simplest case, an uncomplicated parameter which doesn't require any extra consideration:

{ name: "PORTAMENTO", cc: 5, min: 0, max: 127, syx: 17 },

Good old reliable portamento only needs a new object member, I'm calling it 

, to identify which byte is associated with this parameter in the sysex buffer. When we need to read the value of portamento from an incoming buffer, or when we want to modify the value before sending back out to the instrument, we'll be using byte 17 of the converted buffer.

But consider "VCO 1 LEVEL", which we used in Part 1. That's a 10-bit parameter, so its value won't fit into a single byte. Looking at the sysex spec, VCO 1 LEVEL is placed in bytes 54 and 55. So we can do something like this to describe that, adding a 

{ name: "VCO 1 LEVEL", cc: 39, min: 0, max: 1023, syx: 54, syxLen: 2 },

In theory, we could accommodate any parameter length in this fashion. Practically, no single value occupies more than 2 bytes in the data buffer for the minilogue xd. Quick aside: the documentation claims that the high bits are in byte 54 and the low bits in byte 55, but the documentation is wrong -- it's the other way around.

The case of multiple parameters sharing a single byte doesn't come up much in the spec outside of some User Oscillator (Multi Engine) parameters and the Step Sequencer. In the interest of time and focus, the Step Sequencer isn't going to get much attention here (although I may come back to it for a Part 4 at some point, no promises!), so I'll skip over that case for now.

The problem of differing ranges is also rare, it only comes up three times. For example:

cc: 88, min: 0, max: 127, enum: [ ... ], syx: 89, syxOffset: 1 },

In the case of enumerated parameters, the enumeration index is stored in the sysex, rather than the CC value. The MOD FX TYPE enumerated value ranges from 0-4 in Max (representing "CHORUS", "ENSEMBLE", etc.), but 1-5 in the sysex buffer, an offset of 1. We can address this with the addition of 

 data member. When reading data out of the buffer, we need to subtract 

 (if present), and when writing to it we add 

The case of data which has no representation in the Model can be solved simply by adding those parameters to the Model (in a special "sysex parameter" section). The new parameters are, for the most part, settings related to the Step Sequencer and Arpeggiator. In any case, these settings cannot be modified with a CC or NRPN message. The "OCTAVE" parameter, for instance, is sysex-only. You can find these parameters in 

Finally, there's the case of complex mappings, which we'll address below.

This integrative work is all done, and if you take a quick peek at 

, you'll see the sysex-related additions in the parameter arrays at the top of the files.

How does this all fit together? When a sysex buffer from the instrument arrives, we first do all of the MIDI byte processing above (in the 

 function above), which yields a data buffer of valid 8-bit bytes. Then we can do something like this:

if (syx.getParameterValueFromInstrumentData(param)) {

 (a function we'll examine in a moment) is called and, if the parameter has a 

 member —that is, if it is associated with a sysex value — the parameter's 

 member is updated with whatever was received from the instrument. Finally, we tell the parameter's listener that the value changed — just like when a CC or NRPN message arrives from the synth. This will update the View so that we can see the current instrument state.

function getParameterValueFromInstrumentData(param) {

if (param.syxLen && param.syxLen > 1) { // can only be 2

if (param.syxOffset) val -= param.syxOffset;

 member of our parameter model is already familiar. Note that we take the 

 into account if it's present, placing the value of the second byte into the high 8 bits of the value. Finally, we apply 

 if it's present before setting the value using 

OK, now we can address the final case listed above: a complex mapping from our Model to the sysex buffer. Do you remember way back in Part 1 -- I mentioned that a couple of the parameters serve double (or triple, or more) duty? For example, the "MOD FX SUB TYPE" parameter (that is, what kind of Chorus, Ensemble, etc. effect) is a single control in the interface and has a single CC parameter associated with it. But it is stored in five different places in the sysex data, depending on which MOD FX TYPE is currently selected!

We need a way to handle value changes coming from our interface which have a complex mapping to the sysex data. For that, I've extended the Model with a function called 

. The function will be called whenever the value changes (via the View or when setting the value from sysex), and is responsible for setting the actual value of the parameter, as well as handling as any special side effects. Here's the most minimal 

This is what will happen if a parameter doesn't have an 

And here's what we do for "MOD FX SUB TYPE":

In addition to setting the value, we get the parameter which corresponds to the current "MOD FX TYPE" value (that's what 

 returns). For instance, the parameter "MOD FX ENSEMBLE", which is a sysex-only parameter. And then we set 

 value, too. This ensures that the state of those parameters remains in sync with changes we make at the View.

Now we have to take care of the reverse operation: sending the completed sysex data to the minilogue xd hardware. This is pretty straightforward, in fact: all we need to do is iterate all of the parameters in our model and, if they are associated with a sysex byte (or bytes), update the values in the sysex data buffer. Like this, in response to a 'push' message in Max:

if (syx.setInstrumentDataFromParameterValue(param)) {

 (see below), we grab the edit buffer as an array of bytes and send it from the 'js' object's outlet. From there, it can be passed to the 'midiout' object and on to the hardware.

function setInstrumentDataFromParameterValue(param) {

if (param.syxOffset) val += param.syxOffset;

DATAbytes[syx + 1] = (val & 0xFF00) >> 8;

And getEditBuffer()? That gets us the bytes we need to send to the instrument to set the currently edited voice. The function slaps the sysex header (from the description in the documentation which we read above, called "CURRENT PROGRAM DATA DUMP") onto the 8-to-7-bit converted version of the internal data buffer, adds the sysex footer and returns the resulting Array object.

var editBuffer = [ 0xF0, 0x42, 0x30, 0x00, 0x01, 0x51, 0x40 ];

editBuffer = editBuffer.concat(convert8to7bit(DATAbytes));

Armed with all of that knowledge and code, it will be pretty easy to add a couple of last features to our software: I'd like to be able to save any presets I like to disk as a sysex file, so that I don't need to save them manually to the hardware memory — we could add sysex functionality to save our work to a particular memory location, but I'll leave that as an exercise for the passionate reader. Saving implies that we'll also need a way to read those files and send them to the minilogue xd if we want to use them later.

Max's JavaScript implementation gives us a simple way to work with files with the File object, but we need to use Max's 'opendialog' and 'savedialog' objects in order to open an OS file dialog. Like this:

Very straightforward, but note the finesse with strippath to ensure that the filename "sticks" between usages. The 'opendialog' case (in the '

utils.writeSysex(fname, syx.getEditBuffer());

 file with the edit buffer bytes as an additional argument:

exports.writeSysex = function(fname, bytes) {

var f = new File(fname, "write", "Midi");

post("error writing sysex to " + fname + "\n");

) is created with write permissions and type "Midi" (which ensures that the file being written has a .mid or .syx suffix). If it's open, we simply write the array of bytes to it. Then we set the 

 (end of file) property to match the current file position and close the File object again. The 

 technique is used to make sure that there are no extra bytes at the end of the file, if we are overwriting a file which already existed, and which was longer than the file currently being written.

Reading the sysex bytes back out of the file is similarly clean:

post("error reading sysex from " + fname + "\n");

This time, we create the File object with read permissions and, when it's open, read the bytes directly into a JavaScript Array object. 

 is called from the function 'loadsyx', a message to 'js':

outputBytes(bytes); // push to instrument, too

, we first copy that byte array into our sysex data buffer and update our model (as if we had received a sysex block directly from the instrument). Then we output the bytes to the instrument.

We can now save our work to files on disk (and restore them later). Note that these files are not in the same format used by KORG's "minilogue xd Sound Librarian" software: hacking that file format is beyond the scope of this article. But for the motivated hacker, here's a hint: the program file saved by that software (with the .mnlgxdprog extension) is actually a zip archive and contains, in addition to a little bit of metadata, the 8-bit representation of the program data (1024 bytes).

If anyone takes up the challenge and adds 

 export (and import), I can't offer much more than some fame. But fame you will receive.

One very, very last thing: if you look near the end of 

. This is a test function which I used to sanity check the sysex we're generating -- it helped me find and solve a number of problems along the way.

From the name alone, you can already tell that we're into elite MIDI territory: System Exclusive messages (or sysex for short) are MIDI messages with some device-specific format and meaning. 

System exclusive messages begin with a header byte (0xF0), then some value(s) identifying what kind of device the message is for, and then generally some values describing what kind of message is being sent (the instrument might know about 20 different kinds of sysex messages, for instance). Then come a bunch of data bytes -- how many and what they mean is determined, you guessed it, by the instrument. Some manufacturers like to finish up with a checksum -- that's a number derived from the bytes preceding it, generally according to some more or less poorly documented math -- which is used to insure the integrity of the data. And then there's a footer byte (0xF7). 

0xF0 (Header byte: here comes sysex!)
0x43 (Manufacturer ID)
0x20 (Substatus, in this case it means "some kind of a request")
0x00 (Data, in this case it means "voice edit buffer")
0xF7 (Footer, there goes sysex!)

That's how you request the voice currently being edited on a Yamaha DX7. And how did I know that? From this delightful bit of documentation: 

Check it out! It's not so different from the DX7 request message: 

0xF0 (Header) 
0x42 (Korg Manufacturer ID) 
0x3G (minilogue xd) 
0x00 (Data start) 
.
.
.
0x51 (Data end)
0x10 (Current Program Data Dump Request)  
0xF7 (Footer)

If you send this to your minilogue xd, like this (Max converts the hex numbers into decimal numbers inside of a box)... 

There's some stuff to unpack here, but one of the first things you might notice is that it's wrong. The data size of a 1179 byte message with 7 header bytes and one footer byte is 1171 bytes and not 384. This is a harmless typo, but the first rule of sysex documentation, as in nuclear disarmament, remains: 

More importantly, the description refers to a conversion of "7bit" to "8bit" data, and this is where we'll begin. 

System Exclusive data has the same 7-bit limitation of all MIDI data (the status bytes -- the ones which tell us that we're about to get Note data, or Controller data, or, yes, System Exclusive data -- use the 8th bit). A sysex message looks like: 0xF0 ... data ... 0xF7, and all of that data is restricted to 7-bit values. The block of data that KORG describes in Table 2 (PROGRAM PARAMETER), which is the data we are receiving, uses 8-bit bytes. Before we can interpret the 7-bit data packet we received from the instrument, we need to convert it back into 8-bit data.

Clear as mud? It's trying to tell us that every 7 bytes of 8-bit data (56 bits total) will be converted into 8 bytes of 7-bit data by putting all of the high bits into the first byte of those 8 output bytes. Here's my own attempt at some ASCII art. 

byte 0: AAAA aaaa 
byte 1: BBBB bbbb
byte 2: CCCC cccc
byte 3: DDDD dddd
byte 4: EEEE eeee
byte 5: FFFF ffff
byte 6: GGGG gggg

byte 0: 0GFE DCBA // bytes 6 - 0 above, 8th (leftmost) bits
byte 1: 0AAA aaaa // byte 0 from above, 7 remaining bits
byte 2: 0BBB bbbb // byte 1 from above, 7 remaining bits
byte 3: 0CCC cccc // etc.
byte 4: 0DDD dddd
byte 5: 0EEE eeee
byte 6: 0FFF ffff
byte 7: 0GGG gggg

We call it like this:

function recv(b) {
    if (b === 0xF0) { // new sysex
        receiveBuffer = [b];
    }
    else if (b === 0xF7) { // end of sysex
        receiveBuffer.push(b);
        // slice off the 7-byte header and the 1-byte footer
        var data = receiveBuffer.slice(7, receiveBuffer.length - 1);
        var converted = convert7to8bit(data);
        // do something with the converted buffer here...
        var convertedBack = convert8to7bit(converted); // test
        for (var i in data) {
            if (data[i] !== convertedBack[i]) {
                post("there is a mismatch at byte " + i + "\n");
            }
        }
        receiveBuffer = [];
    }
    else if (b & 0x80) {
        post("bad sysex byte, aborting receive\n");
        receiveBuffer = [];
    }
    else if (receiveBuffer.length) { // data byte, append to buffer
        receiveBuffer.push(b);
    }
}

Sysex bytes arrive at the recv() function one at a time. When the header byte arrives, we initialize the global variable receiveBuffer and then fill that buffer with each arriving byte until the footer byte is read. At that point, we strip off the 7 header bytes and the footer byte and pass the data to convert7to8bit(), getting the converted 8-bit buffer back to work with. 

In the code above, you can see that we then pass the converted buffer into the function convert8to7bit() and compare it to the original input. That's just for testing, to ensure that our conversion functions are working properly and symmetrically. We'll comment that out when we're sure everything is correct.

function printName(converted) {
    var name = "";
    for (var i = 4; i < 16; i++) {
        name += String.fromCharCode(converted[i]);
    }
   post(name + "\n");
}

Alright, sysex reception is working. We have a valid data buffer. The next step is integration of the data in the buffer into our existing Model. For the most part, this is straightforward, but there are a few gotchas: 

Some of the data occupies more than 1 byte

Some of the data occupies less than 1 byte, and shares the byte's other bits with other parameters

Some of the data uses a different range than our Model

Some of the data isn't currently represented in our Model

Some of the data has a complex relationship to our Model

Good old reliable portamento only needs a new object member, I'm calling it syx, to identify which byte is associated with this parameter in the sysex buffer. When we need to read the value of portamento from an incoming buffer, or when we want to modify the value before sending back out to the instrument, we'll be using byte 17 of the converted buffer.

But consider "VCO 1 LEVEL", which we used in Part 1. That's a 10-bit parameter, so its value won't fit into a single byte. Looking at the sysex spec, VCO 1 LEVEL is placed in bytes 54 and 55. So we can do something like this to describe that, adding a syxLen member:

In theory, we could accommodate any parameter length in this fashion. Practically, no single value occupies more than 2 bytes in the data buffer for the minilogue xd. Quick aside: the documentation claims that the high bits are in byte 54 and the low bits in byte 55, but the documentation is wrong -- it's the other way around. 

{ name: "MOD FX TYPE", 
    cc: 88, min: 0, max: 127, enum: [ ... ], syx: 89, syxOffset: 1 },

In the case of enumerated parameters, the enumeration index is stored in the sysex, rather than the CC value. The MOD FX TYPE enumerated value ranges from 0-4 in Max (representing "CHORUS", "ENSEMBLE", etc.), but 1-5 in the sysex buffer, an offset of 1. We can address this with the addition of syxOffset data member. When reading data out of the buffer, we need to subtract syxOffset (if present), and when writing to it we add syxOffset to the current (enumeration) value.

The case of data which has no representation in the Model can be solved simply by adding those parameters to the Model (in a special "sysex parameter" section). The new parameters are, for the most part, settings related to the Step Sequencer and Arpeggiator. In any case, these settings cannot be modified with a CC or NRPN message. The "OCTAVE" parameter, for instance, is sysex-only. You can find these parameters in minilogueXD_syx.js.

This integrative work is all done, and if you take a quick peek at minilogueXD_cc.js and minilogueXD_nrpn.js, you'll see the sysex-related additions in the parameter arrays at the top of the files.

How does this all fit together? When a sysex buffer from the instrument arrives, we first do all of the MIDI byte processing above (in the recv() function above), which yields a data buffer of valid 8-bit bytes. Then we can do something like this:

var syx = require("minilogueXD_syx");
var params = getAllParams();
for (var p in params) {
    var param = params[p];
    if (syx.getParameterValueFromInstrumentData(param)) {
        param.setListenerValue();
    }
}

For each parameter, getParameterValueFromInstrumentData() (a function we'll examine in a moment) is called and, if the parameter has a syx member —that is, if it is associated with a sysex value — the parameter's value member is updated with whatever was received from the instrument. Finally, we tell the parameter's listener that the value changed — just like when a CC or NRPN message arrives from the synth. This will update the View so that we can see the current instrument state.

And here's getParameterValueFromInstrumentData(), which is exported from the required file minilogueXD_syx.js:

The syx member of our parameter model is already familiar. Note that we take the syxLen into account if it's present, placing the value of the second byte into the high 8 bits of the value. Finally, we apply syxOffset if it's present before setting the value using setValueFromSysex().

We need a way to handle value changes coming from our interface which have a complex mapping to the sysex data. For that, I've extended the Model with a function called onChange(). The function will be called whenever the value changes (via the View or when setting the value from sysex), and is responsible for setting the actual value of the parameter, as well as handling as any special side effects. Here's the most minimal onChange(), which just sets the value:

onChange: function(newval) {
        this.value = newval;
    }

This is what will happen if a parameter doesn't have an onChange member, in fact.

onChange: function(newval) {
        this.value = newval;
        var param = getModFxTarget();
        if (param) {
            param.setValue(this.getValueForView());
        }
    }

In addition to setting the value, we get the parameter which corresponds to the current "MOD FX TYPE" value (that's what getModFxTarget() returns). For instance, the parameter "MOD FX ENSEMBLE", which is a sysex-only parameter. And then we set its value, too. This ensures that the state of those parameters remains in sync with changes we make at the View. 

After updating the values with setInstrumentDataFromParameterValue() (see below), we grab the edit buffer as an array of bytes and send it from the 'js' object's outlet. From there, it can be passed to the 'midiout' object and on to the hardware.

setInstrumentDataFromParameterValue() is the reverse operation of getParameterValueFromInstrumentData(), which we looked at above:

And getEditBuffer()? That gets us the bytes we need to send to the instrument to set the currently edited voice. The function slaps the sysex header (from the description in the documentation which we read above, called "CURRENT PROGRAM DATA DUMP") onto the 8-to-7-bit converted version of the internal data buffer, adds the sysex footer and returns the resulting Array object. 

function getEditBuffer() {
    var editBuffer = [ 0xF0, 0x42, 0x30, 0x00, 0x01, 0x51, 0x40 ];
    editBuffer = editBuffer.concat(convert8to7bit(DATAbytes));
    editBuffer.push(0xF7);
    return editBuffer;
}

Max's JavaScript implementation gives us a simple way to work with files with the File object, but we need to use Max's 'opendialog' and 'savedialog' objects in order to open an OS file dialog. Like this: 

Very straightforward, but note the finesse with strippath to ensure that the filename "sticks" between usages. The 'opendialog' case (in the 'p loadsyx' subpatcher in the main minilogueXD_Tutorial_Part3.maxpat patcher window) is even simpler.

function savesyx(fname) {
    if (fname) {
        utils.writeSysex(fname, syx.getEditBuffer());
    }
}

which calls into the exported writeSysex() function of the required minilogueXD_utils.js file with the edit buffer bytes as an additional argument:

exports.writeSysex = function(fname, bytes) {
    var f = new File(fname, "write", "Midi");
    if (f.isopen) {
        f.writebytes(bytes);
        f.eof = f.position;
        f.close();
        post("wrote sysex to " + fname + "\n");
        return;
    }
    post("error writing sysex to " + fname + "\n");
}

Here, a new File object (f) is created with write permissions and type "Midi" (which ensures that the file being written has a .mid or .syx suffix). If it's open, we simply write the array of bytes to it. Then we set the eof (end of file) property to match the current file position and close the File object again. The eof technique is used to make sure that there are no extra bytes at the end of the file, if we are overwriting a file which already existed, and which was longer than the file currently being written.

exports.readSysex = function(fname) {
    var f = new File(fname, "read");
    if (f.isopen) {
        var a = f.readbytes(f.eof);
        f.close();
        if (a) {
            post("read sysex from " + fname + "\n");
            return a;
        }
    }
    post("error reading sysex from " + fname + "\n");
    return null;
}

This time, we create the File object with read permissions and, when it's open, read the bytes directly into a JavaScript Array object. readSysex() is called from the function 'loadsyx', a message to 'js':

If bytes are returned from readSysex(), we first copy that byte array into our sysex data buffer and update our model (as if we had received a sysex block directly from the instrument). Then we output the bytes to the instrument.

minilogue_editor.jpg

For the third and final part of our series about building a synthesizer editor in Max with the help of JavaScript, we set about the task of incorporating System Exclusive communications into the editor itself. I’ve also included extensive examples and documentation in the accompanying patches. 

building-a-synthesizer-editor-with-javascript-part-3

I now have a rudimentary librarian for the Blofeld written in ClojureScript and hosted in Node for Max. All works solidly - I was pleasantly surprised that I can pipe sysex byte-at-a-time over the wire from Max to Node and nothing falls over. Software wise the structure is totally different - very functional, using specifications, pattern matching and CSP-style pipes and channels for decoupling and timing control. I am intending to do a write-up when more of it is in place.

Thanks! That's great to hear that you had a good experience with N4M. I know you don't need any programming tips, so looking forward to seeing your Librarian!

Well, I once managed to totally quit Max and leave one of its Node.js processes running at 100% CPU, but so far that's not repeatable.

I've just bought Ableton Suite and got max4live as part of it. 

I was starting to work on a max4live editor for Minilogue XD so I could control everything from my Push2 from the couch :D 

Decided to google to see what others had done and found this!!! :)

Going to try and see if it's possible to get this stuff into Max4Live and the parameters all showing up on the push

This is excellent! 
I've just bought Ableton Suite and got max4live as part of it. 
I was starting to work on a max4live editor for Minilogue XD so I could control everything from my Push2 from the couch :D 
Decided to google to see what others had done and found this!!! :)
Going to try and see if it's possible to get this stuff into Max4Live and the parameters all showing up on the push

What a great series this was! So much good work debugging and then communicating clearly and empathetically to the reader. As someone who has crept painfully through hardware manufactures system exclusive documentation, I am so grateful to see someone else's awesome work doing so.

If only cycling74 would use syntax highlighting!!! 

At least we get a fixed-width font, but man...someone in UX/UI design, help!

Now, If only cycling74 would use syntax highlighting!!! At least we get a fixed-width font, but man...someone in UX/UI design, help!

Building a Synthesizer Editor with JavaScript, Part 3