Fora

I am in the middle of creating a Sysex librarian to edit and write patches that one could load into any sysex editor or dump directly to a synthesiser. I have has some success in using jit.matrix to write a JIT binary file and manually changing the file extension to syx, but that's more of a hack and not a viable solution as I am hoping to distribute my final patch. (Max for live)

After having searched the forum for a few hours, my search to find a solution to write syx (system exclusive) files has lead me to the File Object but I'm hopelessly stuck as Javascript isn't my strong suit and was hoping to pick one of your fine brains!

Would anybody know the best way to get data stored in [capture] and save it as a syx file using JS that isn't written in text format but binary, or can I still use jit.matrix with File Object in JS to write binary files with a different extension?

p.s. I have looked at the mxj.fileout solution but it requires installing Java which Im trying to avoid.

I am in the middle of creating a Sysex librarian to edit and write patches that one could load into any sysex editor or dump directly to a synthesiser. I have has some success in using jit.matrix to write a JIT binary file and manually changing the file extension to syx, but that's more of a hack and not a viable solution as I am hoping to distribute my final patch. (Max for live)

After having searched the forum for a few hours, my search to find a solution to write syx (system exclusive) files has lead me to the File Object but I'm hopelessly stuck as Javascript isn't my strong suit and was hoping to pick one of your fine brains!

Would anybody know the best way to get data stored in  [capture] and save it as a syx file using JS that isn't written in text format but binary, or can I still use jit.matrix with File Object in JS to write binary files with a different extension?

p.s. I have looked at the mxj.fileout solution but it requires installing Java which Im trying to avoid. 

Hi peeps,
I am in the middle of creating a Sysex librarian to edit and write patches that one could load into any sysex editor or dump direc

how-to-use-file-object-to-write-a-midi-syx-file

Ok, I figured out a solution using [Savedialog] using MIDI filetype to Jit.Matrix for anybody else who is struggling to export syx files.

1) [capture] - [jit.fill] - [jit.matrix] 

1)  [capture] - [jit.fill] - [jit.matrix] 
2)  [savedialog] - [jit.matrix]
3) ???
4) profit!

Funny, I just had a very similar need, and didn't come across your post after my initial searches. I've just solved it using Javascript, in case it's still of interest to you - hopefully it's intuitive enough - prepend "file_to_write" in front of full path to your file and send to Javascript, and then prepend "writebyte" in front of each actual byte you want to store. I'm sure it's easily extensible if you have a byte list that you'd like to send to it instead of individual bytes one by one.

function file_to_write (filename) {
    f = new File(filename, "write", "Midi");
}

function writebyte (thebyte) {
    f.writebytes(thebyte);
}

Funny, I just had a very similar need, and didn't come across your post after my initial searches.  I've just solved it using Javascript, in case it's still of interest to you - hopefully it's intuitive enough - prepend "file_to_write" in front of full path to your file and send to Javascript, and then prepend "writebyte" in front of each actual byte you want to store.  I'm sure it's easily extensible if you have a byte list that you'd like to send to it instead of individual bytes one by one. 

function file_to_write (filename) {
	f = new File(filename, "write", "Midi");
}

function writebyte (thebyte) {
	f.writebytes(thebyte);
}

That's great Dan! I was searching online for a javascript solution but couldn't figure out how to implement one due to my poor understanding of JS.

So if I wish to output modified data of varying lengths, do I have to declare it beforehand?

Do you have working patch I could look at?

Another question, is it correct to connect the javascript object outlet to jit.matrix? Or are you sending the writebyte (JS) to another object, just having an issue of seeing any bytes appear on a test file which I managed to create.

Here's an example patch - obviously without the full sysex message, but just a few bytes as an example. I don't have time at the moment to change the Javascript to accept the list input directly, but I can't imagine that's too difficult (I'm a relative Javascript noobie too)....

Here's an example patch - obviously without the full sysex message, but just a few bytes as an example.  I don't have time at the moment to change the Javascript to accept the list input directly, but I can't imagine that's too difficult (I'm a relative Javascript noobie too)....


It works, very elegant solution! many thanks!

Would you know a way to add filename extensions directly into the JS code above, ideally syx?

Would you know a  way to add filename extensions directly into the JS code above, ideally syx?

I worked its out, changed "Midi" to "MIDI syx"

Ok I have also noticed an issue when writing bytes, in which they sometimes do and other times don't.

I am sending about 8kb of data to be written but regardless of the size, bytes aren't always written.

I guess it's because the JS script isn't holding bulk data to be written, like you stated, it isn't meant for lists.

Yeah - there's two things I usually do - I provide a default filename ending in .syx to the savedialog - this automatically selects the "MIDI Sysex" option in the file dialog. But there's nothing keeping a user from stripping that extension off, so I also add a bit of code that forces the addition of a .syx extension, if it doesn't exist already. See attached patch.

There's also sexy ways of doing this (ensuring the .syx extension) with regexp, but I always have a hard time wrapping my head around regexp, so I stick with my clunkier approach you see here. Hope this helps.

Yeah - there's two things I usually do - I provide a default filename ending in .syx to the savedialog - this automatically selects the "MIDI Sysex" option in the file dialog.  But there's nothing keeping a user from stripping that extension off, so I also add a bit of code that forces the addition of a .syx extension, if it doesn't exist already.    See attached patch.  

There's also sexy ways of doing this (ensuring the .syx extension) with regexp, but I always have a hard time wrapping my head around regexp, so I stick with my clunkier approach you see here.   Hope this helps.


Cool, I like the default.syx which forces the syx extension to be on top of the list, also the sprintf is a nice touch.

Ok, i'm now trying to figure out how to write a long list of bytes. if I find a solution, I'll post it here.

Yes I think you're correct re: your problem with it not writing all the data - definitely look into modifying the Javascript to accept a list input, then deliver it all the data at once to write. Will probably be more reliable.

Yes I think you're correct re: your problem with it not writing all the data - definitely look into modifying the Javascript to accept a list input, then deliver it all the data at once to write.  Will probably be more reliable.

    f = new File(filename, "write", "Midi");

    for (i=0; i < arguments.length; i++)

no need to prepend any messages before a list.

inlets = 1;
var f;
function file_to_write (filename) {
	f = new File(filename, "write", "Midi");
}
function list()
{
	for (i=0; i < arguments.length; i++)
	{
	f.writebytes(arguments[i]);
	}
}

It seems this line "f = new File(filename, "write", "Midi");" is creating a file but for some reason, the rest of the code isn't writing data all the time.

Maybe because the two separate functions are running one after another perhaps?

It seems this line "f = new File(filename, "write", "Midi");" is creating a file but for some reason, the rest of the code isn't writing data all the time. 

Maybe because the two separate functions are running one after another perhaps?


When you deliver the full list to the Javascript method, I don't think you have to write the bytes 1 by 1; you need to convert the incoming list into a Javascript array, and then I believe the writebytes method will write them all at once.

You may need to deferlow the byte writing portion of your patch, to allow for the file creation operation to complete before writing to it.  Maybe something like this, just off the top of my head, not sure if will work.  If it doesn't you may have to send a bang out of the Javascript object when you are sure that the file has been successfully created, perhaps by using the "isopen" method?  https://docs.cycling74.com/max5/vignettes/js/jsfileobject.html  

Deferlow doesn't work here, I believe it's an issue with the file not closing when the write_to_file function is called, so with some further investigation, I came across (File.close) which closes any old open file in memory.

I have also figured out how to include .syx within the JS itself, so all you need is a [savedialog] without any filetype message.

    f = new File(filename+".syx", "Midi");

Deferlow doesn't work here, I believe it's an issue with the file not closing when the write_to_file function is called, so with some further investigation, I came across  (File.close) which closes any old open file in memory.


inlets = 1;
var f;
function file_to_write (filename) 
{
	if(f) f.close();
	f = new File(filename+".syx", "Midi");
}
function list()
{
	for (i=0; i < arguments.length; i++)
	{
	f.writebytes(arguments[i]);
	}
}

Not sure why that's working, as you're calling the close() function on a variable whose type hasn't been defined yet? You only declare it as a File object in the line that follows... ?

But anyway, if it's working, I guess that's all that really matters! ;-)

Not sure why that's working, as you're calling the close() function on a variable whose type hasn't been defined yet?   You only declare it as a File object in the line that follows... ?

But anyway, if it's working, I guess that's all that really matters!  ;-) 

I believe it's to close that variable that is still open when calling a new function, if that makes sense.

So without the close() function, the previous JS script worked first time around, but the second time it wouldn't close the file after it was created as it was still open and no bytes were written.

But thanks for chiming in, you've really helped me with this one!

I believe it's to close that variable that is still open when calling a new function, if that makes sense.

So without the close() function, the previous JS script worked first time around, but the second time it wouldn't close the file after it was created as it was still open and no bytes were written.

Ah OK, I see what you're saying. In that case, it might make more sense to put the close after you're done writing to the File, something like this. This is like the blind leading the naked, maybe a Javascript expert will come in and teach us *both* something! ;-)

    for (i=0; i < arguments.length; i++) 

        f.writebytes(arguments[i]);

Ah OK, I see what you're saying.  In that case, it might make more sense to put the close after you're done writing to the File, something like this.  This is like the blind leading the naked, maybe a Javascript expert will come in and teach us *both* something!  ;-)

function file_to_write (filename) 
{
	f = new File(filename+".syx", "Midi");
}

function list() 
{
	for (i=0; i < arguments.length; i++) 
	{
		f.writebytes(arguments[i]);
	}
        f.close();
}

Ok, this is my take on it (and if there are any JS experts willing to chime in and make sense of it, please do)

I believe the (close) resets the f variable(?) which then clears the memory for a fresh function call.

This was seen in the previous JS script I posted whereas it seemed to write the bytes the first time the code was initiated but the second attempt of writing the bytes, the script would fail.

Ok, this is my take on it (and if there are any JS experts willing to chime in and make sense of it, please do) 

I believe the (close) resets the f variable(?) which then clears the memory for a fresh function call.
This was seen in the previous JS script I posted whereas it seemed to write the bytes the first time the code was initiated but the second attempt of writing the bytes, the script would fail.

Glad to hear it, good luck with your project!

By no means expert here, but I would keep a filename in a global var and do all the interaction with file system in one method/function.

apologies for the basic question, what would that involve code wise out of interest?

inlets = 1;
var f;
var fn;

function file_to_write (filename) 
{
	fn = filename;
}

function list() 
{
       f = new File(fn+".syx", "Midi");
	for (i=0; i < arguments.length; i++) 
	{
		f.writebytes(arguments[i]);
	}
        f.close();
}

I have discovered something else now, I'm not using the savedialog but just a message with a name for the file. Now if I write some bytes (say 1 kb worth) and decided to write over the same file but with a smaller amount of data, the file size stays the same.

Do either of you know a way to include a replace message somehow?

Don't know how to do that. I do know that you can't delete a file with straight Javascript, maybe with Node.js?

If you go back to using savedialog, and it prompts you that you already have a file with that name, does it do the overwriting as you would expect?

Don't know how to do that.  I do know that you can't delete a file with straight Javascript, maybe with Node.js?

I believe it does overwrite it with the savedialog prompt, but because I have included the file extension as part of the script, it writes an extra .syx at the end of the filename!

funny, solving one issue only presents another, It's like a leaking ship lol

I believe it does overwrite it with the savedialog prompt, but because I have included the file extension as part of the script, it writes an extra .syx at the end of the filename! 

You could also keep data inside the js object and provide filename only when necessary. I think it's more max-ish approach – other container objects (eg. table, buffer~) behave in similar way.

var data;

// reset data
function clear() {
    data = [];
}

// append to data
function append() {
    for(i in arguments)
            data.push(arguments[i]);
}

// replace data
function list() {
    clear();
    for(i in arguments)
            data.push(arguments[i]);
}

// write data to file
function write(fn) {
    var f = new File(fn, "write", "Midi");
    if (f.isopen) {    
        f.writebytes(data);
        f.close();
    } else {
        error("Cannot open file: "+fn+"\n");
    } 
       
}

// init
function loadbang() {
    clear();
}

var data;

// reset data
function clear() {
	data = [];
}

// append to data
function append() {
	for(i in arguments)
    		data.push(arguments[i]);
}

// replace data
function list() {
	clear();
	for(i in arguments)
    		data.push(arguments[i]);
}

// write data to file
function write(fn) {
	var f = new File(fn, "write", "Midi");
	if (f.isopen) {	
		f.writebytes(data);
		f.close();
	} else {
		error("Cannot open file: "+fn+"\n");
	} 
	   
}

// init
function loadbang() {
	clear();
}

I am trying out the code but I am still struggling to find a way to replace a file with a smaller one in size, the file size doesn't change. the savedialog warns before replacing but it doesn't seem to delete the previous, instead bytes are written on top of the previous file.

Wanted to also say thanks, I was struggling to work out how to contain the data inside the JS object. 

Wanted to also say thanks, I was struggling to work out how to contain the data inside the JS object. 
much appreciated!

ok, adding "f.eof = f.position;" before f.close(); has sorted out the resizing issue.

I have one more issue which I am trying to get my head around, its to do with filenames, specifically when writing over the same file with the same name, because I have added a .syx prefix to the JS code, when I click on the file I wish to overwrite (using the savedialog object) the filename's extension is duplicated.

I was wondering if you know of a way to strip the extensions of a file?

write file name with extension, like write MySysex-1.syx

write file name with extension, like write MySysex-1.syx
and remove   
f = new File(fn+".syx", "Midi");

Here is a working solution. I think it's pretty much done, I'm sure it can be optimised but for now it does the job. I hope it's useful to somebody! 

I would really like to say thanks to everybody that helped me along the way!

        //    post("extension = .syx");

            var f = new File(fn, "write", "Midi");

        //    post("extension = blank");

            var f = new File(fn+".syx", "write", "Midi");

        error("Cannot open file: "+fn+"\n");

Here is a working solution. I think it's pretty much done, I'm sure it can be optimised but for now it does the job. I hope it's useful to somebody! 

I would really like to say thanks to everybody that helped me along the way!

// reset data
function clear() {
	data = [];
}

// append to data
function append() {
	for(i in arguments)
    		data.push(arguments[i]);
}

// replace data
function list() {
	clear();
	for(i in arguments)
    		data.push(arguments[i]);
}

// write data to file
function write(fn) {
	var f = new File(fn);
	if (f.filetype == "Midi") {
		//	post("extension = .syx");
			var f = new File(fn, "write", "Midi");
	}else{
		//	post("extension = blank");
			var f = new File(fn+".syx", "write", "Midi");
 		 }
	if (f.isopen) {
		f.writebytes(data);
		f.eof = f.position;
		f.close();
	} else {
		error("Cannot open file: "+fn+"\n");
	}
}

// init
function loadbang() {
	clear();
}

Nice work and thank you to all who helped.

@Dan.. I was hoping I could get your email as I have some Max for Live project ideas and wondered if you would be interested in collaborating on?

Of course - dan@defectiverecords.com , and take a look at 

Of course - dan@defectiverecords.com , and take a look at https://defectiverecords.com if you've not already...

Awesome! yeah defective, I'm familiar with it. Didn't realise you were behind it! Cool I'll drop you an email

Calling @Dan & @KLSDIZ (or anybody else with JS experience) !

I am facing an issue with the current code, which is working swell for small files but I'm working with

larger data sets now (on a different project) and it doesn't write any bytes to disk, I just get zero byte files.

I tried using the append (function in JS) and [zl group] but it crashes the patch.

Does anybody know what the size limitations for large arrays, lists are and if there's a way to work around that?

I am facing an issue with the current code, which is working swell for small files but I'm working with
larger data sets now (on a different project)  and it doesn't write any bytes to disk, I just get zero byte files.

for zl you must set an argument for maximum list elements (up to 32,000), otherwise it defaults to 256 (like it used to be 15 years ago.)

Thanks for the quick reply Roman, regarding the zl objects, I know this but I am trying to figure out in how to get around 32K limit of the writeline() method in JS

Sorry Fora, I'm afraid I don't have any knowledge of the JS limits....

I think I am getting a little closer to finding a solution, I found this post which has a work around but I'm trying to figure out how to get my data in..

https://cycling74.com/forums/sharing-is-fun-32k-limit-on-file-writeline-method-how-to-get-around-it

Aha! Nice that Peter Nyboer posted his solution - thanks for linking to that for future searches of the forum...

Aha!  Nice that Peter Nyboer posted his solution - thanks for linking to that for future searches of the forum...

It definitely puts me within the ball park but I’m still scratching my head.

Any ideas in what would need changing for it to store a dump from the capture object and write those instead?

Yeah it’s a great little find! 

It definitely puts me within the ball park but I’m still scratching my head. 

Any ideas in what would need changing for it to store a dump from the capture object and write those instead? 

Sorry, you're far ahead of me on this one!

No prob! I will defo post a solution as soon as I come across one :)

SOLVED: How to use File Object to write a MIDI syx file?