Cycling 74 » Topic: Parsing real-time sysex messages

Parsing real-time sysex messages

dhjdhjdhj — Sun, 15 Jul 2012 11:51:53 +0000

Anyone know of an alternative method for receiving and recognizing incoming real-time sysex messages? Unless there’s a hidden option, the [sysexin] object sends out received bytes one by one rather than as a list. Since incoming sysex messages can be different lengths, I don’t see how [match] can be used as it seems to require the exact number of bytes that will be in the message to recognize it.

I looked around but couldn’t see any third party libraries that had such a feature. I’m sure someone must have addressed this issue before me.

Thanks,
D

Re: Parsing real-time sysex messages

dhjdhjdhj — Sun, 15 Jul 2012 12:03:12 +0000

I’m wondering if it would make sense to write an external that would accept sysex in messages and produce a single packed message out of it.

Re: Parsing real-time sysex messages

broc — Sun, 15 Jul 2012 12:19:21 +0000

I think you can use the end marker (247) to trigger [zl group].

– Pasted Max Patch, click to expand. –

Re: Parsing real-time sysex messages

dhjdhjdhj — Sun, 15 Jul 2012 12:46:33 +0000

Nice — that’s certainly a good way to get this working in the short term — I never think about the [zl] stuff. Thanks for that.

I do wish however there was a sysexin variation that produced a single list per received message rather than sending out each byte separately. There can be a lot of real-time sysex data and it seems a shame to have to process them all one byte at a time in Max. It’s both awkward and expensive. I understand it for ‘traditional’ sysex messages where real-time is less important, but if you move 10 sliders at the same time with your hand and each of them is generating continuous real-time sysex messages, that starts to add up.

I wonder if the Cycling74 team would be willing modify sysexin (perhaps with an attribute) that would allow it to buffer sysex messages and send them out as lists.

Re: Parsing real-time sysex messages

dhjdhjdhj — Sun, 15 Jul 2012 12:49:17 +0000

Also, would it make sense to detect 240 before storing any data and send a zlclear message out in case something gets screwed up before a 247 byte arrives.

Re: Parsing real-time sysex messages

broc — Sun, 15 Jul 2012 18:02:27 +0000

Be aware that using sysex messages for real time control is inherently problematic as any status byte (eg. note-on) which is sent after 240 and before 247 may “abort” the current message.

http://home.roadrunner.com/~jgglatt/tech/midispec.htm

Re: Parsing real-time sysex messages

dhjdhjdhj — Sun, 15 Jul 2012 18:18:37 +0000

That’s why I made the suggestion to detect 240 as a means of restarting.

I have not yet done the experiment to see what data comes out of each keyboard if I play notes WHILE moving sliders. Even though those real-time sysex messages are short (my Kronos produces 14 byte messages), I don’t know whether the Kronos will interrupt (and abort) a real-time sysex message that it’s sending out with note on events or whether it will just put them on the front of a queue so it goes out immediately after a real-time sysex is sent.

Re: Parsing real-time sysex messages

willyc — Sun, 15 Jul 2012 21:07:52 +0000

I’ve previously used the following patch to turn a sysex message into a single max list.

I went with coll rather than zl group to overcome the ‘max list length’ of the zl family, though zl would also work assuming you know in advance how long the sysex messages you are dealing with are going to be (e.g. use [zl 1000 group] if you knew you’d be dealing with 1000 byte messages)

– Pasted Max Patch, click to expand. –

Re: Parsing real-time sysex messages

dhjdhjdhj — Mon, 16 Jul 2012 03:23:04 +0000

That’s a nice implementation and I’ll try it out.

I’m a little leary of having to use a bunch of separate objects (with the implication of a lot of messages) to process real-time sys-ex messages but maybe it will work fine.

I have a feeling though that it would be nice to have a single external into which you could say things like “Match sysex messages that have particular values in known locations (e.g, the bytes that represent individual sliders)” and then emit one or two bytes somewhere else in the message, i.e, the bytes representing the new position of the slider.

Re: Parsing real-time sysex messages

broc — Mon, 16 Jul 2012 09:01:58 +0000

Yes, it would make sense to have something like [sxparse] as complement of [sxformat].

Re: Parsing real-time sysex messages

Chris Muir — Mon, 16 Jul 2012 19:24:29 +0000

Because the format of a SysEx message is arbitrary, it would be very tough to build a parser for them. There are a variety of “compression” modes used to fit data bytes into the 7-bit world of MIDI. Here’s a zl solution, which seems pretty straightforward to me:

– Pasted Max Patch, click to expand. –

Re: Parsing real-time sysex messages

dhjdhjdhj — Mon, 16 Jul 2012 19:37:51 +0000

Not sure what kind of parsing you’re talking about — I would be very happy to have a mechanism where an incoming sysex arrives as a single list and then use a SLICE function to extract sublists based on start position and length. A couple of operations to allow a few consecutive bytes to be combined into a single value (e.g, byte0 + byte1< <7 + byte2<<14)

Converting such a sublist to a string would let regex work as well.