converting packed data formatted sysex data

redhexagonal's icon

hello

i'm requesting a sysex edit buffer dump from my evolver synth, and getting my data dump into max in "packed data format" where there are 8 byte packets, with the MS stripped from 7 parameters and packed into an eigth byte, sent at the start of the 8 byte packet.

is there an easy way to convert these packets into a list of the actual parameter values?

dan b's icon

I'm struggling with this too - if anyone has any pointers that'd be massively appreciated.

i've built a very basic patch editor for the synth - attached (which hopefully will be useful to some other people too, because frankly it was pretty boring to build!) To make it really useful I'd like to be able to load the edit buffer into it so I can see what my starting point is and to do that I need to turn the packed sysex into something more useful.

Here's how the manual puts it
"Data is packed in 8 byte packets, with the MS bit stripped from 7 parameter bytes and packed into an eighth byte which is sent at the start of the 8 byte packet"

input data:

1 A7 A6 A5 A4 A3 A2 A1 A0
2 B7 B6 B5 B4 B3 B2 B1 B0
3 C7 C6 C5 C4 C3 C2 C1 C0
4 D7 D6 D5 D4 D3 D2 D1 D0
5 E7 E6 E5 E4 E3 E2 E1 E0
6 F7 F6 F5 F4 F3 F2 F1 F0
7 G7 G6 G5 G4 G3 G2 G1 G0

PACKED MIDI DATA:

1 00 G7 F7 E7 D7 C7 B7 A7
2 00 A6 A5 A4 A3 A2 A1 A0
3 00 B6 B5 B4 B3 B2 B1 B0
4 00 C6 C5 C4 C3 C2 C1 C0
5 00 D6 D5 D4 D3 D2 D1 D0
6 00 E6 E5 E4 E3 E2 E1 E0
7 00 F6 F5 F4 F3 F2 F1 F0
8 00 G6 G5 G4 G3 G2 G1 G0

So it's a bit like a matrix transformation I guess. Anyone have any ideas on how I can stick the relevant byte from the first row into the first column of each of the subsequent rows?

Hex and sysex are pretty new to me. I've managed to pick up quite a bit from this thread
https://cycling74.com/forums/sysex-hijinks-custom-ui-elements-perl
and Peter Elsea's great guide here -
ftp://arts.ucsc.edu/pub/ems/maxtutors/advancedMax/Max&Sysex.pdf
Hopefully they'll be helpful to others too

2020.EvolverTweak.maxpat
Max Patch
Timo Rozendal's icon

I had the same puzzle since I am making an editor for the prophet 08, here is the solution:

Max Patch
Copy patch and select New From Clipboard in Max.

you have to process each slice of 8 bytes with bitshift operators to get the original 7 source parameters

the parameters are ordered according to their NRPN parameter number

Lasse Munk's icon

Hi!
I am also puzzled by SysEx. I am also new to the whole bits'n'bytes :)

I've searched around, but I'm a little in doubt of what to search for.

To clarify - the packets are sent FROM the synth to max in the form:

PACKED MIDI DATA:

1 00 G7 F7 E7 D7 C7 B7 A7
2 00 A6 A5 A4 A3 A2 A1 A0
3 00 B6 B5 B4 B3 B2 B1 B0
4 00 C6 C5 C4 C3 C2 C1 C0
5 00 D6 D5 D4 D3 D2 D1 D0
6 00 E6 E5 E4 E3 E2 E1 E0
7 00 F6 F5 F4 F3 F2 F1 F0
8 00 G6 G5 G4 G3 G2 G1 G0

But before one can actually use that data, one would like to convert it to a data package on the form:

input data:

1 A7 A6 A5 A4 A3 A2 A1 A0
2 B7 B6 B5 B4 B3 B2 B1 B0
3 C7 C6 C5 C4 C3 C2 C1 C0
4 D7 D6 D5 D4 D3 D2 D1 D0
5 E7 E6 E5 E4 E3 E2 E1 E0
6 F7 F6 F5 F4 F3 F2 F1 F0
7 G7 G6 G5 G4 G3 G2 G1 G0

??

And when a package is sent TO the synth, it should then again be converted to the "packaged midi data" form?

Why is this packing used?

All the best,
Lasse

Mr-Bit's icon
Max Patch
Copy patch and select New From Clipboard in Max.

I needed to repack.

julynessi's icon
redhexagonal's icon

I'm having difficulty unpacking and repacking some sysex. What should happen if my packed sysex message length is not a multiple of 8? what do I do with the left over bytes?

So if the packed part of my message is 15075 bytes long, that leaves 3 bytes left after I have unpacked the first 15072 bytes using the patches kindly provided above. What do I do with these extra bytes? Or should I leave the first 3 bytes and unpack the last 15072 bytes? Confusing...

redhexagonal's icon

Any sysex experts out there?

redhexagonal's icon

Basically the question is - is a packed sysex message always made of 8 byte blocks? Or can it be any length, and if so how do I deal with leftover bytes?
It could be I'm the bytes leftover aren't packed at all - I can't tell because there is no documentation for the Rytm sysex. Any ideas welcome, or pointers to resources about sysex, I can't find anything in depth.

FP's icon

Thx a lot Timo Rozendal. Not the first time you saved my time…

Jeremy's icon

Check this thread: https://cycling74.com/forums/unpacking-monomachine-sysex-data for additional strategies here.