Sysex and Live 9 – is it now possible?
This is a question that despite my best efforts I can’t get answered, put simply:
Can an M4L device send SYSEX out through Ableton’s Midi ports? I was previously of the understanding that Ableton filtered SYSEX on the way out, is this still the case with V9?
Basically all I want to do is build devices to edit parameters on pre-CC synths. I have built devices for more modern machines like the Roland Gaia but would like to do the same with things like D50′s etc etc.
it’s my understanding that you can use objects such as thesE:
to get access to sysex data "around" Live, instead of through it.
Thanks Wetterberg, that’s sounds just like what I need. Thankyou very much for the heads-up
any chance you could share your Roland Gaia patch? Would love to integrate my keyboard more closely. Thanks, Gen
Sadly I’ve had some issues getting Leigh’s objects to working in 64 bit, the alternative I’ve come up with is to manage the sysex in/out in Max and build the control surface in M4L, syncing between these OSC. I actually find I prefer this system now for managing patch storage etc.
I’ve got controllers for the D-50, Alpha Juno 2, ESQ-1, DW8000 and a few others I use, I’ll post them when I get the chance.
I have no idea about M4L, I’m uses only Max. not sure this will help you.
when I work with SysEx I need to create patch like below
----------begin_max5_patcher---------- 1071.3ocwXsziapCEdcxuBK1zGJMBargP20EcQUU6c6U5ppJd3j5Jvf.yzLS U+ue8CXBYJSfjg3nYFLbrG7474uym8geubgSbwdZsC38f+CrXwuWtXg1jxvh 1mW3jGsOIKpVOLmjh7bJW3rxzmftWns+oWcGETSE.4nY4M4f3lsaoU.QAnLZ mrK1CTvqwtg9f36Ez50uo6UvR0ufh3e9NufNiaK3BdTNU20GpXQYc8vaxY7L pP6NvtgWsKV8r6Z2CCqnQzMtNqkQhjev369dEMQXBanGQ9OA777UMnfPci7E A9VOeQ495IDs1UY8OKWptr5khZaAoQhHMj.j9KnXKn99Z5di4pHtD5dsK3cx IN3MfeQeUEEjjUTSA6hDz0Cgg91GC8Bc0nFV23YZlaLjS+kL99KH7s.2APAz lyDEPsFMAr39RpIzbXxksGCiiQD3IPDLpOhfVSrFfnHF.3vnh6bgJW.jzBEt 5FehEoHkUzRJOUIOM.n.CuvDl4CTvPOUS.9Z.JJXflx9awGI4D3J4lxoG3I+ AtQ9HV9GRQej2g0ctAf8UWk2CccA99p6L+Z09vD.NDPbUcSPG2srOBV2Adn2 4y8KB+3FNYLNMongqgF7.rjIr0TYTkztfV8cJOJNi1W0bhLnUFAmUlaOPnFS bFOJEiz1nS7BsShWcYFSHoRxMuFJw6b2ox6TZzqNoTM54wGTnYmen8ElTXSb eVXezgLWxRsnSr7vDifRdm3XR9a5gRaBrJJoTpdHKIiFUAfCgU3YFqZmrKGt ZOBjAsfPnEgqGx.5iYuqpnob.vJX1NCvpmeKOznmCvbd6MdVDYpoYZpzvIby lZTWhVW6ESefAFdSvUQSJtQHJ3C.DjyKkQGlm+IeB7LgXWn0EfWPjzHqspYn RgdYaYyDz71Zic92+4qe78eFoVNWotHKq8KQ6UhQGa.cDRLcAmWDOwT8UmBs cxkxYoLFeawPUYbyKx.qSXHHapsnpZmMT5DbN2a5RQj1uxQvYI2pGi9jwO4S DomFk8iwo5hlpjNuscqFvgYJkVKX7HASp5bXLpxu5MnevRSo79ogorZUloFP GdAaptiTgWVayHtiuUcG3XtCwptynKVXq4NjI3NniPvqo23Mk0pQvFojYYgL ItMYBhzk6CMa9gCUehH3Qolym6q9bXiiltiD.IEYEUsNuxmce7BrmW+jvzi. 0e7qMjCgo9gqQX5NgvLvZTXzD7FOq4MOQn8Y3v1CcfSQ+6ID2qt+LobD3zxQ NJ83z4Htjd4HgjqYNBjbyjBfAAVSJPGlvaxpIBBs6pI51DlXKRZw2rbydzTq Dk2FNaaf4A6EkdvqWThtMBPX6cVD3DXrWZsFlByhJKuiVU29J0NhrT0eZfE+ U5GYbyi5BDcpn2w5Fef1RTkrdQgrXwlJSco68wNKUyyeV9+vS4diO -----------end_max5_patcher-----------
hei brooke -
it’d be amazing (!) if you could post an exemplary patch of yours.
and: do you run your max-part of the sysex-code from within the max application or did you create a standalone device?
any clue which of the two is more performant?
How much work does the standalone app do? Are you prepping the sysex in MaxForLive and sending it as a message via OSC, or are you sending the app controller values and then converting that to sysex?
I only have MaxForLive. I wanted to put together a MaxForLive device for the Matrix 6/6R/1000. I believe I could make the device send the sysex pre-formatted over OSC. With that said, I cannot build a standalone device that receives sysex over OSC and sends it off as midi.
Any chance you can share this? I will be most happy to share my Matrix 6 device with the entire audio community if I can make this happen :)
PS – this is a serious limitation in Ableton Live. I understand not implementing a way for Live to parse sysex, but to completely truncate it even when originating from a MaxForLive device is excessive and frustrating to say the least…
I found this – http://cycling74.com/toolbox/midi-and-osc-mapper/ which lead to this http://lamplightforestboutique.com/control-core-midi-osc.html but I am on PC … /sigh
- This reply was modified 1 month by Matias. Reason: added new info
Woops sorry hadn’t noticed updates in the thread.
Benzzzz – Either, packaging it into a max application is just a convenience for people that dont have Max, they are both very fast since we’re just dealing with messages.
Basically I’m working on a way to send OSC messages from Live to be converted into Sysex messages by an app (built in Max, packaged as an executable/app), that has a library of OSC<>MIDI sysex conversions availa. That way you can build a M4L device that has all your gui details and save the device state as a live preset that sends its state as a dump message when it’s loaded.
Previously I’d been building a OSC<>MIDI conversion app for each device I wanted to control, but now I realise building an app that lets you simply select a sysex format to use from a list and tells you what OSC messages to send there would be much easier. Once you get used to writing sysex editors doing a bunch of them is a breeeze, older devices have occasional quirks though.
I’ll work on things later this week then post the patch to github. It’d be great to have some collaborators so we can design nice editors for all those midi devices.
Live not having sysex is a serious limitation, almost as bad as being restriced to midi channel 1 in M4L. But I think designing a OSC to sysex conversion app is actually a superior way to deal with managing sysex.
+1 for the silliness of the midi limitations in live. filtering of duplicate CC messages, no sysex, midi channel limitations. Would love to see this all go away!
i’ve got a stack of external gear here (dsi, moog, korg, kawai, hohner…). i was planning to beginn a librarian-patch asap, but would very much more like to help out / collaborate. if desired also with the annoying part of interfacing / gui (i recently did m4l’s ‘polyrhythmus’ as a visual reference if wished for).
thanks brooke! we need a decent tool for preset management!
- This reply was modified 1 month by bennnzzz.
Thank you very much for your reply Brooke.
I maybe misunderstood what you said, so apologies if the following is already what you were describing.
Would it not be easier to have the standalone simply receive a sysex message in the form of a list over OSC, and then send the individual values of the list off to MIDI? My understanding is that any device will automatically know something is sysex if it receives F0h. So if the standalone receives /sysex "F0 xx xx xx xx xx .. F7" then it can forward that off as raw midi data, symbol by symbol (or int by int, if we do "240 xx xx xx .. 247") to the corresponding MIDI channel to get the sysex message through. Of course the symbols would need to be converted to bytes.
This would allow the standalone to simply be responsible for MIDI routing and all the heavy work to be done inside MaxForLive. The standalone becomes just a wrapper to overcome the lack of sysex in Live. We can go further and choose different network ports for different MaxForLive "sysex controllers," with each port corresponding to a different midi port or channel, and then it becomes a sort of wrapper/router. Please let me know your thoughts. I’m about to patch this together in MaxForLive. I would just be unable to compile it, but I’ll share what I come up with shortly.
PS – once the wrapper implementation is established, I call dibs for the matrix 6,6r,1k for the MaxForLive "sysex controller" design :) I tried to call dibs on the alpha juno, but as I reread the thread, I see you’ve already got one made for yourself!
going off my thoughts from earlier regarding the type of list to send, I think a list of ints is ideal. if you monitor sxformat with a print, you’ll see it spits out the bytes (ints) sequentially as I suggested the standalone should. patcher incoming tomorrow (I hope) .. I became quite busy today, so I apologize for not having something ready by now.
OK! Patcher time. This is a very simple example of what I feel may be the most light-weight and effective way to get sysex out of Live and into our synths.
I originally planned to use OSC messages like so "/sysex 240 xx xx xx xx 247" and then send that off to a port on the computer, where the standalone application "OSC-routes" /sysex to get to the actual list of ints for the midi. Then, I remembered that CNMAT updates their externals versions at different rates for different objects and OSes. Furthermore, there is no need for the tree/branch/twig address arrangement with this implementation. So I wondered why I needed OSC at all. I thought maybe it was necessary to properly use the network functions, but Max sends data just fine without OSC.
Please note the two different ways I thought about transmitting the midi data. I personally feel that it’s more robust to send each sysex message as a list, even if it requires the use of an extra object in the standalone, but you may disagree with these semantics (I just think that we may stall the receiving midi device in the rare chance that the sysex end byte doesn’t get through between udpsend and udpreceive).
If anyone (willing to compile this thing up) likes this idea, I will develop it further with a UI to allow the user to enter midi port and channel. If at that point, things are still running smoothly and the application proves useful, we can work on adding multiple network ports for different sysex receiving midi devices.
(btw this is the first time I post a patcher up like this .. so if something is screwy, I’ll give it another try)
----------begin_max5_patcher---------- 930.3oc0XE0iRDCD9YtD+OzrQSzDDa6BKfuoIZhI5S9nZLkskiZVZ2zVPNM9 e2osKHfGm6UAt6RHa2ssz8qeSmY9l8mO5hNYSzqD1LzKQeB0oyOgd5D5y2Sm 0czIaNaUYEyFlXlR7c8juk0sYLmXkKzubSW5EtJgycUsHtxYRELkurd3oZkS wlGFL6UFIqB8ZcEeyeWxCi.ujmmObSupEykJXYCfftt2ZlqblTc4WMhRW7s0 mLrGtKhNl5axo8FzEQF0C+G..KUDhg0hrMtrxeDvEA2CG59WO5BeKzzs0LTo d9bArk+KJ5iNlhypzJAhUWi.LKjKEVjal.M4JGbGLNxJTbKR5PNMbeErwDbz bIWhp0FWOzSeGZFi6GsTqVJLgIxTHfkEWJLgmVpkbDC8clQArCZpQOG8A1pm sASURknTuPE.1fTLMCtdSC4FLMEjf0fNf3MMjA88ME8OrsAeHayl9MWNILwS g4BHr2pMuGLQHtXorTzXZ93U12rBwpLBF+JDyFMNQCnm7C1IjVErqU5RfEmo stqm6ySg66mL2ODG7GF34PvU4Nl5OTrDOgBXIKAtgN91yMMgLxyGGXkvC2TH C7QOjwgHhE75lnDnw3wjCGhMw3qjjIKJNzLZz+hrHmSxx6fFc47db6xY2lCQ CSlWHz7XS+y+oHNrmN7QjoUZ114g2d+Vj99sH3yfWe8OqeMy.rsSX9pPwlTI 1YOu2Yji9YA6poZyblCQ6iAL5+gQu.IsnGKIvMz9COA5UnCRlGGQir4v6O9S mBEcz7auhtXX4GRB5pj1MB.1HgyqJnaPgGnsyVWIcAUd.UpgKb4RIeAvbMx3 rQ4evrgsl+9cVqMxAOdB5nza+YWJdWAc3GZB5frq0qEzELDrnsaec3mdMcjw IS+OLzzYqM.COE84rsCI+DoOT7myN1JaHIjAOFmoIPbex8bgM3TD1PnoRK2k 5ZNEkIfSkHtGVkfDDYcrcfR1+oIY.s3dk+yVUQgO1TUQxT0cWMTIWpvnjCfb JJTHtHgru6+YLC.wOvdzfUuvTtdit9iLh1BNbg0IULmTq1ZV9uqv1yZljyEp cfJWZ8vOvU2fko8.izFfsO7OC.yWlSKXr6.fUzJfM37Cr7VAL7YGXs6re9YG WsxPd9OfQZEe4UYclA1nVgq+OBqIXKTo6Rn9zlkNBIH4x2zF+yEciOKUwmiI FxLP8Vq+KwpSyXFHQgCxRrvDyxtZTQFLR3EBW9MHjhgF -----------end_max5_patcher-----------
- This reply was modified 1 month by Matias.
Nice thinking Matias, since (all?) sysex lists have the manufacturer and model in the list that makes routing messages easier as well.
It should be fine to manage all this with udpsend/receive, since its just a simpler osc subset.
I’ll post up an example for the D-50 at the end of the week, sorry things are a bit crazy right now. Feel free to come up with a alpha juno editor, mine is just a bunch of number boxes atm.
OK .. here are the beta products.
Max_Midi_oIP_Thru should be run from Max or compiled as a standalone device. It’s a very simple and lightweight device. The user chooses the network port the device should "listen on," the user selects the midi port the data will be forwarded to, and the device repeats whatever comes through the network port as midi bytes to the midi port.
I removed the "iter" object as upon closer investigation of the midiout object, I learned that it can take a list and parse that as individual bytes on its own. I was unable to figure out how to specify what channel it should midi-thru stuff on, and since I don’t have an actual copy of Max, I can’t know exactly how the midi port drop down populates. In theory though, I think the logic is sound; the midi channel may just be defaulting to 1 or something.
Alpha_Juno_SysEx_oIP is the MaxForLive device that transmits the SysEx to the standalone device. It takes user input for every control on the Alpha Juno and transmits it as a packaged sysex message, a list of ints, to whatever port the user chooses within the local machine. I took inspiration from the original Alpha Juno controller, the PG 300… not a potentiometer in sight :)
So for those that have Max, MaxForLive, and an Alpha Juno, I hope this thing works correctly! I shall wait patiently for a kind soul to compile Max_Midi_oIP_Thru into a standalone executable to be able test it out as well :) From here the challenge is figuring out how to select midi channel when using the midiout object, updating the standalone to allow multiple devices to be used concurrently, and creating more templates for other midi-sysex devices. I am open to any and all critique/feedback of both devices, so I look forward to hearing from interested parties!
EDIT – I made a small correction (I forgot to change the key for the VCA envelope type to "Gate"). I updated this and re-uploaded the file thinking it would overwrite what was already uploaded. It didn’t, so now there are two files, the newest one is Alpha_Juno_SysEx_oIP1. Functionally, both files are the same.
throwing a little bump out there :)
Ok here’s a quick (naive) implementation of how this would work. With the alpha Juno. Just checked and all the editing controls seem to work.
Unfortunately after looking at the midi spec the alpha juno doesn’t have a device id, at lease one that follows the manufacturer id, which somewhat spoils the idea of having the sysex message identify what kind of device is sending sysex easily I was hoping for.
I’ve made a few changes to your m4L interface on this basis, and am instead sending the message /sysex/[manufacturer]/[device].
I’m leaning more to handling all the logic in max when it comes to handling preset dumps and updating on patch changes etc. I’m thinking about something like having m4L send and receive /sysex/[manufacturer]/[device/param [control] [value] and /sysex/[manufacturer]/[device/dump [list...] for presets.
But if people want to handle all the midi messaging in m4L, well that’s very easy. I’ve included ‘simple’ versions of the patches for that.
Thank you for checking out the work and offering your insight. I’ll be checking out the edited patch tonight or maybe tomorrow night depending on the time I have available.
I like your idea of using OSC to break up the sysex code. I considered leaving OSC out of the picture due to the extra memory, cpu cycle requirements, and compatibility issues across different OSes. While there is a benefit to using OSC for preset dumps, it is important to note that for real time patch editing, a parameter will start out as a control change, then it will get packaged to an OSC message, and then it will get parsed to sysex. I felt going straight from CC to sysex would be less bloat, but maybe I’m being overly paranoid about overall performance.
I am still concerned though that after the first two sysex bytes ("sysex start" and manufacturer id), it’s essentially an anything goes situation between different hardware, as you’ve found. Never-the-less, we can work around that. If you look at the differences in sysex between the 106 and alphas (http://www.vintagesynth.com/roland/sysex.php), you’ll see that the third byte is still different between the two synth engines. We can utilize it to id our devices, even if the manufacturer didn’t intend that third byte to denote the device id. While I was reviewing the sysex implementation in the alpha juno for the device id being missing, I also figured out where the midi channel is specified, so this can be properly implemented now :)
I’ll check out what you’ve put together and recap again soon. Cheers and, again, thank you very much for your time and efforts.
- This reply was modified 1 month by Matias.
Cool, yeah I see the merits of both approaches. I was just looking ahead to maybe automatically identifying the sysex message format and translating it. But simplicity has some definite strengths.
Check out the simple versions, it’s just an osc to to midi converter than has no expectations for messages (other than maybe 240 … 255 to be added). It’s closer to your vision I think. I’ll look into update management and presets for that when I get a chance
I finally got some time to go over this, and I gave the non-simple version some thought. Attached are the results.
I implemented midi channel selection to the alpha juno template. Then, since you added the osc address to the maxforlive device, and you mentioned you wanted to keep the logic in the midi manager, I took out the "global" sysex out of the messages being sent out of the maxforlive device (ie the alphajuno template).
I edited the midimanager to make it "intelligent" so that it would re-add the correct sysex based on the osc address. I left everything un-abstracted for the sake of visibility / simplicity, but eventually there would be an abstraction for every sysex midi device that the manager supports.
I still only have MaxForLive, so I still cannot test out the work that has been done :( By the size of the patcher windows, I take it you’re on a mac… is there any way for you to compile the midimanager into an executable for windows?