To much OSC data over wifi to the Lemur causes omissions(?)

    Jun 10 2013 | 5:44 am
    I am not so sure about this: I have created a large template with max and the iPad Lemur app. Both interact with each other and in Max I change the user surface with a button within a split second, hiding, relocating or resizing controller objects. However, although everything seems to work, sometimes some objects, that should be hidden or resized, stay as they are. But it's totally random! Most of the time everything is ok, other times this object does not get changed, other times another one ... and then it works again.
    I have checked the messages going to the udpsend object, and they were definitively sent! On the iPad, I do not have an input monitor.
    To my understanding this could only have to do with either my Wifi network omitting data or the Lemur app (iPad 1) suffering from to much data in a short time. I highly doubt the second alternative, though. What would you guys look into first, to hopefully solve this issue?

    • Jun 10 2013 | 11:18 am
      It's not like I'm really answering your question, but there is something more suited for using Max/MSP with iPad as controller, it's Fantastick by Pink Twins, as far as I know there is no such performance issues at complex patches.
    • Jun 10 2013 | 1:32 pm
      when sending lots of messages very quickly, some of them *may* be lost...
      one solution :
      try sending bundles of messages using the OpenSoundControl external.
    • Jun 10 2013 | 2:11 pm
      I have this problem as well, and it's not just with Lemur, it happens with any of my apps that use UDP. It's not necessarily related to sending a large amount of data either.
      I send both continuous messages (slider moves, say) and one-off text messages (to change the label of a slider)
      As long as there are a few slider changes in one go, most of them will make it, but every now and again, a text label doesn't get changed and I have to resend it.
      I wish those apps would allow both UDP and TCP messages to be received (and sent)
    • Jun 10 2013 | 2:45 pm
      Thank you, guys! So I am not crazy ... ;-)
      About that OpenSoundControl external: I have it installed, but I can not really wrap my head around its functionality. Let's say, I have a "send OSCtoLemur" object that collects all data for the Lemur and a "receive OSCtoLemur" object. How do I connect that OpenSoundControl external, "receive OSCtoLemur" object and the "udpsend 8000" to make it work. OpenSoundControl does not have a help file, so I don't really understand how it is used.
      @DHJDHJDHJ: It is not really a few seldom omissions, more like quite a few and frequent ones!
    • Jun 10 2013 | 3:49 pm
      I have found that the quality/strength/distance of the wifi system impacts this significantly. I did quite a few experiments when I first ran into the issue, including sending OSC data from Max on one computer to another computer using both wifi and ethernet.
      When my laptop started to get far from the wifi router, the quantity of missed messages increased dramatically.
    • Jun 10 2013 | 3:59 pm
      That was also my thinking. Maybe I get a better wifi router and try getting it closer to my workspace.
      Regarding the OpenSoundControl external: who knows what it does and is it of any use in this case? Mathieu suggested sending bundles using this object. That sounds like a solution, as it would either send everything or nothing at all. I am just not sure how to do that, as sending OSC messages into that object and watching the output via print, nothing really happens ...
    • Jun 10 2013 | 6:45 pm
      From the help file: "Bang closes all open bundles, sends out the buffer and resets."
      In other words, when you send a bunch of OSC messages into OpenSoundControl they are parked in a buffer. They are sent out all at once when you give OSC a bang as input.
    • Jun 10 2013 | 10:56 pm
      The attached file does it! Setting the delay value to about, I don't get any omissions any more! Thanks for your input, guys!
    • Jun 10 2013 | 10:57 pm
      The attached patch does it! Setting the delay value to about, I don't get any omissions any more! Thanks for your input, guys!
    • Jun 13 2013 | 1:12 pm
      Setting the delay to about "what?"
      I'd love to try this to see if it fixes my issue, which I had just about given up on solving.
    • Jun 13 2013 | 1:43 pm
      To 8 in my case. This patch delays proportional, depending on the number of OSC messages sent per second. When there are omissions, just set the delay up, until they disappear.
    • Jun 13 2013 | 1:45 pm
      ok --- in my case, I was seeing the problem (missed updates) even if I just sent a single message from time to time, so not sure this will help.
      But I'll give it a shot.
    • Jun 13 2013 | 1:50 pm
      Also, what's the reason for the bang object underneath the OSC-timetag? Is it to make the object run at low priority?
    • Jun 13 2013 | 7:11 pm
      You can leave out either the bang object or the trigger bang bang. I just had this in for testing manually. It is important, however, that you send a bang to both inlets of the timer!
      The timer then determines how much time elapsed from one OSC message until the next one. If this is more than one within a millisecond, the if module (if $f1 It does work! However, if you even have dropouts with single OSC messages, this might not help you. My omissions only occured with a large number of OSC messages sent at the same time (dumping a collection).
    • Jun 23 2013 | 6:49 pm
      this hasn't worked for me unfortunately. I'm still seeing the same lost-packet behavior as I was before. I tried a value as high as 50 (I'm sending 264 OSC messages to the lemur in one go).
      has anyone had any luck sending a bundle to lemur? I tried making 8 bundles out of the 264 messages. Each bundle contains ints, floats, symbols, and lists of symbols. Unbundled, and aside from the lost packet problem, everything makes it to lemur correctly. Now, once bundled, only the values that are floats get through. Everything else is ignored.
      I have increased the OpenSoundControl object's buffer size, and tested to ensure that if I dump the individual OSC messages inside of the bundle that they look right in the print window. I could live with converting all values to floats, but converting the ints to floats didn't help. Even if it had, I'm not sure how I would have converted symbols to floats... could it that only lemur faders and pots know how to decode osc bundles correctly? I'm stumped.
      I feel I may be barking up the wrong tree here if it is a lemur limitation, but I thought I'd throw a dart in the dark in case there is something I'm missing.
    • Jun 23 2013 | 11:54 pm
      IIRC there's some stuff you can do with deferlow in this situation, too?
      I've done some heavy-duty things with brute-forcing things in place on Lemurs and ipad-Lemurs in the past, and that usually did the trick, if I'm not mistaken.
    • Jun 24 2013 | 6:39 pm
      I'm still troubleshooting, so take this with a grain of salt. The behavior in my case is that if I command the data dump from Max by sending an OSC message from Lemur to Max, there are virtually no packets lost. Now if I dump the data to Lemur without any prior Lemur-to-Max messages, the packets lost are substantial. As such, I feel there may be an issue with Lemur or maybe the iPad where it is not actively listening, and it seems that making Lemur speak forces it to listen.
    • May 22 2016 | 4:50 am
      I have to revive this topic again ... The solution I found was just temporary. Later, I had again a lot of Data loss! I thought, if I would use a tethered connection, any loss would be really unlikely. I bought an android tablet recently (because it has a big screen and is quite inexpensive) and I used a USB to Ethernet converter to connect the tablet to my router. All devices in this network with static IP addresses (except a small number of DHCP bandwidth for guests).
      However, I still get data loss and for the first time, now, I witnessed some observations of other posters here, too. The data loss seems completely unrelated to dense transfers. I just had one multi pad sending program changes from time to time. Sometimes, those were not sent. It drives me crazy!
      So, did anyone find a real solution to this? Is sending midi to the Mac more reliable? I am kind of pependent on OSC, as I send OSC commands to hide objects or set text, that midi cannot do. But I could may learn Lemur scripting better and then do the optical transformation inside the lemur.
      Or is there a way, OSC will work without any data loss? I am afraid, with UPD there isn't ....
    • May 22 2016 | 5:02 am
      I still have this problem. Just occasionally a set of messages, whether sent as single messages or as a bundle, doesn't seem to arrive at the lemur tablet. I've reduced the probability of this happening by sending my status messages twice, with a slight delay in between but that's a hack.
      My sense is OSC (or really the underlying UDP) is just not reliable for single status messages and just useful with continuous data where it doesn't matter if a single message gets lost.
      Even so, it shouldn't get lost
    • May 22 2016 | 2:11 pm
      Thanks for the reply! So ... Have you (or someone else) been trying out midi for a change? There are some things you obviously cannot do with midi (except for maybe with Sysex, which I do not want to get into), like sending new name lists to menus or multi switches etc. But I think, most of that could be done inside the Lemur with scripting. I guess, I will take that route, now. But I would be pretty disappointed to find out there is the same unreliability with midi messages sent over lan. There are some situations, where even the slightest data loss is critical ... Like a missed note off!
    • May 22 2016 | 2:16 pm
      "I’ve reduced the probability of this happening by sending my status messages twice, with a slight delay in between but that’s a hack."
      Could you elaborate a little, what that is about? Maybe a small example patch? Thank you!
    • May 22 2016 | 3:12 pm
      Hello gang!
      I am late to this conversation, but have experienced a lot of the same packet loss sending stuff to Lemur.
      From the research I have done it seems this is simply a UDP kinda thing - Packet loss here and there is just part of it, so plan accordingly. When sending data from continuous output controllers you don't notice a few packets dropped here and there...
      I had a "grab a filelist - parse the names - send messages to populate a buncha button names on lemur" patch i built.
      Was losing lots of messages, (5% each pass) the workaround was to simply re-run the outputting of the filelist via OSC several times.
      When I put a lil delay in each message send I had better luck. (aproxx 50 a second instead of 300 )
      SO - even tho I would lose several messages on each "burst" if I re-ran it 4 times 99.5% of the time all my buttons would end up getting their messages. Not perfect, but plenty good enough for my needs.
      I concur, weak signal is your enemy.
      Hope this helps any
    • May 22 2016 | 4:07 pm
      Year, 4 times is better than twice. But my iPad with lemur is only about 10 feet from the router so I'm not buying the weak signal argument.
      Granted there are no guarantees in UDP but sending a few sporadic messages every 5 minutes (once for each song) really should work in that close situation.
      Now having said this, I did try touchosc once and it did seem more reliable but it missed some features I needed.
    • May 22 2016 | 5:42 pm
      Not suggesting your signal is weak, is just one of the things that can contribute.
      The takeaway thought for others studying this topic is your line - "Granted there are no guarantees in UDP "
      Awesome? Yes.
      Fast? Yes.
      Bulletproof? No.
      I personally have less packet loss using a adhoc network (Mac>Lemur) rather than thru a wireless router, and better (less) latency.
      My experience with this was sending to 8+ pages of buttons, 100 buttons on each page, with two 8 character text strings for each button (top line/bottomline). (1600 text lines per batch, all sent as separate messages to individual buttons)
      Thru a router i had one to three messages that would get dropped per page if I only sent the whole batch once...
      Ad hoc I would have maybe 1 loss per page.
      Pulling at straws / pure fud for thought ....
      Is your router connected to the net? (as in - is there any chance the Ipad is talking on the net/updating facebook/app store/etc. etc. in the background when it should be being a good little monkey and listening intently/raptly for your OSC?)
      With my project the Ipad 3 lost fewer messages than the Ipad 2 (possibly less horsepower to devote to incoming osc????)
      Someone above mentioned (to paraphrase) "Maybe Lemur is not always actively listening" I sense that is very possible - maybe the Ipad gives up priority to other processes if you haven't been banging Lemur w OSC in the last x number of minutes. like i said, fud for thought...
      Hope this helps any.
    • May 22 2016 | 6:02 pm
      No, my setup is specifically for live performance so there is a linksys router connected to my laptop (on the LAN side) and then I have two iPads connected to that router over wifi. Both iPads are within 5 feet or so of the router. The router is not connected to the Internet and no other programs are running on the iPads (and even background refresh is turned off)
    • May 22 2016 | 6:10 pm
      Yes turning off the background refresh is goot, goot thing.
      Maybe try the adhoc... see if it helps...
      I think i remember being able to connect two Ipads to my mac once...
    • May 22 2016 | 8:05 pm
      As I said: My Tablet is connected to the router with a cabled ethernet connection. So, there can be no dropouts due to signal loss, there. And I do loose some signals by simply tapping a pad some times. In 2 Minutes of doing so, there will be one lost signal. Sending the OSC message multiple times is not a solution, as midi is timing critical in music.
      I switched to sending midi only, for now and I did not get any signal loss so far. I assume, it has to be some kind of problem of the Lemur app itself.
    • May 23 2016 | 8:23 am
      ""Maybe Lemur is not always actively listening" I sense that is very possible"
      "I did try touchosc once and it did seem more reliable"
      I've only been getting into Lemur lately, but have tried a few OSC apps and found that TouchOSC was the only one that worked well on an existing network rather than Ad-Hoc. TouchOSC sends out a ping every minute I think to the host which may just keep that connection alive and ready. That could be scripted on Lemur I expect and might be worth a shot.
    • May 23 2016 | 9:55 am
      I guess one could also set up a TCP-ish handshake thing with the lemur, for some mission critical controls
    • May 23 2016 | 10:50 am
      I avoid using WiFi connections with Lemur or TouchOSC in concert halls as much as possible. I had too often connection loss, even with ad-hoc networks or dedicated switches. TCP would be nice…
      When possible I use MIDI over cables (and interfaces…), but my use is generally only to display informations on iPad/iPods, so only one way communication.
      friflo, it is possible to send text over MIDI. Format you text in a sysex, ans use Lemur's arraytostring() function.
    • May 23 2016 | 11:34 am
      Here's a Lemur setup that sends an OSC ping every minute, I'd be interested to see if it helps anyone!
    • May 23 2016 | 11:35 am
      Hmm, didn't like Lemur files, lets try a zip...
    • May 23 2016 | 3:29 pm
      Useful Lemur setup! thanks!
    • May 24 2016 | 6:33 am
      This conversation reminds me of MobMuPlat LANdini.
      ""LANdini" : A custom networking protocol for many-to-many communication, with guaranteed delivery and ordering. See the "LANdini" example for more info on LANdini. All devices must be LANdini enabled to be aware of other devices and receive messages from them. Each client pings out its name and other information, and every client keeps a record of the other clients' states.
      All messages are prefixed by 1) sending protocol ("/send", "/sendGD", or "/sendOGD"), 2) a destination ("all", "allButMe", or a user's name), 3) the rest of the user message. The messages then show up at the destination (without the first two elements).
      Example PureData messages:
      Send a guaranteed-to-be-delivered message to all clients except the sender:
      [list /sendGD allButMe /myParameter myValue ("
      Also, to anyone doing this wirelessly, one may ask, is your network SSID name being broadcast? If so then people's devices may be trying to connect to it. Perhaps if people, so to speak, are trying to connect their phones to your wireless router during your performance then perhaps that could detract from it's performance? In that case, it would be the router and it's settings which would be at fault.
    • May 24 2016 | 6:53 am
      p.s. see also "Ping & Connect" in the MobMuPlat OSC bullet point above LANdini on the same page.
    • May 28 2016 | 1:23 pm
      I have switched to using midi Sysex and, while it isn't just a straight forward to program as OSC, I figured out how to make everything work as I want it to and so far, have not witnessed any data loss at all. Let's see, what happens, when the template gets larger ... Although I think it won't matter much, as I have seen data loss with OSC even with small amounts being sent and haven't observed any of that with midi ...
    • May 28 2016 | 5:54 pm
      You have me wondering how feasible it would be to implement a simple ACK system so that when Max sends an OSC packet to Lemur, it must receive an ACK within a short time otherwise it will just send it again (TCP for dummies!)
    • May 28 2016 | 11:27 pm
      That sounds like a great idea, but for now, I will keep midi! Maybe another time, when I will start over from scratch. ;-)
      Seriously: For my purposes, Sysex is ok. It is a bit more complicated to set up right to label textfields etc, it is also limited in size per junk, but apart from that, it does everything I want for now.
    • May 29 2016 | 2:22 am
      If you have an example of the Lemur side of things to handle sysex data, I'd love to see it. I've never been quite able to figure out how to make scripts work in Lemur, documentation is unclear and the lemur editor is not conducive to helping one figure things out.
    • May 29 2016 | 1:31 pm
      Yes, the lemur forum is nowhere as helpful as this one, nor is the documentation ...
      But with a lot of cursing, you can figure it all out! ;-)
      When you send Sysex to Lemur, in max you have too send your text (message or textedit or whatever) as a list to an 'attoi' object, which converts it to ASCII, the 'prepend 240' and 'append 247'. That goes directly out to the demon Lemur midi in. That is really all there is to it in Max, as long as you just want to send one sysex bulk to be converted to one label (Text, Monitor or labeling of other objects).
      On the lemur, you set up a script, execution on_midi, ‘System Exclusive‘ (the other parameters don't matter for Sysex).
      This means, you set the attribute "content" (meaning the actual text), for the text object "Text1" to a vector (array) of the received midi arguments, which gets converted to a string (from ACII back totext). Actually, in this case you could probably just leave out the subarray, as the whole MIDI_ARGS would do it as well:
      But with the 'subarray()' call you can select a certain portion of the text, if you want to split the Sysex data to different operations. With one or two numbers at the start of the Sysex dump (in Max prepend before the 240, which defines Sysex start) you can kind of create an ID, which Sysex should trigger what, which has to be filtered in the Lemur script by one or two 'if' conditions accordingly.
      So, e.g., if you would want the sysex from before only executed on this text-object, you would send from max: 240 0 {your text in ASCII} 247. The Lemur script would look like:
      The first line lets only Sysex starting with zero be executed by this script. Because you don't want the zero be part of the text, you have to define the length of the array excluding the 240 and 247 = 'sizeof(MIDI_ARGS)', start your text with 'MIDI_ARGS[1]' (which is actually the second position of the array). The last argument of the subarray is the length of the subarray, which is that of the whole array minus one, as you start on the second position.
      I hope that helps and will get you started!
    • May 29 2016 | 5:08 pm
      Thanks for the detailed instructions. As it happens, the only part that I've never quite been able to figure out on Lemur is the "setup" bit
      On the lemur, you set up a script, execution on_midi, ‘System Exclusive‘
    • May 29 2016 | 6:07 pm
      You are welcome. Yes, if you want to convert to text, you have to, because the arraytostring() thing is only available there. For simple value changes, with or without calculations inside Lemur you may also just set up the x value of incoming midi and use a small formula with x in the parameter fields or as an expression.