live api input buffering ? / strange latency issue
Hello Everyone / anyone , hope all is well.
I have been working with a new system that involves a teensy code that is encoding a mixture of boolean and float data from varius sensors as an array of different osc messages and sending at 50hz over a 5g network and received via the udp-receive object in max, the messages are then unpacked with OSC-route and sent to various other amxd's maybe around 50 or so throughout my set, the messages in most cases are then mapped into live via the live api.
the problem is that the input data seems to be buffering , the system in total has 2x the teensy and 5g tx feeding 2 seperate udp objects , with a total of around 50 inputs of a mixture of the boolean and floating, the floating inputs are the ones that generally cause the issue, though after the problem starts the booleans are also effected.
i have been experimenting with different maxqueuesize arguments , as my college who wrote the code suggested that in an ideal situation we dont want any queuing and from a design perspective we would sooner see dropped packets for this application.
so ive been working with a maxqueuesize of 15 , this did improve slightly , but the problem still persists.
for example i have one of the accelerometer axis mapped to a dial in an actual ableton plugin for example beat repeat via the api objects , and after some time of running the set with transport playing and constantly running all inputs at once the input begins to lag and after i stop the physical movement stops with the accelerometer , the dial still shows the remainder of the *buffered packets as they come threw .... and in worse cases , the input continues to be buffered untill it shows no refresh at all in locks up and becomes unusable , also if i turn off one of the systems feeding either one of the udp objects the problem also disappears , it seems to be the combination of the 2 systems data that is causing this queuing / buffering, as if i was to run one system from the start its impossible to create this issue.
although i have the queue on nearly nothing , and note this still happens with a maxqueuesize of 1
so it seems maybe there is some secondary buffering going on within the api objects , or something similar ?, i can only assume that because live does not support OSC and was engineered to deal with consumer midi input devices and that this midi protocol is based on some kind of max input rate ?
The reason i say this is because im able to pretty much eradicate the issue by adding a speedlim 50 on the accelerometer data line just after the accel/ argument is unpacked , on one of the teensy systems , inputs but i know this is kind of bad practice and id really like to have my code working / designed to suit what im trying to interface with in this case the live api.
so my question is, is there any secondary buffering or am i just completely making this up in my head .. am i missing something here ? if there is any limitation on the speed that the data can hit the live api , can someone please direct me to this information or explain it to me ive really been looking everywhere.....
and if i was to change my codes send rate of 50hz on the floating inputs to something slower would this help solve the problem ? (im going to try this today)
also i have been looking for the scheduler settings for amxd's ? namely event interval refresh rate and etc , as ive been told to look into these settings if the issue was something solely in max , not m4l.
another side note is i usedto have this same issue with my old system with completely different hardware and firmata for similar tasks sent over 2.4 xbee.
apologies if my terminology is not so great , and thanks so much anyone willing to give me some advice here !!
thanks for your time.
this is somethign i found that might be a hint for me.
"The LiveAPI Object and the Scheduler
Beginning with release 6.0 of Max, it is no longer possible to configure JavaScript functions to run in the high-priority thread of Max's scheduler. The LiveAPI object cannot be created or used in the high-priority thread, so users should be sure to use the defer or deferlow objects to re-queue messages to the js object.
"
i do use the defer and defer low on some inputs , but this makes no difference on this issue.
I have since slowed down my messages in the code to 30hz as i had a speedlim50 that was doing the job on the max input end and checked the midi baudrate and did a conversion to hz , wich is 30 and this seems to be working well , but would still like to know what was happening here just for future.
thanks
Hi Forum,
I am bumping this because I am stumbling on the same problem.
Udpsend is buffering data. I have isolated it to this single object at the end if the chain.
I have tried CNMAT's OpenSoundControl formatting of packets/bundles, resetting it's buffer, but it seems internal to Udpsend data that keeps ending up in some queue.
Setting the maxqueuesize does not do much, if anything at all.
It's a rather urgent matter to me. Any help appreciated!
Sono
Hey SONO
i know the frustration,
i think it was around this time that i ditched max for live, and i haven't used it since for any of my hi res input stuff , i still send midi over to live, but from max stand alone, and i get none of the same issues i was getting at all , its a massive massive difference and i now have even more floating inputs at very high send rates, so id strongly suggest this route, unless since 2016 the api has changed, but i think at some point Ableton or someone told me that m4l runs in the graphics thread , so yeah there is some unspoken headroom there that seeming made these questions
^ unanswerable , but that said im sure someone could answer than but noone ever did ... though i do remember just when i made the changeover they did add some changes that were seemingly based on what i had speaking to cycling about , but i honestly cant remember... there was some feature added so an object ... ill look for it , but really i just wouldn't wast my time on mfl for this kind of large amount of realtime input work.
ok found it , was 15 october ish , they added the smoothing functionality to the live.remote~ wich is where i found the issue to be happening ....
even though a few days before that i rebuilt everything / moved to max standalone and completely ditched m4l, i'm sure at the time i was thinking the smoothing may have helped and i was in contact allot with andrew and i'm sure it was implemented because of our discussion's <3 at least i like to think so, ...
no idea if it will help but thats defiantly when i stopped thinking about it.
Hi Face head,
Thanks for getting back.
The problem I found is about udpsend buffering data, causing lag. It does not appear to relate to anything else within the patch. In any case it doesn't match your description regarding live.remote~.
Cheers