newtfish

Ive noticed, if Im observing 100s of parameters of a device for instance, with JS, if I change the preset of the device (so all the parameters move to the preset), live locks up completely for quite a while, especially with big devices like operator.

Im wondering why this is? Is there any way of stopping it? Is it not possible to observe this many parameters effectively using JS. Is there a workaround, like slowing down the output of the observers?

js-live-locks-up-when-observing

hi, just checked this out with 5000 observed properties and it is noticably slower... never really noticed this before - will do some more checking over the next few days.. there's only about 70 parameters that get updated so it's not that much really...

Its the same with the normal objects. Max seems to crack completely when a lot of parameters that are being observed change at once. Like for instance a large instrument (operator), if you observe every parameter and then use "hot swap" to swap a preset, live locks up completely. Quite often it will take a few minutes for live to come back again.

Observing in Python doesnt seem to cause this problem.

ive narrowed my problem down to udpsend. Im really not sure how to change the speed of message throughput to udpsend. qlim and speedlim dont work, because they remove messages. Should I be using something like zl queue?

EDIT:its really strange, initially m4l works just fine, changing presets fast. Then after a while it just becomes sluggish, taking longer and longer to load the observed params. I cant quite pinpoint where it is happening...

ive narrowed my problem down to udpsend. Im really not sure how to change the speed of message throughput to udpsend. qlim and speedlim dont work, because they remove messages. Should I be using something like zl queue? 

EDIT:its really strange, initially m4l works just fine, changing presets fast. Then after a while it just becomes sluggish, taking longer and longer to load the observed params. I cant quite pinpoint where it is happening...


Hi. not my experience - as soon as I start observing stuff it slows down - I have no udpsend involved at this point... I'm wondering whether the data structures used aren't particularly efficient and don't perform well under hi load.

Interesting you say this seems to be the same for non-js objects - was going to try that this eve.

Could be the python->MAX interface that is the bottleneck. Might be worth raising a question with the Abes/C74? as I'm not sure this should really have such a dramatic effect if using appropriate data structures

Interesting you say this seems to be the same for non-js objects - was going to try that this eve.
Could be the python->MAX interface that is the bottleneck. Might be worth raising a question with the Abes/C74? as I'm not sure this should really have such a dramatic effect if using appropriate data structures


btw when I say "dramatic" I am talking 1000s of observers - with a few 100 the difference is noticeable but not significant

300 observers (maybe 70 on the operator, the rest over the set) - approx 0.5s to hot swap Operator

approx 5000 observers (still around 70 on the the operator) - 8-10s to hot swap operator

e.g. no observers - immediate
300 observers (maybe 70 on the operator, the rest over the set) - approx 0.5s to hot swap Operator
approx 5000 observers (still around 70 on the the operator) - 8-10s to hot swap operator


Yes, totally agree 8-10 seconds with 1000s. However, when deleting observers and creating new ones after a time, it will slow to a crawl. It sometimes takes around 1 minute to change all the parameters in operator. It seems to take longer and longer, until you delete the device and add it back in again. Once the device is fresh its zippy, but it soon slows down.

I wish there was a way of refreshing the device without deleting it. Cant be sure whether udp is the bottleneck, when I put a speedlim in front of it, the slowdowns get better, but then thats because speedlim is deleting 90% of the data being sent out.

Additionally, I've noticed that closing down Live completely (not just deleting the device) and starting afresh can make a big difference, but unfortunately not to swapping the presets (was testing this last night) - but general scanning of 1000s of parameters happens much quicker....

btw. do you have a notifydeleted() in your js to clean up all observers when you swap/delete the device?

try removing the link to the udpsend - that should indicate whether it's causing an issue for you...?

Support msg sent. But im starting to think that it might be faster to delete the observers and recreate them again. This would involve figuring out when a lot of changes are happening (for individual IDs) and then just deleting/refreshing the whole lot.

Total fudge, but im not sure whether im left with any other options :/

I need to look at optimising some of the code, but for me the set with 5000 observers takes about 15s to initialise them all which is longer than the 8-10s to hotswap... interested to know how long it takes you to initialise?

I've looked into how this works on the Python side and unfortunately, the code that executes when swapping a device and the like is not very efficient, and gets worse the more observers that are added to the set - unfortunately there's little that can be done about this without the Abe or C74 doing some work on their side...

Thanks for testing Lee. Im going to try create a test patch for c74 to debug the problem. It sounds like youve tested many more observers than me. Im running into major problems just with devices like operator or analog, and these only have around 200 observable parameters (takes around 10 seconds). There also doesnt seem to be much difference between the speed of JS and standard Max observers. It seems a general M4L problem :/

you are right - it is a general M4L issue, not specific to JS - it is to do with how the module (called MxDCore) handles the processing between Ableton and MAX (whether it's js or not). The problem is actually related to the number of observable parameters in the device being changed and the number of observers currently in use, so you can see why the time gets quite long quite soon.

Its around 2-3 seconds to initialise, with no slow-down in live (ie no "not responding" message). However, thats with only the operators parameters (around 250). I dont notice the initialisation too much, I have a script including each parameter observer as a maxpat file, which I then give each id.

When the observers are first set up, hot swapping presets seems fast (around 3-4 seconds). The speed gets much much slower when I delete those observers and create new ones (around 10-20 seconds or even more) (with a "not responding" message during that time). Its almost like those observers are still in the background working away (even though theyre deleted).

It such a shame. If you use live.object to bang out all these parameters its incredibly fast, but observing is so much slower and degrades with the more observers you delete/create.

Im thinking to delete the observers when the hot swap button is activated and then, use live.object to bang out the values. Then recreate the observers when hot swap is deactived (seems quicker, but total fudge). This strategy also wont work in live 8 because there is no hot swap available in the api.

Quite interesting that the MxDCore python script has the answer. Seems that even if C74 solve the problem with the slow observer, it still wont be available to live8 (because any updates to the python script would only be available to L9 users). I wonder if there is a way to edit the MxDcore script to make it faster? I wish my python skills were better :/

Its around 2-3 seconds to initialise, with no slow-down in live (ie no "not responding" message). However, thats with only the operators parameters (around 250). I dont notice the initialisation too much, I have a script including each parameter observer as a maxpat file, which I then give each id. 

Im thinking to delete the observers when the hot swap button is activated and then, use live.object to bang out the values. Then recreate the observers when hot swap is deactived (seems quicker, but total fudge). This strategy also wont work in live 8 because there is no hot swap available in the api. 

Quite interesting that the MxDCore python script has the answer. Seems that even if C74 solve the problem with the slow observer, it still wont be available to live8 (because any updates to the python script would only be available to L9 users). I wonder if there is a way to edit the MxDcore script to make it faster? I wish my python skills were better :/


"Its almost like those observers are still in the background working away" - from the code they are still there - they're not actually doing anything, but they're slowing down operations that iterate over the arrays unfortunately. there is a python routine to dump all the parameters a device is currently observing and start again from scratch, but it's not exposed unfortunately

"I wonder if there is a way to edit the MxDcore script to make it faster" - it should be possible for them to make quite a considerable performance gain by addressing a couple of things I think, but whether they are likely to do anything about it is an another thing

I have a good idea how things are working now so I may have a play over the next week - it's a shame that this bottleneck exists for the M4L route that is not there when using Python... I'm surprised it hasn't gained more attention in the past

Sent detailed description to Ableton(this is their code) who will investigate... fingers crossed!

This is brilliant news, Thanks Lee. Really hope they can do something about it. It's one of the main areas where m4l falls down IMHO.

Hi, I've just uploaded a fix for this issue to 

There'll be more M4L performance improvements and tweaks coming in the future, but this should be a good start.

Hi, I've just uploaded a fix for this issue to Livelink MxDCore

JS live locks up when observing