Heinrich Lenz

Hello all, happy to join the C74-Forum :)

for my BA I'm working on a rather large project in Max. It uses lots of different subpatches with multiple instances interacting with each other. A lot of them are integrated within bpatchers but there's also a couple of polys going on. Recently though I have been running into some issues. (I very much hope I included all necessary info, please hint if something might be missing)

After a while of using the patch signal processing within the subpatches just seems to... stop. First the GUI elements (like line representations of playback progress) start stuttering or don't move at all and eventually they just cease to process audio data. I know this is not a programming issue as it affects all modules within bpatchers. Deleting and then undoing the delete of any bpatcher instance will restart signal processing in ALL bpatcher modules, for a short time untel the problem resumes.

Two things are very strange about this behaviour: CPU and RAM usage is not even particularly high (usually 20 - 30 % CPU and ~40% RAM), so it seems it shouldn't be hardware ressource problem. I'm also not getting any stutter or dropouts, the patches simply stop working while the audio processing in the parent patch continues completely fine, with some rather expensive convolution reverbs working without even a hickup.

I'm using Max 8.1.8 on Windows 8.1. with an Intel Core i7-4720HQ and 16 GB of RAM.

Does anyone have experience with large patches, subpatches or bpatcher causing this kind of problem? Or any other idea how this might come about? Very grateful for any ideas and thank you very much for taking the time to read this!

I'll gladly supply the patch if needed, but the collective is unfortunately larger than the 5MB file limit. Also the issue is not 100% consistent, as there is no specific action that reliably produces the problem.

I'll gladly supply the patch if needed, but the collective is unfortunately larger than the 5MB file limit. Also the issue is not 100% consistent, as there is no specific action that reliably produces the problem.

best

Heinrich

Hello all, happy to join the C74-Forum :)
for my BA I'm working on a rather large project in Max. It uses lots of different subpatches with 

subpatches-guimsp-freeze-in-large-project-despite-low-cpuram-load

Hi there, since you mention GUi elements and the representation of playback progress, there might be some optimization possible. For instance, if you poke a signal with [snapshot~] or other [number~], you might look at the rate: it's certainly not necessary to get such information faster than 25 times per second, but a lower rate might even work for you.

It sounds extremely strange to me that DSP stops in subpatchers without reason - I've personally never experienced that. When that happens, is it in a poly~ that would be muted? You can check that with a print object, and when that happens to you, you could send a "get mute state" kind of message, to see if that's the case or not.

Or does this happen with subpatchers including externals, not standard Max objects? There might be something to check in that domain...

Another thing is about your Audio setup: are you working with scheduler in Overdrive and Audio Interrupt on (I would recommend that in most cases)? Does the bug happen regardless of these settings? Improvements or not with a larger vector size?

Also, since you mention poly~, do you send~/receive~ signals across the poly~ barrier (not recommended)?

Sound like I'm fishing, but the problem is fishy.

It would be great if you could reproduce the problem with a small version of your patch, ideally with "just a bpatcher", or a couple of them. You could delete elements you feel don't matter, and keep the most suspect things (gui elements that refresh often, the processes with externals, poly~ processes), then see if you can trim down your patch to something small that would still show the problem.

Hi there, since you mention GUi elements and the representation of playback progress, there might be some optimization possible. For instance, if you poke a signal with [snapshot~] or other [number~], you might look at the rate: it's certainly not necessary to get such information faster than 25 times per second, but a lower rate might even work for you.
It sounds extremely strange to me that DSP stops in subpatchers without reason - I've personally never experienced that. When that happens, is it in a poly~ that would be muted? You can check that with a print object, and when that happens to you, you could send a "get mute state" kind of message, to see if that's the case or not.
Or does this happen with subpatchers including externals, not standard Max objects? There might be something to check in that domain...
Another thing is about your Audio setup: are you working with scheduler  in Overdrive and Audio Interrupt on (I would recommend that in most cases)? Does the bug happen regardless of these settings? Improvements or not with a larger vector size?
Also, since you mention poly~, do you send~/receive~ signals across the poly~ barrier (not recommended)?
Sound like I'm fishing, but the problem is fishy.
It would be great if you could reproduce the problem with a small version of your patch, ideally with "just a bpatcher", or a couple of them. You could delete elements you feel don't matter, and keep the most suspect things (gui elements that refresh often, the processes with externals, poly~ processes), then see if you can trim down your patch to something small that would still show the problem.

thank you very much for your thorough reply! Definetily a couple of things to chase for, although like I said, the ressource usage isn't even 

 high so it seems weird. Anyways, I do use quite a few Snapshot~ and I will investigate lowering the reporting interval on all of those. I tried to find information on how taxing Snapshot~ truly is and couldn't find any.

The "freezes" happen in regular bpatchers and, what is even stranger, the main audio playback and processing that is happening inside a fairly sizeable poly~ keeps running just splendid, even though the higher ressource load definitely happens there. There is no DSP management in the subpatches affected by the bug.

I do use both Scheduler in Overdrive and Audio Interrupt and couldn't see any influence of these settings on the prevalence of the bug. Same goes for vector size. I use some externals but nothing crazy and none of it is in the parts of the patch affected by the problem.

An interesting avenue to pursue would be the send/receive that you mentioned because there is a 

 of sending/receiving in and out of polys. I was never aware that was an issue and completely getting rid of that basically means rewriting my entire core program, so if it is at all possible I'd hope to avoid it. Do you know why that is an issue/not recommended? And does that extend to 

send/receive or just to those at audio rate (with a ~)?

I will absolutely try to narrow the patch down and see if I can reproduce the issue in a smaller framework. It seems a bit tricky as the issue does not seem to have any clear cause and happens somewhat at random, but I'll see. It will probably happen after the holidays though.

Once again, thanks so much for your answer and have happy holidays, as much as circumstances permit!

thank you very much for your thorough reply! Definetily a couple of things to chase for, although like I said, the ressource usage isn't even that high so it seems weird. Anyways, I do use quite a few Snapshot~ and I will investigate lowering the reporting interval on all of those. I tried to find information on how taxing Snapshot~ truly is and couldn't find any.

The "freezes" happen in regular bpatchers and, what is even stranger, the main audio playback and processing that is happening inside a fairly sizeable poly~ keeps running just splendid, even though the higher ressource load definitely happens there. There is no DSP management in the subpatches affected by the bug.

I do use both Scheduler in Overdrive and Audio Interrupt and couldn't see any influence of these settings on the prevalence of the bug. Same goes for vector size. I use some externals but nothing crazy and none of it is in the parts of the patch affected by the problem.

An interesting avenue to pursue would be the send/receive that you mentioned because there is a lot of sending/receiving in and out of polys. I was never aware that was an issue and completely getting rid of that basically means rewriting my entire core program, so if it is at all possible I'd hope to avoid it. Do you know why that is an issue/not recommended? And does that extend to all send/receive or just to those at audio rate (with a ~)?

I would really try to avoid [send~]/[receive~] accross a poly~ barrier (just the signal versions) (but that would apply if you "hack" send-receive (not signal) to carry signals!)

One way to think about why is to consider that what happens in a poly~ is in a separate world, with a (potentially) different sampling rate, etc. The [in~] and [out~] are designed to handle well the paperwork at the poly~ border. Using [send~] [receive~] is a little like avoiding the border (and not paying the import fees): it might be OK, but it might result in problems. 

A related issue is that a [send~][receive~] usually doesn't add latency. If you create a signal feedback loop in a top-level patcher, it will introduce a desired latency of one signal vector. But accross a poly~ barrier, the latency could be hard to predict, and will depend on the structure of your patch (potential feedback loop in the audio network, etc.)

This issue is especially confusing since, over the long history of Max, a significant number of patches have been created using [send~] and [receive~] this way.

I would really try to avoid [send~]/[receive~] accross a poly~ barrier (just the signal versions) (but that would apply if you "hack" send-receive (not signal) to carry signals!)
One way to think about why is to consider that what happens in a poly~ is in a separate world, with a (potentially) different sampling rate, etc. The [in~] and [out~] are designed to handle well the paperwork at the poly~ border. Using [send~] [receive~] is a little like avoiding the border (and not paying the import fees): it might be OK, but it might result in problems. 
A related issue is that a [send~][receive~] usually doesn't add latency. If you create a signal feedback loop in a top-level patcher, it will introduce a desired latency of one signal vector. But accross a poly~ barrier, the latency could be hard to predict, and will depend on the structure of your patch (potential feedback loop in the audio network, etc.)
This issue is especially confusing since, over the long history of Max, a significant number of patches have been created using [send~] and [receive~] this way.

👆i like this description very much, perhaps too much, i had to envision the entire scenario in detail:

'you are coming from the country of sampling?' 

'have you received your amplitude-envelope vaccinations?' 

'very well, i will stamp your passport with a *CLICK* ...but suggest you make a trip to the VCA Pharmacy right over there should you ever start to feel erratic, thank you, move along...'

'you are arriving from the country of filtering?' 

[diplomatic mission: to shape the views of our countries towards a lighter disposition] 

'oh very good, a mission of peace no doubt.. though i see you are utilizing IIR: yet without proper limits to the coefficients this could be used as an explosive, however, because you have diplomatic immunity, i will overlook it this time, and simply stamp your passport with a *POP* ...but highly recommend you find some smoothing for those quick parameter changes... we are not heathens here! move along...'

The [in~] and [out~] are designed to handle well the paperwork at the  poly~ border. Using [send~] [receive~] is a little like avoiding the  border (and not paying the import fees): it might be OK, but it might  result in problems.

👆i like this description very much, perhaps too much, i had to envision the entire scenario in detail:
'you are coming from the country of sampling?' 
[yes] 
'what is your business in this poly~?' 
[just vacation and sightseeing] 
'have you received your amplitude-envelope vaccinations?' 
[no, sorry, i didn't see the need] 
'very well, i will stamp your passport with a *CLICK* ...but suggest you make a trip to the VCA Pharmacy right over there should you ever start to feel erratic, thank you, move along...'

'you are arriving from the country of filtering?' 
[yes] 
'what is your business in this poly~?' 
[diplomatic mission: to shape the views of our countries towards a lighter disposition] 
'oh very good, a mission of peace no doubt.. though i see you are utilizing IIR: yet without proper limits to the coefficients this could be used as an explosive, however, because you have diplomatic immunity, i will overlook it this time, and simply stamp your passport with a *POP* ...but highly recommend you find some smoothing for those quick parameter changes... we are not heathens here! move along...'

'you are arriving from the country of?' 
[divide-by-zero] 
'*ILLEGAL!* SHUT DOWN EVERYTHING!'

it took me a while to get back to coding and working on the project. Thank you so much for your thorough explanation, this makes a lot of sense and I had no idea this problem even existed. It's also unfortunate as the send~/receive~/poly~ combination makes for such a convenient way to generate discrete, indexed audio streams, but alas. I have put your suggestions into practice, as well as some general optimization.

However, it turns out my problem was indeed not at all a general audio processing stop but simply snapshot~ ceasing to function for no reason at all, giving the impression processing had stopped completely, as the subpatches were subsequently not outputting anymore. This problem has been reported here: https://cycling74.com/forums/snapshot~-dysfunction and in many other threads.

After some consideration I decided against burning my laptop and everything max-related that has ever been in my life, instead opting to replace my snapshots~ with meter~. That seems to be more reliable but the fact that this seems to be the best (that is, the only) solution at the moment is infuriating beyond belief. I will report further if I come across other solutions.

With that being said, performance could still see some improvement, especially pattr-interpolation causing some hiccups. I have seen some people use shark to get a more detailed view of what is happening within the processes, would you happen to know a windows-based alternative by any chance?

it took me a while to get back to coding and working on the project. Thank you so much for your thorough explanation, this makes a lot of sense and I had no idea this problem even existed. It's also unfortunate as the send~/receive~/poly~ combination makes for such a convenient way to generate discrete, indexed audio streams, but alas. I have put your suggestions into practice, as well as some general optimization.

However, it turns out my problem was indeed not at all a general audio processing stop but simply snapshot~ ceasing to function for no reason at all, giving the impression processing had stopped completely, as the subpatches were subsequently not outputting anymore. This problem has been reported here: https://cycling74.com/forums/snapshot~-dysfunction and in many other threads.

After some consideration I decided against burning my laptop and everything max-related that has ever been in my life, instead opting to replace my snapshots~ with meter~. That seems to be more reliable but the fact that this seems to be the best (that is, the only) solution at the moment is infuriating beyond belief. I will report further if I come across other solutions.

With that being said, performance could still see some improvement, especially pattr-interpolation causing some hiccups. I have seen some people use shark to get a more detailed view of what is happening within the processes, would you happen to know a windows-based alternative by any chance?

thanks, best

- snapshot~ works well if you bang it instead of using the built-in rate. I often bang it with a [qmetro] instead of [metro] if it's for low-priority events.

- with pattr interpolation, check how often you ask for an interpolation. That's very important. For instance, if you use a [line] object with a Time Grain of 20 ms (that's the default), you are actually asking pattrstorage to compute a new interpolation every 20 ms. Is it necessary given your application? That depends. In some cases, a much lower "Time Grain" for line is completely acceptable (i.e. sounds almost the same, but avoids hiccups).

A couple of thoughts:
- snapshot~ works well if you bang it instead of using the built-in rate. I often bang it with a [qmetro] instead of [metro] if it's for low-priority events.
- with pattr interpolation, check how often you ask for an interpolation. That's very important. For instance, if you use a [line] object with a Time Grain of 20 ms (that's the default), you are actually asking pattrstorage to compute a new interpolation every 20 ms. Is it necessary given your application? That depends. In some cases, a much lower "Time Grain" for line is completely acceptable (i.e. sounds almost the same, but avoids hiccups).

Subpatches GUI/MSP freeze in large project despite low CPU/RAM load