if it has not changed since max 4, GUI obejcts are processed also when they are hidden. simply because they cant know wether they are visible or not.

you can easily test that by opening a subpatcher or offsetting a bpatcher. is stuff drawn or emtpy? :)

general rule is, when you dont want to see them, dont use gui objects.

Thanks for the response. To make sure i understand how it all works, i have a follow-up question: Since M4L is always in overdrive, the actual drawing of the graphics (as with my table) are processed on the low-priority queue and only get taken care of when there's nothing more pertinent happening, and therefore will not actually slow anything else down anyway, while the data itself (which is being compiled and analyzed in important ways via the table) remains high-priority, correct?

"which is being compiled and analyzed in important ways via the table"

sounds like the table is not just used for display but actual processing of data. in that case, it will slow down at the point of input/output from the table(or to put it more technically: data will be deferred to a lower-priority at that point). for more efficiency, you could process the data through 'coll'/'dict'(or even use peek~ to read/write non-signal/scheduler-rate into a buffer~) in similar ways as table, and only use table as a display for the info. ...you could then gate the table so that it's closed to any data or control changes while it's not in view(closebang/thispatcher/etc.) and this would save you a little more cpu as well... if a gui object is not redrawing its graphic, it doesn't take up as much cpu.

having said that, you can get away with alot in Max, so don't hesitate to try the less efficient things if they're easier for you, and just keep watching CPU... it'll get easier to notice where to trim computations.

roman's general rule is best, though! (less GUI also makes for more interesting live-performances in my humble opinion... nothing like seeing someone staring at a screen to take me completely out of the moment :p... although, if you're live-coding, and the audience can see a projection of your screen, that's a bit different... now i'm just rambling, teee heee! >:D)

there are people here who know it more accurate, but as far as i know, GUI objects (read: their drawing) kinda run in their own thread, i.e. they should not be more hungry when they receive messages in high priority.

you could, however, make it sure by putting a defer before each of them.

"run in their own thread....they should not be more hungry when they receive messages in high priority."

all true. (my words not so great here, but "data will be deferred to a lower-priority at that point" is where i meant the same... if using the outlets of 'table/itable' for something crucially scheduled, speed is not so great, otherwise, just using 'table/itable' for visualization and infrequent update/control is fine :)

you can get away with alot in Max, so don’t hesitate to try the less efficient things if they’re easier for you, and just keep watching CPU… it’ll get easier to notice where to trim computations.

Yep, that's basically the way I've been going about it. I've successfully hacked together so many things that i couldn't even have dreamed up before i started working with Max. This here's just a refinement of what's already a pretty well-functioning machine.

From what you guys say, it looks like i should convert the table to a coll. Shouldn't be too difficult. I'll need to utilize a dump, in order to determine the highest value in the array, which is something I've already experimented with when trying to clean it up previously. (See https://cycling74.com/forums/how-many-loops-before-a-stack-overflow )

Peek~ + buffer~ is interesting; something i hadn't thought of. It did occur to me a little while ago that the way a professional would go about things would be to encode as much data as possible in actual signals, playing to the strengths of an audio language (sample-based accuracy and such). In this case I'm not sure i'd be able to find the highest point in the buffer~. Any ideas on how to? I will however most likely convert another part of my device to signal rather than coll-based data storage, probably with seq~. (See https://cycling74.com/forums/the-best-way-to-get-sub-millisecond-accurate-timing-please-advise/ )

As always, thanks for the help

I joined a thread last week about the same topic. Looks like they are indeed processed even if hidden, but they spend an (even impressive) chunk of your CPU if they need to be redrawn every tot msec on your screen.

I think the best and easiest way to see the extra CPU your *visible* GUI objects are demanding is to minimize/hide the whole window-patch in you dock while it is running. Then, of course, try put it back to front window, so that they are visible again. CPU can go from 70% to 10% as you minimize it. And get back to 70% when you maximize it again.

When I use my Vj software in jitter and its running I send a hidden 1 message to all float and integer boxes (just to be sure) ;-)

Andro, how do you do that? Is it possible to use universal for it (somehow doesn't seem to work here)?

EDIT: OK got it....

FWIW, GUI objects will not paint when not visible, so this cost should not be present. However, they will add themselves to check to draw which has a small cost and they will process any incoming data.

Cpuclock is telling me that the process takes longer on average with coll instead of itable. Why would this be? Even though it works (slowly) in my test patch, I can't get it to work once it's implemented in my actual device. It's like it's too slow to work under pressure or something? It actually does work in my device but only in edit mode and only using the MIDPLATEAU subpatch... I was surprised to see coll understood the inv message in the test, but when it fails in the device this seems like it might be where it's getting hung up... Very frustrating. Any ideas?

(I keep trying to post my test patch but apparently it just makes the site freeze)

(i've been having trouble posting copy-compressed patches too)

"(i’ve been having trouble posting copy-compressed patches too)"

lemme see if i can post somethin random...

edit: so that works, but i always delete "pre" and "code" tags(site's never actually used them)... maybe that makes a difference, maybe not....

Test patch:

That did it Raja! Nice.

edit: so does anyone see the same slowness i'm getting with coll? In the test, version 1 takes ~.3ms with table and ~.9 with coll, while version 2 takes ~15ms with table and ~18ms with coll.

ya, i got the same results(actually 17ms with table and 18ms with coll on version2... maybe i've confused table with multislider... or maybe i confused itable with table... or maybe i just don't know what i'm talking about :D).

BUT! now that i see what you're doing, you probably don't need to worry about anything i wrote. you're not setting the table from outside, you're just dumping it(you could experiment with a deferlow right after the 'dump' message... but other than that, doesn't seem necessary to worry about timing-priority here...). if you were setting the table from the outside(anything besides the mouse) at a frequent rate(especially writing from audio-sig into the table, looping a new set of entries at a frequent rate), i would recommend paying attention to priorities, but otherwise, i'm not sure my advice applies as much here, haha :p

"It did occur to me a little while ago that the way a professional would go about things would be to encode as much data as possible in actual signals, playing to the strengths of an audio language"

Just wanted to mention: max is not a straight audio language, it is more open-ended, so ideally you'd want to play to the strengths of its open-endedness(use audio-rate processing only where appropriately needed, use scheduler-rate processing for all other musical-timing that doesn't need to be sample-accurate, and use deferlow(or qmetro is the same as metro->deferlow) for anything that is least crucial in timing(but much of the time, i don't feel like i even need to bother with that many deferlows in my apps... depends on what you do...edit: maybe worth mentioning, though, that i do use 'qmetro' as well which is the same as 'metro'->'deferlow'...) this is why it's best to experiment in order to get a feel for each context.

Hope i haven't confused.

2nd edit: Josh wrote: "GUI objects will not paint when not visible, so this cost should not be present."

in that case, it might be better to use table instead of itable(?)... table's GUI is only available when you double-click on it ;D

Actually I am setting points in the table in a fairly massive amount. Here's what's feeding into the table:

Every time a midi note is received roughly 20 numbers are sent into the SPREAD subpatcher, which turns each of those into roughly 61 (depending on settings) points to map. So now that you mention it, this is probably where the bottleneck lies. The whole issue is that I'm getting blips in my audio at each midi note. Currently I'm reorganizing my device so I can attempt to defer just this input and then pick up on high priority as soon as it's done. I'll let know how it pans out...

Still, would you say that coll should accept input faster than table? I wish i could get coll to work in the actual device. It seems like it should be a simple swap-out, but it's not turning out that way in practice.

Also, i tried table instead of itable a while ago and there was no difference.

"Still, would you say that coll should accept input faster than table?"

unfortunately, i can't say for sure...(cycling74 would know definitively about the specific performance between objects) i would think that all objects of the same kind(max, msp, or jitter), when receiving the same type of message will 'accept' input just as fast as any other...(i would think that table/itable/multislider would process input through them to their outlets slower than coll because they have to draw graphics as well... but it all depends on whether you can create the same functions(in order to create same exact input) at their inlets... but you wrote: " It seems like it should be a simple swap-out, but it’s not turning out that way in practice." so maybe that's a prob there...)

i don't say this completely as a wild guess, though. from studying the MaxAPI to create externals myself: https://cycling74.com/sdk/MaxSDK-6.0.4/html/index.html i can guess that messages of the same type will be received/accepted in the same way.... but then it all depends on what further processing goes on within the object.

(however, if this is an issue of 'accepting input faster', then maybe you don't want to worry about deferring priority, but instead write these values at sample-rate into buffer~, etc... i still don't know for sure what you should do, because i don't know what the rest of the app does... but... as we're peeling back layers of your onion, i fear asking you more about it because it already looks pretty complex and i don't want to get in over my head, hahahaha :D but seriously, hope it helps :))

Can't seem to get any sort of defer scenario to work. I think the best strategy might be to do like you say and use a buffer~, especially if there's a way to change the value of a range of it all at once, thereby eliminating the need for a SPREAD subpatcher multiplying the work 61 times. I'll have to delve further into the msp objects than i have thus far. Any place i should start looking (other than peek~ + buffer~)? I'll still need a way to determine the highest point in the buffer~ or it'd all be for naught...

Btw, my device is pretty complex but i doubt any of it is over your head. If you do want to see it in action and its entirety just email me at the address in my profile and i'll send you the .amxd, since you've been so helpful. I'm hesitant to post my unpatented secret million-dollar project up on the internet and just leave it there unguarded haha.

especially if there’s a way to change the value of a range of it all at once

yikes, not that i'm aware of :p (but... this would probably take the form of a list, which might have its own drawbacks similar to using uzi...)

so although i think a buffer~ would be a good thing to try, too... i've taken a closer look at your patches here, and i just realized you're resetting the 'range' and 'size' of the itable much too frequently(this is causing extra redrawing where unnecessary). use the 'change' object to catch and block repetitions... (because of these things, i'd go so far as to guess that uzi isn't as much your prob...) another small thing is that you might be using number-boxes unnecessarily(they are UI objects too). use 'int' object instead.

here's a small example of some things you could change to tidy up along the above ideas, just to set the range for itable(and then you'd want to do similar for 'size' as well):

you may even want to figure out a way you can do all this without having to constantly change the size and range(pick an absolute maximum and work from there maybe?)... that would make a substantial difference(changing the data of sliders/indices that already exist requires less work than redrawing the newly stretched shape of the graphics when changing the number of sliders/indices as well as changing the data at those indices...).

another thing, i remember now... long ago in max4, reading a thread that proved that it was 'multislider' that is actually faster than 'itable'/'table'... so you might not need to use buffer~(if you want to try buffer~, you'll need to range your values within 0 and 1(or if you're crafty, between -1 and 1, ...because it's audio)... usually done by taking reciprocals when working with whole numbers, or dividing by a known maximum for other-than-whole numbers, and then writing that value in, while taking the reciprocal of that when reading the value out(or multiplying by the known maximum).... you can still find the peaks, too, but it's all a bit more complex... if you can get your patch to work with simpler changes to itable or multislider, i'd recommend that instead at this point, will save you a bit of work and time, and may even be more efficient on cpu in the long-run...)

here's a patch that shows multislider works more efficiently. if you haven't seen this, it's Chris Muir's "Benchmark Timer" patch, very useful for comparing efficiency between various objects. here, you can see, even though the messaging into multislider requires a few more objects, it is still faster than itable:

(everything above the 'random' objects is from Chris Muir's original patch...)

anyways, those things may or may not help substantially enough to solve the cpu-choking you mentioned in the other thread. not sure. if uzi is still a prob, i might try what mattyo mentioned in the other thread(try a slower rate of bangs/counting using qmetro->bline or similar... if on the other hand, it's more essential to record histogram data as fast as possible, then try buffer~ instead...)... keep us up on whatever you find. hope it all helps :)

(P.S. i just looked and realized i deleted ableton a year ago because of frustrations with M4L, i guess i shouldn't try to help on m4l threads anymore, hahahaha(i usually just make standalone max or supercollider apps for live-performance and then use DAWs for pre-performance/studio purposes only.... it has had the added benefit of never running into any crashes for nearly a decade(at least not when my patches are finished and locked)... whereas with M4L, i've run into crashes for reasons i couldn't suss out as easily as in max alone). then there was this issue and other latency things i didn't like which killed m4l and ableton in terms of live-input-capture for me: https://cycling74.com/forums/can-i-get-the-audio-buffer-size-of-live/ all that is just to say(nothing political! i'm still excited to see what others do in m4l... but....)... sorry i can't look at your .amxd to see the full picture. and also, if you have the full version of max, it may be worth it to try and run your patch in max alone to spot some differences or probs specific to ableton.)

you’re resetting the ‘range’ and ‘size’ of the itable much too frequently(this is causing extra redrawing where unnecessary). use the ‘change’ object to catch and block repetitions…

Oh man! How did I not catch that! I added [change] and it took care of it. Or maybe i should say it mostly took care of it. I'm still getting a blip intermittently, but it only seems to be when the command to reset everything comes through the [r ---reset] (which is something I thought i took care of a long time ago by defering the 'clear' message to the table, but apparently not.) The only possibly expensive things happening at that time are the clearing of the table and maybe clearing the [histo 100000](?), which i tried deferring also, but no luck. I'll have to look further into it, but it's a huge improvement!

Multislider still may be the way to go, although i might still have to resort to a dump method of getting the address of (i should clarify that what i need is the address of and not just) the highest point, as i'm not seeing a replacement for the inv message in the documentation, and that would slow things considerably. BUT, the normalize message is very intriguing; it could be very useful for something else i have been looking to do in the patch for a long time...

No unfortunately right now i don't have a full version of max, but i should mention that these blips only give me troubles when i run the device normally and not when it's in edit mode, so that's how i've been averting the issue all along. It works but it's annoying.

So yeah, needless to say I'm glad you are still offering help in the m4l forum! i'll post back if i figure out anything regarding my remaining blip.

no prob, glad to help. i think your remaining prob is probably mostly to do with sending 'clear' to itable, not so much histogram(checkout an itable by itself, with 2000+ values, draw into it with mouse, then send clear... it chokes a bit before it's able to clear... you might even have better luck using uzi to bang 0s to each of itable/table's indices... if you use multislider, you'd have to do something like this to send 0s to all sliders because i don't think there's an equivalent 'clear' message...)... then again, maybe you don't need to clear the itable at all(just clear histogram, since what comes out of histogram determines the range/size/data of itable anyways)? but i just realized too... you definitely want multislider instead of itable/table. multislider has the 'maximum' message which finds the peak for you ;)

I haven't looked at this is much detail, but I thought I'd point out a couple of things if you are interested in trying it with [buffer~] and [poke~] or somesuch...

1. You can set a [buffer~] to hold numbers outside of -1/+1 by sending it the message "format float32". Then you're just storing 32-bit floats

2. have a look at [mxj buf.Op]. you can query sum, average, max, min, etc. you can also effectively change the range with the 'multiply' message.

Don't know if this is useful, just thought I'd let you know...

\M

Hello !

@SUN i did not read everything on that topic but i get the idea of the problem ... I have a question ! ... is it crucial to represent "exact" data visually in your project ? visual approximation of real content is not a deal here right ? im thinkin of dividing your data cells into fixed amount of rows and interpolating its output as a representation ... unless u want to draw on top of it.

#EDIT im also wondering if it is not an overkill trying to represent thousands of rows in a lets say 200px width rectangle ,where u cant display those thousands of rows anyway. maybe its unnecessary effort trying to display exact content ?

1. You can set a [buffer~] to hold numbers outside of -1/+1 by sending it the message "format float32"

ah, i shoulda tried this long ago :p my bad. (i guess the 64-bit float world of gen~ is a bit too convenient for me sometimes) thanks from me, too! :)

Solved!

then again, maybe you don’t need to clear the itable at all(just clear histogram, since what comes out of histogram determines the range/size/data of itable anyways

At first i thought you might be right about this, but in my case only around 1220 (20 * 61) of the (usually) around 8000 indices get updated with each new note and the residual data in the table could easily mess things up. No matter, as it didn't alleviate the problem anyway.

Spending more time with it, i discovered the remaining blip wasn't actually happening at the time of the reset command, but with the first midi note after it. This was because the first note brought with it a flood of new maximum numbers to bound the table with, almost as if the [change]s had not been inserted. All i had to do was give the table a fixed width and it went away.

… is it crucial to represent "exact" data visually in your project ?

Visually? Not at all, but i do need it exact for the sake of analysis. The whole reason i'm using an itable (when it isn't even on the UI) is basically that it understands the 'inv' message. Up above i wasn't as careful with my words as i could have been, but what i need to find in this data is the index of the highest point (not the value of it). As my test patch up there demonstrates, sending the table a 'max' message to get the peak value and subsequently an 'inv' message to get the index of it is dozens of times faster than using a dump method of getting that info. With a dump method i was getting blips on every midi note. It would seem that the go-to non-UI object for this purpose would be coll, but it doesn't understand 'inv' and, according to my test patch, seems to be even slower on the dump. Although apparently it's faster at setting indices

1. You can set a [buffer~] to hold numbers outside of -1/+1 by sending it the message "format float32". Then you’re just storing 32-bit floats

2. have a look at [mxj buf.Op]. you can query sum, average, max, min, etc. you can also effectively change the range with the ‘multiply’ message.

Thanks for the info! I would like to compress the data (on both the x-axis and y), but since itable has only integer precision, that would mean corrupting it. 32-bit precision would be far more flexible! And it looks like [mxj buf.Op] can normalize and its max message finds the index as well as the value.. So i must say eventually buffer~ is where i wanna end up, but for now it's working and that's what counts! Thanks for throwing ideas back and forth; it's really helpful to troubleshoot this way.

Hi Sun !

Ok ,so u dont need to "see" anything . You are about to find the peak value in certain period ? all the rest of the history doesnt matter at all then ? And what index represents actually ? is it just count of an event or it has specified role in all that "system" ?

This is long thread ,so maybe it was discussed already as what comes to my mind ... is that u dont need to fill so much data anywhere ,i would just look for last highest value on the fly . i could imagine it like this ...

im sorry if this is out of the scope ,its just what came to my mind after your last "description" . maybe i should read all the posts , but seems to be late as perhaps you just solved your problem already . ok , have fun !!!