load pattrstorage preset at low priority, or other strategies?

    May 13 2013 | 9:28 am
    I have an issue with a Jitter GL performance patch. It's a visualization system with 8 render channels, each with over a dozen parameters. I'm using pattrstorage to store presets and fade between them. I'm experiencing big framerate drops when fading (from 50 to 20 fps). The stuttering is well visible and thus not acceptable.
    The stored parameters are mainly float numboxes. A couple are linked to pattr objects, most not (named objects>autopattr>pattrstorage). They feed into shader slabs. From testing it seems that even when disconnected from any slabs down the line the graphical updating of the numboxes causes a big framerate drop.
    Is there a strategy to not overload the scheduler while fading between presets? Like making pattrstorage recall at low priority?
    I am aware that GUI objects shouldn't be updated in real time, very careful to avoid that. But how to get around it when you need 'm for your system control and the problem surfaces when fading between presets?
    I already limited the rate of my midi fade control input to 10Hz. It only makes a marginal difference. I deferlow'ed the midi input and tried overdrive on and off to no avail.
    All input very appreciated. I'm gigging with this on thursday...

    • May 14 2013 | 7:20 am
      You can try to send to pattrstorage message"changemode 1". It should solve the problem . Let me know is it works.
      best wo
    • May 14 2013 | 8:01 am
      That's already been on all the time ;)
    • May 14 2013 | 9:10 am
      hmm, so maybe "deferelow" between variable and jitter object?
    • May 14 2013 | 9:41 am
      maybe sequencing the order in which the presets should load with help of pattrstorage "priority" message can be handy. it can reuire a bit of paching but becomes useful when there are many presets. i am not sure if it works with autopattr, you might need to add a pattr object to each UI you need
    • May 14 2013 | 10:01 am
      If you want to work with deferlow remember to turn "scheduler in overdrive" to 1 in menu: Options/AudioStatus.
    • May 14 2013 | 10:02 am
      > you might need to add a pattr object to each UI you need
      yes that's what I did, at least for now, with a couple of tricks around it. i'll post my solution asap, deadline rush right now...
    • May 17 2013 | 12:19 pm
      Here's my concoction. While certainly not a perfect or universal solution it worked in this case. The main point is to prevent loads of numboxes from updating in realtime while crossfading pattrstorage presets, which induces heavy GL rendering lag.
      The unnamed/unstored numbox feeds into the pattr object. When recalling/crossfading presets the value is sent out the left outlet in realtime (connect to whatever your controlling) and out the right outlet to the numbox at a very reduced rate (1000-1100ms). The speedlim rate is randomized a bit so all the numboxes don't update at the same rate. I have 80 of these in my patch.
      For the record, before doing this I tried all the obvious measures like setting changemode, deferlow, overdrive on/off, etc.
      Different ideas/techniques are most welcome!
    • May 20 2013 | 11:58 am
      I think it would be bit better to remove gui msg with "set $1" and put there "prepend set" instead. Is shouldnt be a big step forward but it always save cpu resources a bit. I would also put pattr object outside a make an universal object to slowdown pattr actions. Check the code below :)
    • May 20 2013 | 2:44 pm
      Did you confirm a (set $1) message box slows down processing like numboxes? Never considered that since it doesn't change appearance. Gotta test that sometime.
      I have my stuff sitting in an attraction for reusability and updating. The pattr inside is named with an argument to the object (using the #1 placeholder). I changed it a bit for easier posting here.
    • May 20 2013 | 9:21 pm
      yes, without gui msg code is faster, for me its 30% (MPB, OS X 10.7). Can you show me your solution/ abstraction in code? Also instead of using "random" you can add to speedlim this "#0". This way u can be sure that each pattr got it unique refresh time.
    • May 21 2013 | 8:57 am
      Here is the abstraction. Random is much faster than filling in 80 arguments ;) I send one global speedlim rate which gets an automatic random variation in each.
    • May 21 2013 | 9:56 am
      When you use #0 you dont have to fill it like #1, #2 and so on. #0 is uniqe number of patcher instance, the most personal number for your patcher friend. You don't have to fill it by your self. It is done automaticly during initialization.
    • May 21 2013 | 10:04 am
      Nice, didn't know about that one.
    • May 21 2013 | 10:16 am
      try this
    • May 21 2013 | 12:30 pm
      My point might very well be stupid, but if you don't see the ui objects beeing updated, won't it be faster ?.. i mean you could send all your flonumber objects a "hidden 1" message (iirc you can send a message to all instances of a class of objects inside a patcher, maybe with thispatcher) and even if they interpolate, it won't update the graphics and maybe thigns will be faster ? sure you won't see what's happening, but depending on what you're doing, it might not be a problem when you're interpolating presets... it's also a genuine question since i'm wondering if this could work.
    • May 21 2013 | 2:19 pm
      gonna try that...
    • May 21 2013 | 2:21 pm
      > yes, without gui msg code is faster, for me its 30% (MPB, OS X 10.7).
      I tested [prepend] vs (set $1) but didn't see any difference. Below 's the patch and attached the abstraction. AFAIK it shouldn't matter that the numboxes are hidden in the poly.
    • May 21 2013 | 2:39 pm
      @vichug : I had a look at thispatcher. Looks like it only operates on named objects. A bit too much work to go and name 'm all right now. I have a feeling it won't have effect. Numboxes hidden in subpatchers and poly's also still eat CPU cycles as if they were visible. I'd be happy to be proven wrong though ;)
    • May 21 2013 | 3:06 pm
      To send a message to all instances of the same object you can use universal.
    • May 21 2013 | 3:24 pm
      I think its good to keep gui in patch simply to be able to notice what is going on, to get better understanding of your magic multimedia situation.
      time factor: it seams that gui always consume more cpu than non gui elements. Below i put patch which can show you the difference - please open it in presentation mode.
    • May 21 2013 | 3:36 pm
      > To send a message to all instances of the same object you can use universal.
      Thanks for the tip! Now what is the message to hide a flonum...? (flonum hidden) doesn't yield an error but doesn't work either. (flonum hide) throws an error.
    • May 21 2013 | 3:42 pm
      > time factor: it seams that gui always consume more cpu than non gui elements. Below i put patch which can show you the difference – please open it in presentation mode.
      Nice test! The message way takes about double as long on my system.
    • May 21 2013 | 4:00 pm
      Btw, yet another route I've considered is to set the pattr's to 'thru 0' so they don't output their crossfading values immediately but bang them out sync'ed to the rendering qmetro, with a [change] on the output. Haven't built it yet but I give it little chance of success, given the pretty huge update latency I had to set in the speedlim construction. There would be up to 80 or so flonum's updating at rendering framerate (30-60fps, ideally even faster), depending on how many parameters change in the preset crossfade.
    • May 22 2013 | 12:55 pm
      I think it could be done bit simpler. I realize that we are trying to slow down pattr output instead of changing the way we query pattrstorage. So I think it's good idea to remove speedlim connected directly to abstraction and use line object to query pattrstorage. It is possible to change line accuracy so effect is same like with speedlim but you dont have to load dozens of them - one line is enough here. If you manipulate pattrstorage interpolation via knob/hand simply change ramp time to few millisecond and you are done. Example of line accuracy below.
    • May 22 2013 | 1:15 pm
      Well the problem then is that whatever you're controlling changes state at that very reduced rate. I already have a speedlim on the midi control that's going to my preset crossfader at 50ms. I'm not willing to compromise much more on that. The fade gets too choppy.
      The issue is not slowing down pattrstorage but the updating of connected GUI objects. Connected non-GUI objects (jitter, MSP, etc) should get their inputs at full rate.
    • May 22 2013 | 1:52 pm
      You are right but still in abstraction you post there is speedlim after pattr object, so it's chopped any way. With line solution you dont have to load 80 speedlims which is obviously better for cpu. You can set 20 ms limit on line. Results are the same but it should be faster. Also sending data to jitter objects much faster then actual frame rate sems pointless. Of course I would keep speedlim for gui objects.
    • May 22 2013 | 2:04 pm
      Ok, I see. The speedlim between pattr and the gl objects is actually obsolete because of the speedlim between my midi fade input and pattrstorage, though I don't expect a big improvement from that. Needs testing. I'll try making a demo/test patch asap that I can share here. Will make the discussion less hypothetical.
    • May 23 2013 | 12:38 pm
      Here's a test patch lifted out of a bigger project. Please forgive the idiosyncrasies.
      _pattrstorage test 2.maxpat is the main patch to open, the rest are abstractions/presets.
      rota_fx.maxpat (in a bpatcher) is the one holding the shader fx, GUI controls and pattrstorage. It's messy but the gist is that each flonum is linked to a patr.maxpat. Inside is a pattr with the speedlim construction we discussed above.
      In patr.maxpat I've added a pipe before speedlim because I realized that without it the first recalled value will output for all of the pattr's at the same time. The speedlim only kicks in after that.
      There are 3 parameters to the patr construction: update_rate : the base speedlim rate rate_spread : the random variation to the base rate pipe_spread : random variation to the initial delay
      The blue slider will fade between preset 48 and 47. The messages to line do it automatically.
      Hit the red toggle to start. When running without any fading I get 70fps constant. When fading fps drops down to between 20 and 50, depending on the update settings. The more flonum's update at the same time the lower it drops. I haven't found settings that keep it close to 70fps yet. (And in the actual project I have 25 more flonum's pattr'd)
      It's possible to set it so the GUI only updates after some seconds. It's then visible that the fps remains steady at 70fps until the GUI updating kicks in. That makes it clear that it's not the jitter processing causing the lag but really the GUI objects.
      The slowdown effect is also visible when making a hard preset change (by clicking in the preset object or changing the purple number box).
      Any optimizations are very welcome! Actually I hope there's just a toggle to enable somewhere that makes all of this totally obsolete...
    • May 23 2013 | 12:43 pm
      Just noticed the situation is particularly terrible with overdrive off. Drops down to below 10 fps.
    • May 23 2013 | 1:57 pm
      Btw, that test patch will not function properly if the abstractions are being edited (opened) while the main patch is running. Best to close all and reopen the main patch after edits.
    • May 30 2013 | 7:24 am
      Hey Wojciech, any ideas about that patch?
    • May 31 2013 | 10:09 pm
      I have got no time to study your patches but they do not work for me. I've got sth between 2 and 10 fps (OS X 10.7, MBP, i7) and yes, when I try to play with fade max almost freeze :)
      In attachment you can find my solution in progress, please let me know is it work for you.
    • Jun 01 2013 | 12:01 pm
      Hey, sounds like your gfx card can't handle my patch.
      Thanks for your solution. If I get it right the main difference with mine is blocking the GUI updates altogether? That should definitely alleviate the issue (if taken care of not updating all the objects at the same time when resumed). Though the question raises whether usability isn't impeded too much by just switching off the GUI objects.
      I wish the C74 whizkids would chime in on this... I feel we're constructing bulky workarounds for something that shouldn't be so complicated.
    • Jun 02 2013 | 11:20 am
      Yes, it blocks GUI updates altogether and line wraper helpes to set proper refresh interval for GUI and pattr objects.
      I have to say that I am not sure is this really a bug. Please noticed that all GUI object dedicated to be redrawn constantly got refresh attrubute - all dsp object like waveform~ meter~ and so on; also live.dial. So maybe this is wrong way of using GUI?
      I think you can also try to limit "numdecimalplaces" for float object - this is my last idea :)
    • Jun 02 2013 | 12:11 pm
      Not saying it's a bug :] It's indeed about finding a best practice for handling GUI objects in such a case. I've been considering building the control interface in openGl to get rid of the GUI objects altogether. Though probably feasible in a project with a clear plan, it would certainly take the spontaneity out of unplanned, trial and error development processes. Would be a nice challenge to come up with a system for this that still allows for quick 'n dirty prototyping. Gonna give it some thought...
    • Jun 02 2013 | 12:54 pm
    • Aug 30 2014 | 8:57 pm
      @DTR: i know this post and your performance is well outdated (hope it all went well), as will be any found solution - but you were wondering how to 'hide all flonums' in patchers.. the modus operandi is to 'send flonum hidden 1' to a 'universal'-object, 'thispatcher' only handles scripting names as opposed to object types. When working with nested bpatchers etc. the universal object can be placed in the host patch with an extra argument 1 to send messages to all sub patches too:
      toggle 1/0 -> prepend send flonum hidden -> universal 1
      best shgn
    • Aug 30 2014 | 9:00 pm
      Tanx, that should come in handy at some point!
    • Aug 31 2014 | 8:51 pm
      neither do I have Max on this computer to look at your patches nor am I experienced with Jitter, but could the 'priority' attribute of pattr objects be of any use to your purpose? you can view and manually edit all pattr clients' recall priority which determines the order they will update on preset change (interpolation too I guess) when sending a 'clientwindow' message to the parent pattrstorage object. alternatively you could send iterated messages to pattrstorage, and hereby setting the clients' priorities (priority subpatch::flonum[1] -10 -> pattrstorage); not sure if this works with autopattr too). if you set up a logic to define these priorities automatically, you could save these to pattr objects @invisible 1. the hierarchy only works in single patchers, not across nested bpatchers.
    • Sep 01 2014 | 8:09 am
      I just looked at the PATTRSTORAGE-XFADE-TEST.ZIP and the problem is with (blocks the CPU), use instead. I used 0 500 and 1. 500, connected to the trigger bang bang and it's all fine ! Peter Elsea has a tutorial on animation that explains the issue very well.
    • Jan 26 2015 | 10:17 pm
      Hi Nnimar, sorry I totally missed your reply. Just now I came back to this topic to post that we're in good company. Stretta has the same issue: https://github.com/stretta/BEAP/wiki/Modules-To-Do
      I tried your bline alternative but it doesn't yield better results for me. Which one is Peter's animation tutorial? Is it here? http://peterelsea.com/maxtutorials.html
    • Jun 03 2017 | 9:52 pm
      Any built in solution to this problem in 2017?
    • Jun 04 2017 | 3:35 am
      I am glad this thread was brought back to life... My current strategy is to just simply not support fading between presets - which is not a great strategy really...
      pattrstorage is an area which I hope gets some optimization attention in future updates.
      I would even be ok with a solution that doesn't implement fading - but can switch between presets (with ~100 attributes saved/recalled) within 33ms - even better if < 16ms.
      Also would love to get a message that fires whenever all values have been updated - or a way to synchronize the firing of the qmetro clock with the finished loading of a preset.