multiple [coll]'s

    Aug 30 2012 | 1:31 pm
    Hello everyone !
    ive got strange feeling about the [coll] . It helps me to automate and process a huge amount of data . i can mix symbols ints floats . I can use [coll] with the same name in many places in the patch . this is very helpful as it SHARES the content over the path . i can even STORE all the data within a patch ,so i can use it AND reprocess it in another instance without loading any files . GREAT !
    the strange thing about it is that : it shares the content with other [coll]'s with the same name ,BUT the same time it will increase the ram usage . i can notice that also when im letting the [coll] to be stored within a patch - it will increase the PATCH FILE SIZE as well (i can prove it via - copy compressed - i can see that repetition of data) . thats really wired in manner of sharing data . it just copy its data .
    i do have problem with it and cant find any useful alternative for such surprise .
    im building "sampler with sample banks"
    im using [coll]'s to store huge amount of mixed data - its a sort of configuration file for my max patch . it also stores huge amount of paths to files on the disk (mainly WAVE AIFF files) to call them into a dynamically created buffers on the fly .This way im preparing some SAMPLE BANKS with dedicated settings (like individual tuning , start point loop etc) . I want to be able to change those settings in different instances , so i need to be able to base on one BANK FILE to start from and store these changes within its unique instance in my DAW . i cant care for saving banks with different names for every other use , i will end up with mess .and its actually out of the "sampler with banks" concept . my patch with [coll]'s (stored within patch) is getting over 10mb itself and it will increase as project progress . it takes only 500kb without saving [coll]'s inside its patch . Im not storing samples within a patch . im loading them dynamically as needed on the fly . i do care for memory use .
    do u think is there a different solution for such scenario as mine ? or why [coll]'s do not share their content but copy its data ?
    i cant use it :(
    gretttingss K

    • Aug 30 2012 | 4:27 pm
      Yes, if you set coll to save its data in the with the patcher then that data is saved in the patcher file.
      You can save the coll data as a separate text file. If that file is in your search path, then it will be read when the patch is started if you use the filename as an argument. [coll myFileName], for example.
      You've been using the refer message? coll objects that are referring to the same name have the same memory pointer. They don't copy any data.
    • Aug 30 2012 | 5:31 pm
      Hi Mike !
      thanks for your reply mate ! well yes ive used "refer" before . it does copy data into memory as many times as you refer "main" [coll] to the others in the patcher :/ . it does exactly the same as using the same coll names to "share" its .txt file . and yes ,ive implemented the whole system that retrieve data from the text file (as some sort of preset file ,bank file) .
      i wouldnt care about it all ,but its really getting BIG . im using [coll] to process that data in real time ,i need to have it in many places in the patcher, as u know it refers to the same file that keeps all the settings . and its really hard to simplify the architecture of my device , at the same be able to dynamically process all the data back and forth :/ . as checked - using the same colls in many places increasing memory usage
      and cant see alternative
    • Aug 31 2012 | 8:37 am
      The source code for coll used to be distributed with the SDK. This is no longer the case, and it's possible that coll might have been redisigned for Max5, although it is fairly likely that it's using the same model. Which was:
      Behind every coll there is an invisible xcoll object. Colls with the same name all use the same xcoll object, and the xcoll is where the data is stored. Still, each and every coll has its own data structure for maintaining stuff like the box position, color, patch cord connections, and anything else that is specific to the box.
      However, my instinct doubts that this is really a lot of memory in RAM. How much memory are you seeing being used when you add multiple coll instances? Any at all?
      Data storage is probably a bit different. When saving to disk, I'm not surprised if every coll instance writes the full complement of xcoll data into the file. You may have named colls in different patchers all referring to the same name, and the xcoll doesn't "belong" to any of the patchers. So the coll objects can't hope that "someone else" will write the xcoll data to the file. Each guy needs to look after it's own needs.
      Theoretically it might be possible to check if there is more than one coll of the same name, and suppress writing the content data to a file more than once. But that would require breaking the encapsulation principle that each object takes care of itself and the patcher doesn't actually know what each object does internally. Plus, who cares if the file size is a couple of extra hundred kB? Sheez, why should anyone worry about a couple of hundred MB?. Data is cheap compared to the days when we thought a 50MB hard disk was large.
      In short: what's the problem? 10MB isn't that big a patcher file.
    • Aug 31 2012 | 11:45 am
      first of all . Even if i sounds like - im not complaining . This all is a sort of brainstorming i do everyday .
      Hello Nicolas !!! thank you , that thread is very inspiring ,thank you ... anyhow i do not think that i need such advanced implementations inside my current project which is just a sampler with a "quick load area" for banks . :) its data process must be fast and simple as it is by using [coll] . this is just obvious and amazing object . i just didnt know its secrets as these that it will occupied memory multiple times only if its "embedded" as Ben Bracken introduced to me . This is still strange and not logic to me . because it mean that if you open the text file it will "share" data among other objects with the same name refering to only one that is loaded manualy. but if you store that data within a patcher it will be multiplied ,as other colls are treated as separated data even if contains the same :/
      Hello Peter ! thanks for your time and detailed elaboration !!!
      You are not surprised that the data storage acts different way ? . i am surprised because in every programming language you are able to create ONE ARRAY or data TABLE and refer only to this . ive also patched in different environment and i didnt have such unclear scenarios . as this - that when i load a TXT file it shares the content but when im storing within a patch it multiplies . when i spotted it i thought that - if it stores that data multiple times (as many of [coll]'s) then perhaps its also loading this way into memory . ive noticed that when data have been embedded . i was surprised . I dont see any problem why xcoll would not be able to know "what from where" to get as in the CODE youve got objects with the same name . it should store DATA one time and spread it where it should in the patcher . if there are colls with the same names then this data is dedicated to em . even encapsulated . if you load your patch - program anyway decode all the encapsulated patchers so it can be processed . or i just dont understand its principles .
      ive been testing the situation where all the [coll]s where placed next to another . embeded into patcher .saved . multiplied the size .
      But what do you mean by that xcoll doesnt belong to any of the other patchers ? do you mean other separate programs ? or patcher inside patcher ? or xcoll is a sort of global variable or something in this fashion that is floating between other instances ? like BUFFER~ with the same name ?
      yeah SIZE . well ,i do care for many reasons . First of all this project will be my "GOTO" device that i will use from 10 to 20 times inside one audio project . Im not a fan of the workflow that other companies offer to me with their devices . im a oldschool AKAI and MESA guy where ive been able to create banks on the fly . im not a fan of using Ableton's native sampler to handle and manipulate my WHOLE library (Ableton also likes to import samples into ONE directory when preparing your own banks - my god its a mess there and its doubling your library ) . So im forced to build my own device . And im far beyond building just for fun where i could experiment and forgive some sort of things in the name of having just fun . I would love to do it and do not care ,but i have no time for such things anymore :/ i need to consider the workflow , cpu memory and disk usage . And to be clear - ive started to learn MAX because other environments gives MORE problems and limitations considering sample manipulation and does more resource impact .
      soooo , One of my instances occupies over 10mb disk space when [coll]'s are embeded (without saving within a patch it cost ONLY 500kB ). it doesnt mean that when i load it into max or m4l it remains the same . max itself needs some momory too , so its around 25mb when loaded once . when i multiple my devices its getting bigger (becomes 500mb of "not shared" data in op memory). And im talking about just clear instance ,without any samples or such . There is one more observation - 30kb TXT files loaded into [coll] occupies around 100kb of memory as i spotted . because is interpreted as a BINARY code so it needs more chars when uncompiling ?
      much love guys ! K