[sharing] config file to coll parser

Zachary Seldess's icon

Hello Maxers,

For the past 14 or so months I've been managing, mostly on my own, several large multi-channel audio systems within an even larger visualization research facility filled with smart people who have their own areas of expertise and who do not know Max/MSP/Jitter and are not especially familiar with real-time digital audio processing.

As such I have become increasingly enamoured with the idea of using simple config files to dynamically build and configure the various custom audio Max apps installed throughout the facility. Config files that are easy to understand, easy to teach to non-audio-centric, non-Max programmers, and that allow me to make meaningful functionality changes to the apps/patches remotely without needing to view and mod a patch. It doesn't work in all contexts of course, but I've found this approach within my own context adds a significant amount of convenience and fluency to my workflow and creative process. So I thought I would share, in case others find it useful, even though I'm sure it's an idiosyncratic solution - but hey that friendliness to idiosyncracy is one reason we all love Max so much.

The abstraction (z.parseConfig), as well as helpfile, and a simple Jitter example, are attached (put it all in your Max search path). You can also download it along with my little collection of abstractions here: http://www.zacharyseldess.com/z.abstractions.html

Here's a basic description of what's happening:

z.parseConfig parses a simply formatted text file and stores the contents in a variable amount of coll objects, using predictable naming conventions for easy use and reuse within a patch. z.parseConfig requires two arguments: a 1st argument specifying the text file that will be parsed, the 2nd argument specifying a unique string to concatenate to the beginning of all coll names created via the parsing process (this allows you to use the same config file multiple times without namespace confusion).

The config file format supports code blocks embedded within codeblocks (in a pseudo-OO parent/child manner). A unique coll object is created for every block of text placed within left/right curly braces in the config file, and named in a predictable dot syntax according its own code block name and the names of all of its parent code blocks. All data is stored in coll objects, so it's all global, but the naming convention lets you conceptualize in parent/child relationships. Since coll names are predictably autogenerated within the patch, it's easy to build patches ahead of time without knowing what the precise values within these "namespaces" will be. Each generated coll object, in addition to storing the fields specified in the config file, also contain a "children" field which lists the names of all "child" colls (that is, child blocks within the config file that become coll files with . names). There is no limit to the amount of parent/child relationships. This fake parent-child awareness lets you do some pretty cool stuff (IMHO), for example using poly~. The abstraction populates an embedded subpatcher with all of the generated colls, placed visually according to their "namespace" hierarchies (this is all shown in the helpfile).

Fields within each config file block are assigned values with the traditional assignment operator '=' (making it easy for a non-Max user to edit and understand, vs the coll syntax). All blocks must be provided a name before the left curly brace. Any fields defined outside of all blocks will be stored in a "global" coll object (again, everything's global with coll of course, so this is just a naming convention).

The automated naming conventions, along with each coll's 'awareness' of it's children, etc. allows for some pretty cool things, such as deeply embedded and dynamically built poly~s within poly~s within poly~s, etc. Again, it also makes possible the creation and editing of a single, easily understood text file for more complex projects, that requires minimal training to manage (i.e. non-Maxers).

There's more detail below, but if you're still reading you might want to just look at the attached patches at this point...

Here's an example... let's say we have [z.parseConfig sample.txt test] in our patch.

contents of sample.txt:
--------------------------

// single-line comments work (something I've always wanted coll to have!)
// some global fields
blah = bloop; // 

--------------------------
end of sample.txt

Banging the object in our patch will generate the following coll objects:

    [coll test.topLevel1] - with the following contents:
        loop, 1;
        file, cherokee.wav;
        speed, 2;
        children, null; // if no child blocks exists within this block, children will be set to 'null'

    [coll test.topLevel2] - with the following contents:
        children, test.topLevel2.file1 test.topLevel2.file2;

    [coll test.topLevel2.file1]
        loop, 1;
        file, cherokee.wav;
        speed, 0.7;
        children, null;

    [coll test.topLevel2.file2]
        loop, 1;
        file, rainstick.wav;
        speed, 1.1;
        children, null;

Additionally, z.parseConfig will always generate three more coll objects to store:
    - all "global" fields
    - the names of all top-level "parent" blocks
    - and a complete list of all namespace blocks and their children associations.

So for the above example, the following coll objects would be created:

    [coll test.global] - with the following fields
        blah, bloop;
        blap, blub;

    [coll test.parents]
        parents, test.topLevel1 test.topLevel2;

    [coll test.children]
        test.topLevel1, null;
        test.topLevel2, test.topLevel2.file1 test.topLevel2.file2;
        test.topLevel2.file1, null;
        test.topLevel2.file2, null;

Hope someone else finds this as useful as I do. Would love to hear any comments or feedback.

best,
Zachary

1278.z.parseConfig.zip
zip
Zachary Seldess's icon

I bet no one bothered to read that long post - I don't blame you ;)

Basically, the abstraction is just a nice way of getting data from a single text file into multiple coll objects in your patch without having to mess with countless different text files, the coll file syntax, or a lack of commenting functionality.

I've just corrected a few things, and added C-style mult-line comment filtering (both single and multi-line filtering can be enabled/disabled as an attribute/message). If you find a kink in the behavior, let me know. You can download the updated abstraction at http://www.zacharyseldess.com/z.abstractions.html

At some point soon, I'll port this to a JSON-to-coll parser...

Relatedly, I added [z.minify] to the collection, which filters out single/multi-line comments and superfluous white space from a text file.

Attached is another example, simpler than the one provided with the abstraction, that doesn't utilize embedded blocks, but I think it still shows the human-readable benefits to using a more intuitive config file syntax, vs coll file syntax (especially when other people are looking at it).

best,
Zachary

1294.AudioMonitors.zip
zip
Ben Bracken's icon

Very cool Zachary, thanks for sharing!

-Ben

yair reshef's icon

great help Zachary