Need advice on data structures

Oct 29, 2012 at 11:17pm

Need advice on data structures

I’m working on a large patch that requires data structures that are more complex than anything I’ve dealt with in Max, and am looking for some advice from some of you more seasoned patchers.

Basically I need to be able to create objects containing arbitrary key/value pairs that also allow me to nest objects inside of them (and objects inside of them, etc), ultimately resulting in a tree-like structure that will have (hopefully) interesting musical/visual results. I also need to be able to add/delete/edit any value at any layer dynamically, save the state of any of these objects for future use, and cherry pick individual objects at any layer to parse and do sexy things to. Also, performance is a concern as this component will be used pretty heavily in the overall patch.

After digging around a fair bit, here are some options I’m considering…

*dict*

On the surface this looks like exactly the kind of thing I need and was a big selling point on upgrading to Max 6. That said, after messing around a bit and reading forum posts it seems pretty convoluted to me. Nesting dicts seems difficult/clumsy, as does referencing and altering individual values at different layers. I may very well just not be educated enough on how these work but am yet to be convinced this is the best option or that it can do everything I need it to. I’d like to use this if possible since it’s native and seems to be intended for what I want to do.

*JSON*

My Javascript isn’t super but this could potentially be a great project to start changing that. The idea being a custom Javascript parser that talks to Max and does everything I need it to and saves these objects out as JSON files. Also open to out-of-the-box solutions but have yet to come across anything.

*coll*

The idea here is that I’d serialize these objects and write a parser that would dump everything I’d need into a coll(s) that I could then edit at will. This is more in my wheelhouse than other options I’m considering but sounds pretty messy and error prone (and fun). I’m guessing there are better ways to do this than what I’ve sketched out.

Those are the big three I’m considering right now but am totally open to anything at this point, even if it requires significant learning. This is a system I intend on using for a while and want to do it right.

Thanks much in advance,

-jic

#64996
Oct 29, 2012 at 11:33pm

Hi,

I also might suggest you XML as a structure. I implemented three XML parsers in my library (see http://cycling74.com/forums/topic.php?id=42930 ), two of them, [sadam.rapidXML] and [sadam.dom] are the ones that you could use for this project (I suggest the rapidXML as that is way much faster than the other). You can build XMLs from scratch with it, save/load them to disk, also serialize them as Max messages (and later load them from this serialized form). You can also query branches of the XML structure and edit/modify every single element (and attribute) individually.

Inserting a whole branch into the DOM tree is something that you can achieve with the object, however, you’ll need to code a lot to make it work (it is not supported by default, but since you can query sub-branches and you can insert new elements at arbitrary points, the mass-inserting problem can be solved with some extra patching – for example, by using a second rapidXML object that would temporarily store the branch and then feed that back to your main storage object).

However, I need to warn you that the use of rapidXML for very complex data management can be quite complex to program. On the other hand, I got very good feedback from various users regarding the performance of the object itself. :-)

Hope this helps,
Ádám

#234412
Oct 29, 2012 at 11:45pm

Hi Jic,

I believe the answer to your questions is FTM ( http://ftm.ircam.fr/index.php/Main_Page ). It is a fantastic set of externals from the good guys at IRCAM’s iMTR group.

You get a dict. object which is very straight forward and powerful (no idea how it compares to Max 6′s dict since I’m still on version 5). It is extremely efficient as all operations are done ‘in place’ meaning they access the memory address where your data is stored rather than copying it and sending the output somewhere else. Amongst the highlights of this library you get:

FTM message object. Most of the things you can do with Max’s message object plus the ability to execute operations in a single object. Calling variables and objects by their name and operate on them remotely without patch cords.

[fmat]. A floating point matrix object that has some advance commands to make all sorts of operations on matrix data.

[mat] object which allows you to nest other mats, fmats or dicts within a single matrix.

Gabor and MnM. Higher level sub libraries to do FFT and complex matrix operations.

The FTM email list. A great community that is low traffic and quick response from the devs.

No idea what your actual project is all about but from your description it sound like that might be just the right thing to use.

Join the list and we will be happy to help you getting started.

– Miguel

#234413
Oct 30, 2012 at 12:58am

Thanks to the both of you! I’ll check both of these out over the next few days to see what works. Much appreciated.

#234414
Oct 30, 2012 at 1:14am

I believe max also has sqlite built into it; but i cannot advise on it as I’ve never used it, but it is there (and accessible).

#234415

You must be logged in to reply to this topic.