Announce: Embedded SQL inside MXJ


    May 06 2006 | 12:57 pm
    I've been working on embedding the HSQL database inside a library for MXJ. HSQLDB is 100% Java, and in its default mode is 100% plain file based, so it's pretty easy to wrap up and link in to Max's search path machinery. The result is a full SQL database with basically no installation or configuration needed, and easy access to the data files.
    It also seems to be robust. A couple of months ago I wrote an OSC sequencer object, using SQL to timestamp and store all the data (coming from some wearable computer gear which was putting out messages at 10msec intervals). Load and cue time were not inconsiderable on my TiBook, but it all held together. Worked fine on Windows too (and the data files are portable).
    The only actual MXJ object I have to release at the moment is something which takes in full SQL statements and puts out result rows as lists of Atoms; at some stage soon I want to rewrite the cassiel.registry object to be SQL-based, and also put out some example objects which do proper PreparedStatement JDBC (like the OSC sequencer, although that's a bit too complicated to serve as an example).
    Oh, and all this code should work, unmodified, on any forthcoming Max/ MSP running natively on an Intel Mac.
    Info/download at: http://www.loadbang.net/space/Software/ net.loadbang-SQL
    If there are any holes in the software or the documentation, please let me know. I've not zipped up the sources yet - will do so when I have a spare moment. Also, if you do use it and want to be kept informed of updates, drop me a line.
    I'm also keen to put together some more examples of utilities and applications. Anyone got any ideas?
    -- N.
    nick rothwell -- composition, systems, performance -- http:// www.cassiel.com

    • May 06 2006 | 9:33 pm
      Nick,
      This looks interesting. I wonder if you could say just a few words about how it would be applied musically.
      Cheers, Gary Lee Nelson TIMARA Department Oberlin College www.timara.oberlin.edu/GaryLeeNelson
    • May 06 2006 | 10:11 pm
      On 6 May 2006, at 22:33, Gary Lee Nelson wrote:
      > This looks interesting. I wonder if you could say just a few words > about > how it would be applied musically.
      I haven't got a clue.
      But it's not really a musical mechanism; it's infrastructure. It's no less, and no more, musical than the coll object, or the file drop- box, or ubumenu, or counter.
      An SQL database is essentially an extremely flexible storage structure, allowing sophisticated linkage and querying of data. I've used SQL databases of various flavours for years, so when I found a way of embedding a complete database inside MXJ, I jumped at the chance, even though I had no immediate application in mind.
      I've already written an OSC sequencer (asynchronous record/playback system - it can record and play at the same time, which I'm rather chuffed about), and SQL provides a pile of date and time functions, so it's possible to catalogue and timestamp data objects pretty easily. (I use a jit.cellblock for the OSC recordings, labelled with date/time of recording.) I want to replace cassiel.registry with an SQL implementation, and I also have a need for a robust, sophisticated, general-purpose sysex librarian, for which an SQL database seems reasonably well suited.
      Other than that, well, *shrug* - now that the machinery is there, I'm sure I'll find things to do with it, and it would be kind of cool if other folks did as well. As and when I come up with examples, I'll upload them. (I'd like to decant the OSC sequencer into a general- purpose data recorder and analysis tool, for instance.)
      -- N.
      nick rothwell -- composition, systems, performance -- http:// www.cassiel.com
    • May 07 2006 | 3:41 am
      This is really odd because I have been working on a similar idea. However, I was going to use a mjx java app with embeded mysql.
      The main thing that sprang to my mind was being able to create relations between different musical parameters and the pathname of a particular sound file. For example, you could have attributes such tempo, time signature, duration etc. So then you could search for all sound files with at 108bpm, in 4/4 which lasts for 4 measures. These sound file patchnames could then be loaded into a ubumenu or whatever you want.
      -Nick
    • May 07 2006 | 7:44 am
      Hi Nick,
      really interesting...
      > I've already written an OSC sequencer (asynchronous record/playback > system - it can record and play at the same time, which I'm rather > chuffed about), and SQL provides a pile of date and time functions, > so it's possible to catalogue and timestamp data objects pretty > easily. (I use a jit.cellblock for the OSC recordings, labelled with > date/time of recording.) I want to replace cassiel.registry with an > SQL implementation, and I also have a need for a robust, > sophisticated, general-purpose sysex librarian, for which an SQL > database seems reasonably well suited.
      Will the OSC sequencer be available, in the future ?
      All the best
      Alessandro Fogar
    • May 07 2006 | 10:13 am
      On 7 May 2006, at 08:44, Alessandro Fogar wrote:
      > Will the OSC sequencer be available, in the future ?
      No reason why not; it's just a case of finding the time to tidy everything up, document it, and so on. It's on my to-do list.
      -- N.
      nick rothwell -- composition, systems, performance -- http:// www.cassiel.com
    • May 07 2006 | 10:27 am
      Nick,
      very well.
      Thanks
      Alessandro Fogar
    • May 07 2006 | 10:51 am
      On 7 May 2006, at 04:41, Nick Inhofe wrote:
      > This is really odd, because I have been working on a similar idea, > except that I was going to a mjx java app with embeded mysql.
      I went for HSQLDB because it's 100% Java, and it can be rolled into MXJ with a couple of JAR files and be binary portable between Mac (PPC and Intel) and Windows; MySQL is C/C++. I didn't know about the embedded server, though.
      (My other problem with MySQL is that integrity constraints are off by default (or rather, the default table type doesn't support them), which I always found irksome.)
      > The main thing that sprang to my mind was being able to create > relations between different musical parameters and the pathname of > a particular sound file. For example, you could have attributes > such tempo, time signature, duration etc. So then you could search > for all sound files with at 108bpm, in 4/4 which lasts for 4 > measures. These sound file patchnames could then be loaded into a > ubumenu or whatever you want.
      Absolutely; SQL is all about fancy querying. (Well, OK, fancy updating as well.) It's pretty easy to formulate SQL statements to support taxonomies.
      -- N.
      nick rothwell -- composition, systems, performance -- http:// www.cassiel.com
    • May 09 2006 | 2:16 am
      Thanks for doing this! I was going to have to do something like this myself for a project that requires storing data from analyzed videos. Now I don't have to implement the whole maxdatabase thing. Much appreciated and nice handiwork.
      cheers, wes
    • May 09 2006 | 11:19 am
      On 9 May 2006, at 03:16, Wesley Smith wrote:
      > Now I don't have to implement the whole maxdatabase thing. Much > appreciated and nice handiwork.
      Let me know if the documentation isn't sufficient, or if seeing some source code (such as for the Noddy MXJ object) is a priority, and I'll do something about it. Also, let me know how you get on with it.
      -- N.
      nick rothwell -- composition, systems, performance -- http:// www.cassiel.com
    • May 09 2006 | 7:15 pm
      Sorry, I do not manage to install your packages to get your 'db-test.Noddy.mxt' to work, I keep on getting error: Could not load class 'net.loadbang.sql.mxj.NoddySQL'. It doesn't seem to be included in the packages.
      Would you mind clarifying the installation process?
      Thanks
    • May 09 2006 | 10:35 pm
      > Sorry, I do not manage to install your packages to get your 'db- > test.Noddy.mxt' to work, I keep on getting error: Could not load > class 'net.loadbang.sql.mxj.NoddySQL'. It doesn't seem to be > included in the packages.
      Hi Sebastian,
      That demo MXJ object should be in net.loadbang-SQL-1.0.jar - I quickly checked it now and it's there. Are you sure you have all the JAR's?
      -- N.
      nick rothwell -- composition, systems, performance -- http:// www.cassiel.com