myUniverse project // Statements & questions after some explorations

    May 28 2012 | 8:37 pm
    Here are some conclusions, some questions, some feelings for my system myUniverse. First, I won't use COSM. It doesn't fit my needs and extensibility doesn't seem to be its strong point (which doesn't mean it isn't a good system!) In my visuals & sounds & spaces research, I also need to build my tool. It is a kind of pre-requisites and those gorgeous artists you are will understand that, I know. Second, I digged some parts, not all, because I cannot dive between to have that global vision we all need at that point.
    Thanks for your time.
    Little abstract before
    I'm building a 3D based system in which I'll placed objects. Each object has visuals & audio properties. All audio will be made using an external system (SC probably) I'll use the system to compose & play my music/visuals : it means I'd have to build a light gui/system to place objects, move them, change their properties and to save the whole parameters, to retrieve them too at load time.
    What I really need I need to have a system able to save all parameters & retrieve them from presets (a preset is call a map to figure the spaceness of the concept) I need to be able to build efficient visuals new objects to make the system evolving (not at runtime, I mean, prepare new objects, create maps Those objects would be easily instantiable and could be queried for their attributes & properties (even custom ones)
    The system designed following this would be upgradable & totally open for me to create more visuals modules/bits to use them in this map, or that other one. I'd have a lib of objects (visuals only, visual + audio, etc) that I could use (=instantiate & play) in some compositions.
    I'd need a way to place objects in my 3D map and to change their properties. This part could be done manually (I'd instantiate not only abstractions but some numbox & lists too and I'd tweak that in the patcher... UGLY and unmaintenable) But I'd love to do that on a centralize way and I could even make a little GUI for that. I would like to be able to select an object in the map, to have the GUI filled by the properties of the object selected, tweaking them etc. This is important and would involve a lot of message between the central stuff and the bits/objects.
    Some bits I explored
    Here are some ways I explored in the previous days. Of course, I didn't dig them to the end, because it would be impossible at that point.
    Each way involve: - a core central system which build/call/instantiate the other part, take care of restore/save map, propagate messages to each objects, request objects for their position, know about the position of the user/cam - some bits instantiated for some maps which would be in charge of visuals & which would message the core to trigger audio - a way of work (develop but compose too) ; this part is very important.
    -1/ JAVA as the core of all stuff In that case, I'd build the whole stuff in JAVA. It means I would have to translate my jitter bits to JAVA, creating them from scratch at map load time etc. UGLY. And probably not efficient at some point
    -2/ JS loading abstractions In that nice case, I would "only" call & place previously made abstractions in my patcher at load time (and during my composition sessions) It means I would create powerful little visuals bits using jit.gen, & more . I would have a bunch of objects like that and I would load one of this type, or this another type in my maps. The BIG problem is the communication between the core & the objects.
    Indeed, I'd need to create all getters / setters using a complex system of messnamed/receive/send objects. I would also have to build my abstractions AND the wiring of them. Maybe, a full system of receive/send would save that part ... at least. Of course it could be simplified... I mean, standardized according to my standards. For instance, all objects would have to be informed about some master clocks.
    -3/ JAVA loading abstractions Same than before but with JAVA.
    My questions
    Even in the mood of my words, it seems obvious I'd prefer the 2 or 3 parts. But the problems involve with the objects communications etc seem to be annoying.
    What would you advice to me about the way to follow ? What would you make for the messaging system ? What would be the most efficient between to instantiate a lot of abstractions (one per visual objects, some object being versus to create all encapsulated in JS or JAVA ? What could be used to make a GUI without to have to rebuild ALL each time I add an object ? (I thought about an external system but ..)
    It makes a bunch of stuff. Things begin to have shapes in my prototype patchers, in my notes, in my head. Even if you have one word for a particular part, it would help me a lot. You ever helped me a lot since 4 years on this forum so I'm asking some help here.
    I'd be very interested by a global vision about my system. Even in several words.
    Especially about the architecture core+bits (where bits would be those famous objects in my 3D maps), and about the GUI to interact with it at composition time.
    Last point: I have to begin, choosing one way but I know I'll refine & sharp the system in the next months, years. Maybe, some parts (GUI, communication system with external) will be translated in C external (using open frameworks for instance) for efficiency. I know that. I mean, I'm conscious I won't build THE system that fits at the first hours of code. But this step-point is important for me to really begin to code.

    • May 28 2012 | 9:41 pm
      Have you had any sleep in the last week? ;)
      Sorry to say I can 't give any advice at this level. It stops at pattr, preset and coll text files for me...
      But I am very very curious to see how this works out! You'll have my support with issues closer to my expertise.
    • May 28 2012 | 10:00 pm
      Agreed, get some sleep! kidding...stay awake and keep working! This sounds very ambitious and I'm very interested in the outcome. I assume you've seen the video in the Interview with Tarik Barri?
      His "Versum" system looks mind-boggling and sounds a bit like your system, maybe. I know he used several languages together to make it all work.
      I say if you're comfortable and enjoy working in the code-based side of things, that might be helpful in keeping things clean. Personally I'm just OK in javascript and not skilled in Java or C much at all, so I do most everything in Max. For this project I would use an abstraction for each object and have every parameter you might ever use included in it. Activate as many as you want (stackable bpatchers are usually my preferred method to deal with a lot at once), and use a master [pattrstorage] to save and recall any settings you want. If the bpatchers are small enough you could have banks of them (like 8 on the screen at once), and have these in their own bpatcher too, then stack banks on top of each other. Very easy to switch around with the keyboard or a [tab] selection for the "layer".
      Lots of ways to do things, of course, but this could be a good direction. I would also not instantiate/destroy instances, I'd just load a ton of them and use active/inactive for both the GL drawing and the audio. I don't think processing would be an issue if you do that, but then again, maybe you're thinking about thousands of objects... :)
    • May 29 2012 | 7:39 am
      dtr, thanks a lot for your support. I would have needs with pattr & preset. I probably have to design an xml storage system, but I won't go there if I can do all with native objects. I didn't check Dictionaries yet... it seems very powerful
      seejayjames, thanks for your answer. I even met Tarik at the m4_u event in Leicester. I have been invited to present something but finally I only played music (which was very cool) Indeed, I could take benefits of JS loading ALL at the loading time. and play with active/inactive stuff. This would be the way I could keep in mind and I could begin like that. The idea of making different types of objects, with a kind of standards (like "masterclock xxx" messages parsable for any objects, "requestposition" to make the objects sending "position ID-object x y z" to the central core etc) could be the way to go. I'm just afraid about performance BUT, anyway, even by instantiating all in the code (JAVA I mean), it would be all the same. About your idea of bpatcher. I would not go there because in my system, I'd only have to control things remotely : I mean, the central core would be able to communicate with each objects and I would plug a GUI and some communications systems to the core too (for OSC, MIDI if required etc) It means, I could even create a lot of subpatchers in the standard template Patcher. One subpatcher by type of objects. NO Wires, only send/receive more or less specialized (to have global communications & more specific com too)
      About the number of objects. I would have around 6 or 7 type of objects (already sketched here) . No more But of course I could have a lot of objects in my 3D world at the same time (inactivated if required of course) A lot means... I don't really know. If I think about one piece I made with a boring linear DAW I'm using everyday ... let's say ... I would have around 100 objects MAXI and around 6 or 7 activated at the same time (considering 3 or 4 visuals + audio , and some other for pure visuals fx for instance)
      I feel the design becoming more precise. I just hope not to make a strong walk in a wrong direction.
    • May 29 2012 | 3:43 pm
      I'd say that if you're comfortable in approaching this in a procedural language--especially the data portion--that's really the way to go. You can approach things like this in Max, but after a certain level of complexity, it gets to be a real pain. There's probably a reason why the object inspectors are no longer written in Max... (well, probably many reasons, but complexity and maintenance probably factor in)
      Personally, I would do it in Java because that's what I know best, and it's pretty easy to get things happening. If you're more visual, you can even use the Matisse GUI designer in NetBeans to mock stuff up. MVC is already implemented in Java and it works. This will make your life substantially easier, especially if you have functions (e.g. orbits...) that are altering said celestial bodies. I would say that the biggest knocks on Java in Max are that you can't do it within the patch (has to be in its own window) and the load time is a bit slow. IIRC there is an OpenGL extension to Java, so it's possible that you could even keep the rendering in Java. The bigger slowdown from Java comes mainly from memory bloat, but it doesn't sound like a throw the kitchen sink type project, either; in tight loops, java performance isn't awful (~1.5X that of C), and I've used it to mock up C code since it means not having to reboot Max to test it.
      Also, Java has a good set of generic collections, things like HashMap, EnumSet, ArrayList, and they can make your life so much easier. (example, you can define a custom sort function that will sort the list by activity/inactivity). The one thing I would check early is the latency via OSC vs. messages from Java.
      Communication: I would highly recommend OSC. That way, if the load becomes too big on one computer, it's relatively painless to add another as a mini-render-farm for audio/video. Also, you get hierarchical namespaces, which is a very good thing, and SC has native support.
      A fun trick in surround is to apply very small (often single-digit) frequency shifts to the different channels to provide a weird spaciousness/detune.
      Anyways, my two cents.
    • May 29 2012 | 4:06 pm
      Hello Peter and HUGE hugs for your huge answer.
      JAVA I'm not that killing expert, but I can read it and write it almost correctly.
      I'd like to keep Max as the central core. Especially because I'd extend the system to sensors and physical world one day and I feel well with that part in max. Of course I guess it can be done in "pure" JAVA but.. I'd prefer to do that. But the main reason for Max is Jitter. I begin to make nice little things with it, simple, fast & aesthetically ok for my requirements. I'm sure I will make great things quite soon (soon could be months, but ... yep, it is soon)
      What would you think about instantiating all Jitter stuff in JAVA VS calling already made patcher/abstractions ? Making ALL in JAVA is an approach I could feel well BUT it would mean I'd have to sketch ALL using Max, then to translate ALL in JAVA (instantiating JitterObject s, connection, texture...) all in code. And I'm not sure about the performances compared to directly loaded (even with JAVA) abstractions already made by patching, in the patcher.
      As you mentioned MVC, I feel the difficulties already around messaging. My system seems complex at first sight, but indeed, it isn't really. I'd just need to be able to instantiate & track a bunch of objects from a central CORE which would take care of activate/inactivate things, to broadcast messages like masterclock bang etc.
      I played today with JS & some abstractions (drawing or not OB3D in the opengl renderer ). JS calls & places a bunch of abstractions in the patcher (keeping references by using arrays which sucks totally if I want to only remove one abstraction not at the end of the array... basicaly)
      In order to have positions of ALL objects I have to send them request (what a "noise" ... I mean what a lot of data) But I could also use the JS core to move them, which would mean the JS core would have everytime their position updated without requesting them... nice tip.
      This would mean I'd have all calculation made in JS
      Anyway, your answer is very interesting for me. Probably, I would use JAVA to replace my actual JS... but I'm totally stuck because I didn't even succeed in calling and placing an abstraction in the patcher with JAVA ... Jesse tried to help me there but I'm probably tooo nOOb to understand :-/
    • May 29 2012 | 4:34 pm
      I haven't done scripting with Java, but it's also one of those things you probably just have to get "over the hump" with and then it'll be fine. Have you gone through the MXJ api? I think there's some stuff on scripting in there, IIRC. The important thing to know is that MaxObject is really just a way for your classes to interface with Max, and not all of your classes have to subclass MaxObject; it's just the ones that directly talk to Max. It's the part that makes your java code have inlets/outlets/etc. Also, it can talk to pattr, which could be a big plus.
      I greatly prefer Java over JS because of the generic question. I know there are ways of removing single values from the middle of arrays in JS, but it's probably something like slicing and then recombining. You could probably make a lambda function to add this, but it would be nice if JS had more built-in data types... I have a former student of mine who is a JS with Max ninja, and I can send you his info via PM if you like.
      You can always keep Max for the rendering. I just find that it becomes a bit cumbersome when you get to problems involving state. (e.g. where you have several functions that could be changing the value of 1 variable, and each needs to know the value of said variable; in a procedural language, there's more built-in support for this, so it's less ugly)
      Curious as to what the abstractions are doing in terms of calculation (orbits? Any comets?). What sort of classes are you looking at?
    • May 29 2012 | 11:19 pm
      Peter, again big thanks for your answer.
      Yes I made things with JAVA in Max. As you wrote, yes, MXJ is the interface between the code & Max runtime (as written in the doc WritingMaxExternalsInJava.pdf (available in Max distribution and there too: It also provides a way to send messages to any [receive] objects easily.
      In my case, my abstractions (made with Max & jitter) are created in the patcher dynamically. (using newdefault : file://localhost/Applications/Max6/java-doc/api/com/cycling74/max/MaxPatcher.html#newDefault(int, int, java.lang.String, com.cycling74.max.Atom[]) )
      This provide me a nice way to assemble my visuals bits on a side (a bunch of abstractions made with Max/Jitter) and the core would placed them as required by my (future) presets system (using probably XML)
      All my abstractions will be built on a basic standard (my standard I mean) I already tested one of my visual object (a kind of glowing sphere which isn't cute yet) Each abstraction being created by the code, I have a reference to them in the code. That way, the code (I mean the JAVA compiled stuff... of course), will be able to communicate with the abstraction to send, for instance, the master clock, to inactive/active them considering the position of the camera etc.
      I tested with some object, it works fine.
      I'll use data type providing more efficiency than Arrays in JAVA. (Map Object seemss nice) That way, I could have a more comfortable composition session, creating, removing objects in my 3D space without to have to take care to empty array's slot indeed..
      About the Max and cumbersome side of things. I guess I decided to make ALL operations using the JAVA core From the composition (a little GUI will be used to compose my objects in space) to the playing session where I'd have 2 physical controllers for instance, everything will be driven by that JAVA core (maybe splitted in 2 or 3 mxj, sharing data using namespace stuff shit :)
      About abstractions for visual purposes. I'll make some objects visually interesting. compound of gridshapes and lights and using shaders (when I'll be ok to program mine :p) I have a lot to learn too. And this is pretty exciting :) Robert (Ramirez) pointed me to the production of texture in screen space which frightens me (the terms I mean) but I'll study further that concept I didn't totally get yet..
      asap, I'll show results for sure
    • May 30 2012 | 8:10 pm
      I made huge progress and I wanted to thank all my precious helpers here :)
      Especially, I began to code the core and to design the messaging system. If anyone was interested in details, feel free to ask me that here or anywhere. Indeed, I wrote that : but it would be a bit light for you, gurus !
      Anyway. I'll use message busses. Some will be objects specific (in order to create a kind of point to point between the JAVA core & the objects Some others will be like broadcast. I'd like to make something more advanced like a subscription service to the message bus etc etc but not now. except if something like that would be very easy to set up. My system works fine with visually ugly objects at the moment. and I can communicate with all objects, propagate messages like master clock etc.
      Now my main question is: would I split that JAVA Core into pieces or not ?
      I'll create a special GUI Patcher plugged into the JAVA Core and, for instance, I could delegate all GUI stuff (basically only for composition my pieces into myUniverse) to another JAVA ... sharing the same memory space, variables etc..
      what do you think ?