Crossole - Listening Machines 2011 GTCMT

    Crossword of Sound - Crossole


    Max receive OSC message from the c++ module(OpenFramework + Kinect + OpenNI + OSCeleton). And then based on received OSC message, MAX generates a sequence of MIDI notes of broken chord in specified tempo. Key point is that disassociating tempo and its sequence. It means you can change the a set of MIDI notes that will be played regardless of what is being played. MAX patch will either apply new sequence at the moment it received message or switch to the sequence from the point when next note will be played based on message type. Additionally, Max also helps to generate random MIDI notes with random interval. MIDI notes are sent to Ableton.
    The Max Patch is mostly done by Anosh Daruwalla.

    • Mar 17 2013 | 6:19 am
      Amazing project :) Well done! I love the variety of sounds that you are able to create with the different components of the installation.
      I was wondering though, how did you manage to get the swiping along the axis to work? I can see that when you swipe your hand up and down or left to right you are able to chose different parameters. Also, how did you create the graphics for the grid display?
      Thnaks in advance :)
      Great stuff!
      Marco
    • Mar 18 2013 | 7:51 am
      Thanks for generous comments. For a short answer, all gestures and graphics are implemented using Microsoft Kinect and OpenFrameworks. Swiping gestures is just based on the hand position cross the certain threshold into four directions. For a long answer plus motivations behind the project, there's a paper about this project you can read. http://scholar.google.com/scholar?hl=en&q=crossole+&btnG=&as_sdt=1%2C23&as_sdtp=
    • Mar 18 2013 | 9:20 am
      Thank you for your reply. And for the information. I will look at the paper on the project.
      Just out of interest, what operating system did you use to build, code and run Crossole on?
      Thanks again.
    • Mar 18 2013 | 11:27 am
      It was built and performed with 10.6.x OSX and used many api for kinect osc generation.
    • Mar 19 2013 | 3:52 am
      Thank you for the information :). Crossole is a really fantastic project.
      I am creating a project which also uses Kinect however i'm using Java and Processing instead of Openframeworks as i am not very confident with C++.
      Also, is OSCeleton a tracking programme?
      Thanks and sorry for all the questions.
    • Mar 19 2013 | 10:28 am
      Thanks!! I think Java/Processing will do without problem. You can switch later if you feel the program is slow. Yes, I used OSCeleton. If you have iOS device, you can download an app called "echobo" and see the block interface used in crossole implemented on a mobile phone for a completely different purpose.
      Good luck on your project and let me know if you post a video. :)
    • Mar 19 2013 | 1:51 pm
      Great :) thanks for all the help!
      With regards to the swipe gestures, would you mind sharing the code? No problems if you can't :)
      Thans again.
    • Mar 19 2013 | 5:30 pm
      not a problem. It will be extremely hard to read though. The code is at https://github.com/panavrin/crossole. I would start from GestureDetect() in Visual.cpp
    • Mar 20 2013 | 4:42 am
      Thanks for that :) You are right though, it is hard to read!
      One last question (I promise). How did you get the Kinect to understand where you are standing. I see on your video that if you stand in different parts of the stage, the screen displays different visuals. I have been looking for a way to do this for months but I can't figure anything out.
      Thanks again for your help :)
    • Mar 20 2013 | 11:08 am
      Well that's easy. I wonder if this is exactly what you need though. It's whether z coordinate(depth) of torso is within threshold or not. :)
      If you approach to camera the visual will change to grid sequencer view.
    • Mar 20 2013 | 3:00 pm
      How do you assign a z coordinate? I thought the kinect only could track X and Y.
    • Mar 20 2013 | 3:06 pm
      Kinect does measure depth (which is its core difference with other cameras.) OSCeleton will send you OSC message with xyz coordinate of your joints. See https://github.com/Sensebloom/OSCeleton#joint-message---message-with-the-coordinates-of-each-skeleton-joint
    • Mar 25 2013 | 3:08 am
      Thank you very much for all your help :)