Kinect to MAX/MSP for Windows

Aug 25, 2011 at 2:36pm

Kinect to MAX/MSP for Windows

I just thought id post about i program i created for the start of my final year research project as i imagine some people here may get some use out of it.

Using the offical SDK for windows the program takes data from the Xbox Kinect and sends it to Max/Msp via OSC. By using the Ventuz OSC C# wrapper it packs all the X, Y, Z data for all the skeletal points into an OSC bundle and sends that to Max where it was unpacked in the patch.

The data sent from the Kinect is the unchanged co-ordinate system of values in metres rangeing from -1 to 1 for all three axis for both players.

That being said the data sent from Kinect to Max can be used by any device that is capable of reading OSC packets .The format of the OSC message is

/joint/skeleton_[1/2]/[joint] x, y, z

This is a simple implementation allows quick and easy access to the full power of the kinects skeletal tracking in windows for users of the offical SDK over OSC. I hope it can be of some use.

Iv tested this on my Windows 7 x64 machine running Max 5, the download includes the program, source code and patches.

[attachment=169593,2568]

Attachments:
  1. screenshot.jpg
#58619
Aug 25, 2011 at 3:10pm

Wow
looks like a lot of time and effort went into this; I can imagine many applications, not just creative interaction but health services and industries, HCI etc. My default attitude to remote tracking (video/infrared/wiimote et al) is usually “meh”……but this looks very intriguing. Good luck with the continued research.

Brendan

#210711
Oct 23, 2011 at 2:10am

Hey Jordan,

This is awesome! I’m having trouble getting any OSC data to read in my patch though. I’ve linked the OSC-Route external, Opened the DepthView.exe, Opened Kinect for Max and opened port 7710. I’m not getting any OSC data in the skeleton patch.

I even turned my firewall off to see if that was the problem but no luck.

Thoughts?

#210712
Oct 26, 2011 at 9:18pm

Hey,

when you open the program do you see the depth image and get the messages of player 1/2 being tracked?

In maxmsp when you open the main patch and double click skeleton 1 do you see the data there?

#210713
Jan 4, 2012 at 9:36pm

Hey Jordan,
I’ve been trying to run the patch but I can’t get any data running through it. I did the same as sirjimbob and I’ve got no result, also tried with my firewall off but still nothing. I am running win 7 x64. One thing that I noticed is that when I run your patch or the “Kinect to Max.exe” the projector on the kinect doesn’t light up, but wen I run the “Sample Skeletal Viewer” from the official SDK it starts. Would you suggest what might be cousing the problem?
Thanks

#210714
Jan 9, 2012 at 1:30pm

I have just recompiled the program against the latest SDK. That should fix the problem.

Attachments:
  1. KinecttoMax.zip
#210715
Jan 9, 2012 at 8:05pm

Hello,
i’ve been trying to open Kincet to Max app, but it just stopped and doesn’t work.
Need I more software to run it? or its just necessary SDK drivers?
Is necessary make changes to “KinectCOMLib.dll”?

sorry for maybe noobie questions but i dont know what to do..

thx ;)

#210716
Jan 10, 2012 at 7:13pm

Hey Jordan. First off, great work! This is a serious project and you’ve done a great job. A couple things though.

First off, I’m on a Windows 7 x64 machine with Max 5 & 6.

Everything seems to load and run OK, but for the life of me I can’t get the skeleton to lock. Any depth, any length, any position, no luck. Any ideas? The MS Sample Skeletal Viewer locks on with pretty much no problem.

Also, it’d be nice if the viewer had the option to show as a mirror image. It just makes working on screen a bit easier.

Thanks so much man, and again, great work!

Jeff

#210717
Jan 11, 2012 at 3:39pm

I tested that version as well and still cant get any data, same as before.
I am runing win 7 x64 and max 6 like Jeff, could that be the problem?

#210718
Jan 11, 2012 at 8:48pm

Hey ElectronicElement. Are you able to get the Kinect to Max program to lock on to you?

#210719
Jan 12, 2012 at 9:05am

Jeff, I’ve been trying to, but unfortunately I cant get the patch to work, I cant get the depth viewer (Kinect To Max.exe) to work neither. Just doesnt show anything. I contacted Jordan by email, and he was kind enough to compile couple of different versions with different versions of the SDK drivers, but again with no success. Since yesterday I am trying with processing and then to send the data to Max, I still havent got any result but if I do I will post the code here. Otherwise the official SDK and its Simple Skeleton Viewer works fine…

#210720
Mar 27, 2012 at 8:22am

This is an incredibly late response but did you buy any chance sort your problem Jeff? Sorry i couldnt help you EE

#210721
Mar 28, 2012 at 9:35pm

Hey Jordan, I still havent, I left it aside for that time I was working on a sequencer controller and just finished it few days ago and again I am looking to get that kinect working so I could finish my project untill the end of June. I had to reinstall my computer several times since last month and I will install the kinect drivers right about now and see if there is any change since the last time.

#210722
Apr 3, 2012 at 12:09pm

Hi Jordan, have you compiled any version with the final kinect SDK? the once I have does not work again :/ also I wanted to ask you whether with your program I can have the raw kinect data as well?

#210723
Apr 5, 2012 at 2:41pm

As it stands its compiled against the beta SDK v2 which still supports the 360 Kinect as I dont own a Kinect for Windows, if i can get my hands on one ill recompile it but until then it wont work on the latest SDK sorry

#210724
Apr 5, 2012 at 8:05pm

The final Kinect SDK works perfectly with my xbox kinect. Even the speach recognition works fine and as far as I remember it didnt with the beta SDK. So you might want to try the final SDK as well. Also I found that external called synapse http://synapsekinect.tumblr.com/post/6307752257/maxmsp-jitter I tried to test it but because of lots of additional stuff which it requires I couldnt get it to run, but thats most likely because I am doing something wrong with the additional stuff. I will give it a try again tomorrow and will post if there is any result.

#210725
Apr 10, 2012 at 1:04pm

Awesome, did not know that, ill try recompiling against the latest SDK. As for synapse I did look at that but it doesnt have access to the audio API which i needed for my project.

#210726
May 19, 2012 at 11:39pm

Hi there, I’ve also been trying my hand at updating the source against the new SDK. These two articles have been indispensable:

http://robrelyea.wordpress.com/2012/02/01/k4w-code-migration-from-beta2-to-v1-0-managed/

http://imaginativeuniversal.com/blog/post/2012/03/16/Quick-Guide-to-moving-from-the-Kinect-SDK-beta-2-to-v1.aspx

As you can see there’s been a fair amount of updates. If you haven’t had the chance to get started on this yet, JordanRS, maybe we can collaborate: I’ll send you what I have and maybe you can help me figure out why I’m getting a black window frame instead of the camera data from the Beta version.

It seems to be a worthwhile project as the new SDK seems to be a lot more robust than the beta.

#210727
May 20, 2012 at 12:14am

Here’s what I have so far…

#210728
May 20, 2012 at 3:02pm

meanwhile you might want to check this jit.openni external http://hidale.com/category/software/
after i tried everything I could find on the net this was the only one which worked for me so far. I managed to get synapse runing as well, but then realized that the jitter external for synapse is for Mac only…

#210729
May 24, 2012 at 6:08am

Well it turns out the 1.5 SDK just came out on Monday so it would do well to compile against that one and forget about 1.0, I’ll have another look at the project.

I am likely going to also start working on a 64-bit Windows version of Synapse since it only supports 32-bit Windows at the moment (if anyone did get it to work let me know!), so if you have any requests for it let me know.

#210730
Jun 23, 2012 at 4:26am

Ok so I’ve mixed together a bit of code to get the functionality I was personally looking for (simply transmission of my two hand’s XYZ coords) implemented by mixing together the OSC portion of Jordan RS’s codebase in with some code from the Beginning Kinect Programming book, Chapter 4.

http://www.amazon.ca/Beginning-Kinect-Programming-Microsoft-SDK/dp/1430241047

The advantage here is that developments on this codebase will now be able to take advantage of new features of the 1.0/1.5 SDKs, and also already includes some extra code bits from the book’s base, including a method to detect which hand is closer on the depth plane to the camera than the other (which would be great for air drumming with the hands).

The GUI application is simply drawing the skeleton, scaled/truncated ints for the XYZ coords of the hands, and also showing which hand is closer to the camera. There’s no color camera layer as for my purposes it just takes away focus from how well the skeleton itself is being mapped.

I’m working with the Kinect for Windows hardware now but it’s been tested as working fine for XBox Kinect as well. I’m really interested in getting more into the seated mode and perhaps the face tracking as well (see my avatar!).

At this stage the codebase isn’t really good enough to share publicly here, but if anyone wants it I can send it to them privately.

I’ll also look into implementing any feature requests.I’m sending raw data to Max, as its better to scale the data only once within Max rather than scale once on the SDK side and then again within Max which would introduce some precision loss.

This is really all a side project to add Windows support for a Max application I’m going to release soon for Kinect, 2 WiiMotes, OpenGL/Jitter, and surround sound. If anyone has interest in that main project specifically, I’m starting a thread to try to gage interest so look for it.

#210731
Jun 23, 2012 at 6:22pm

hi Jordan.. firstly, thanks for your sharing on this… :)

im on max6 // win7 64bit sp1 with kinect sdk 1.5 and a xbox kinect model camera. drivers and camera working fine with other apps.

just downloaded your files, give a try to both .exe’s and absolutly nothing coming out or in…
even the leds of the camera dont change or the 3d sensor gets on… so it seems the software isnt initializing the kinect…

is your piece of soft only for new kinect for windows camera? or must work also with xbox model?

thanks…

#210732
Jun 26, 2012 at 6:21am

That code was designed for the beta 2 version of the SDK, as mentioned above. I’m working on a version for the 1.5 SDK that is now functional but I’d like to add a little more before releasing it. If there’s any particular functionality you’re hoping for now is a good time to mention it!

I should have it out by mid July.

Edit: Whoops almost forgot. So it turns out the Candescent NUI is in C# too so I’m going to see about merging some of that codebase to get some finger tracking information going into Max as well. With any luck it’ll all be working together within a few weeks, but this fork of the project will probably only be able to support the Kinect for Windows with its Near Mode (see Candescent FAQ)

#210733
Jul 9, 2012 at 2:54pm

Hey Kcoul,
I’m looking to integrate Kinect with my MaxMSP/Jitter project this year during my first Semester of my final year at University.
I’m thankful that you’ve been working hard on the project and I can’t wait to see how it looks.
The main thing I’m looking for is the ability to control various things with the sensor. I.e; Pitch, Volume, Speed etc.
An easy way of linking objects to the Kinect Sensor would also be a nice touch.

How is the ability to having multiple people? Say two people on screen at once, is it possible to simultaneous control four or more objects in unison? (depending of course that the people are in sync).

I can’t wait to see the project and give it a spin for my presentation.
Feel free to message me directly if you want to discuss anything, It’d be good to talk.

Thanks Again,

Drazzan

#210734
Sep 5, 2012 at 6:29pm

Hi Drazzan,

Sorry for a late reply as I was quite busy over the summer. Are you starting the semester this week, or did you start already?

The code I am working on can easily be modified to handle two people, I’ll take a look at doing that. I lost my code when an SSD I had everything on died so I will get it up and running again.

The way it works is all the joint data is being piped into Max/MSP and you can do whatever you want with it. I’m looking to add a few additional messages, such as a boolean for which hand is out further.

I’m not sure how much work I want to do on the Max/MSP side, aside from perhaps one patch as an example: the idea is to abstract it as much as possible so I don’t give the impression that the patch is only useful for a specific range of things. Hopefully I’ll get my code back to where it was before the SSD died by the end of this week or next, then if I can clean it up a bit I’ll be happy to post a first version, so check back soon.

Thanks!

#210735
Mar 19, 2013 at 3:38am

Sorry for the long wait, here is a fairly stable version that works with the latest 1.6 SDK. As you can see I am really only interested in upper torso data.

The wait is for a good reason though, I am now working on a centralized gestural dev environment called “GestureLab” that will map input from WiiMotes, Kinect, PS Move, and LeapMotion. It looks like the first version will probably be in C# for Windows. I’ll try and post an update, and check this thread to support this code if anyone has setup questions.

#210736
Jul 13, 2013 at 8:13am

hi Jordan today I get a kinect xbox 360 model 1473. I using sdk v 1.7 , I am really need advice what should I do for using kinect on max msp on pc window 7 with your patch.
thanks

#255894

You must be logged in to reply to this topic.