LSka

3d footage captured with MS Kinect using the dp.kinect Max external by Dale Phurrough (hidale.com) elaborated in real-time with Jitter.

Music and video synched using LiveGrabber (http://showsync.info/index.php/tools/livegrabber/) and a custom Max4Live device.

Video recorded with jit.gl.syphonserver (https://github.com/Syphon/Jitter/releases/) and Syphon Recorder (http://syphon.v002.info/recorder/)

Music and video synched using LiveGrabber (http://showsync.info/index.php/tools/livegrabber/) and a custom Max4Live device.

take-care

@lska : wow, this is a really impressive work, well done.

I´m still working on a similar Project for University. Could you give me any advice how to setup my project. Which map is the correct one to get particles into silhouette? It would be great to point in a direction.

I use the @depthonlyplayers 1 attribute in the [dp.kinect] external, and record matrices with [jit.matrixset], then I sift out all "black" matrix cells using Wesley Smith's [xray.jit.sift] 

http://www.mat.ucsb.edu/~whsmith/xray/xray.jit.sift.html

 , so there are no spare points in my silouhette matrix on the Z axis.

To control the particle system's attraction and rotation, I adapted a gen patch I found in the forum 

https://cycling74.com/forums/sharing-is-hairy-brains-gen-particles/

(again, thanks to Wesley Smith! I just realized I have to edit the credits in the video... ;) ), using the recorded matrices to attract the particles.

Hi gantzgraf,
I use the @depthonlyplayers 1 attribute in the [dp.kinect] external, and record matrices with [jit.matrixset], then I sift out all "black" matrix cells using Wesley Smith's [xray.jit.sift] http://www.mat.ucsb.edu/~whsmith/xray/xray.jit.sift.html , so there are no spare points in my silouhette matrix on the Z axis.
Here's the "Zfilter" I used:

To control the particle system's attraction and rotation, I adapted a gen patch I found in the forum https://cycling74.com/forums/sharing-is-hairy-brains-gen-particles/
(again, thanks to Wesley Smith! I just realized I have to edit the credits in the video... ;) ), using the recorded matrices to attract the particles.


thank you for answering quickly and your help. I think the worst is yet to come but i see a light at the end of the tunnel .... ;)

I am new learner of MAX, and I need to do a similar project now. I don't understand the detail of this sentence, could you help me?

"use the @depthonlyplayers 1 attribute in the [dp.kinect] external, and record matrices with [jit.matrixset], then I sift out all "black" matrix cells using Wesley Smith’s [xray.jit.sift] 

http://www.mat.ucsb.edu/~whsmith/xray/xray.jit.sift.html"

Could you tell me the detail of how to do it? Thank you very much!

Hello LSKA, @lska
I am new learner of MAX, and I need to do a similar project now. I don't understand the detail of this sentence, could you help me?

"use the @depthonlyplayers 1 attribute in the [dp.kinect] external, and record matrices with [jit.matrixset], then I sift out all "black" matrix cells using Wesley Smith’s [xray.jit.sift] http://www.mat.ucsb.edu/~whsmith/xray/xray.jit.sift.html"

Could you tell me the detail of how to do it? Thank you very much!


Before answering your question, I want to warn you that this project could be quite complex for a new Max/Jitter user, as it involves a good knowledge of Jitter, Gen, OpenGl, and MSP, since audio analysis is also involved.

I don't want to share the complete patch at the moment, because it's really messy and crafted for this very project, so hardly useable for other projects. But sometime in the future I may come to a more "general purpose" patch to share with all the community.

dp.kinect is a Max external made by Dale Phurrough, which I use to capture and record Kinect data. 

What I use is the the so-called "depth map" which is a collection of 3D coordinates of the scene captured by the Kinect, output as a matrix.

The data that comes out is then recorded and reproduced as a stream of matrices using jit.matrixset.

I apply the "@depthonlyplayers 1" parameter to dp.kinect to isolate the bodies from the rest of the scene.

Doing so, I get a 320x240 matrix in which most of the cells are empty (i.e. their value is 0). This leads to a lot of artifacts when feeding this data to jit.gl.mesh, as the object will draw points whose coordinates are 0,0,0.

Thus, the xray.jit.sift object allows me to filter all cells with value 0, and send only the relevant data to jit.gl.mesh.

Hello @alicexin,
Before answering your question, I want to warn you that this project could be quite complex for a new Max/Jitter user, as it involves a good knowledge of Jitter, Gen, OpenGl, and MSP, since audio analysis is also involved.
I don't want to share the complete patch at the moment, because it's really messy and crafted for this very project, so hardly useable for other projects. But sometime in the future I may come to a more "general purpose" patch to share with all the community.

Answering your question:
dp.kinect is a Max external made by Dale Phurrough, which I use to capture and record Kinect data. https://hidale.com/shop/dp-kinect/
What I use is the the so-called "depth map" which is a collection of 3D coordinates of the scene captured by the Kinect, output as a matrix.
The data that comes out is then recorded and reproduced as a stream of matrices using jit.matrixset.
I apply the "@depthonlyplayers 1" parameter to dp.kinect to isolate the bodies from the rest of the scene.
Doing so, I get a 320x240 matrix in which most of the cells are empty (i.e. their value is 0). This leads to a lot of artifacts when feeding this data to jit.gl.mesh, as the object will draw points whose coordinates are 0,0,0.
Thus, the xray.jit.sift object allows me to filter all cells with value 0, and send only the relevant data to jit.gl.mesh.

This is an old thread but I am hoping to still get some help. I'm doing a similar effect but I'm struggling with getting the Kinect stuff hooked up to the particle bits..

Please, help me I'm trying to get this done asap for an assignment.

Attached is progress so far using the dp.kinect patch which I have registered and other particle examples.

I repeat LSKA's caution; a patch to achieve this effect is an advanced patch and to understand the working of such a patch you'll first need Max basics then learn the topic areas LSKA listed. Max provides tutorials which can help you build this foundation of knowledge.

I *do* want to do is provide you support for dp.kinect. I'm ethically not comfortable with providing a fully working patch for an assignment. So, I'll contribute mid->high level feedback on the patch while diving deep on any specific questions about dp.kinect's functionality.

Look at the demo patch that came with dp.kinect. Now look in the subpatch. There you can see how to draw a pointcloud with jit.gl.mesh. Remove all the stuff you don't need like the sound and speech parts.

You can then use one or both of the filtering methods LSKA provided above. Then, feed that new matrix into jit.gl.mesh.

Or, you could do it by distance; eliminating any points greater than x meters using jit.op. Then, feed that new matrix into jit.gl.mesh.

Or, use the playermap and jit.op to compare the playermap and pointcloud matrices and only allow "through" the pointcloud matrix cells that have a matching playermap cell other than a value of zero. Then, feed that new matrix into jit.gl.mesh.

Or, you could also multiply the pointcloud and playermap matrices if you can guarantee the playermap has only values of 0 or 1. Then, feed that new matrix into jit.gl.mesh.

Testing these five approaches would be needed to check which meets your specific needs and performs best. There are good tutorials and help patches that come with Max to give you examples of using jit.op from which you can learn.

Look at the demo patch that came with dp.kinect. Now look in the subpatch.  There you can see how to draw a pointcloud with jit.gl.mesh. Remove all the stuff you don't need like the sound and speech parts.

You can then use one or both of the filtering methods LSKA provided above. Then, feed that new matrix into jit.gl.mesh.
Or, you could do it by distance; eliminating any points greater than x meters using jit.op. Then, feed that new matrix into jit.gl.mesh.
Or, use the playermap and jit.op to compare the playermap and pointcloud matrices and only allow "through" the pointcloud matrix cells that have a matching playermap cell other than a value of zero. Then, feed that new matrix into jit.gl.mesh.
Or, you could also multiply the pointcloud and playermap matrices if you can guarantee the playermap has only values of 0 or 1. Then, feed that new matrix into jit.gl.mesh.
Testing these five approaches would be needed to check which meets your specific needs and performs best. There are good tutorials and help patches that come with Max to give you examples of using jit.op from which you can learn.


regarding your patch, I can tell you that the "trick" is in modifying the jit.gen patch that handles the particle attraction: in yours all particles are attracted to a center set by a param. You have to replace that param with an in which you'll feed with the point cloud coming from dp.kinect.

Thanks so much for your responses, I have developed my patch but I seem to receive a "jit.scanwrap-Missmatch plane" error, is it possible to use this filter for any jit.gl.mesh? For example, create a silhouette of the sound responsive triangles in my patch below? Or does this not work the same way?

I appreciate this is somewhat ahead of my knowledge of Max but I am really determined to have something of a silhouette effect and I have been able to figure most of Max out by fiddling except for this one bit!!

Have I connected something wrong? Am I missing an object or an argument somewhere?

Thanks again for the help I would buy you both a coffee if I could.

Have I connected something wrong? Am I missing an object or an argument somewhere? 

Take Care

Unknown

Unknown