Pheba

I'm starting from dp.kinect2 point cloud example to build a live chronophotography only applied to the visible user.

Since the picture of the user is taken from a jit.gl.mesh object I guess I have to turn it into a jit.gl.texture one to work with it but I have no idea of how to do this.

Hi there,
I'm starting from dp.kinect2 point cloud example to build a live chronophotography only applied to the visible user.
Since the picture of the user is taken from a jit.gl.mesh object I guess I have to turn it into a jit.gl.texture one to work with it but I have no idea of how to do this.
Could someone please help?

dp-kinect2-pointcloud

I googled chronophotography and from some pictures, I'm writing the below. If this isn't what you mean, please clarify more your inquiry.

Both 2D color images and 3D pointclouds are both matrices. They come out of the colormap outlet -or- the pointcloud outlet.

Both of these matrices can be manipulated to only have pixels/voxels for recognized humans.

- You can can write custom image filtering algorithms.

- You can use the depthmap and @align it to the color matrix. Then do some filtering based on depth to remove any background.

- You can use the playermap matrix coming from the 4th outlet to use for filtering with jit.op or your own code.

- You can use the @depthonlyplayers feature of dp.kinect2 to filter your color or pointcloud matrix

Now with your filtered matrix (pointcloud or flat 2D color image), you want to store it as a snapshot in time to use later.

You could use something like jit.matrixset to store/access a series of them. Maybe one every 1/5 second.

Then you have the artistry of how you merge together several of the point-in-time matrices into a single image matrix.

You could probably also do something similar by rendering everything to OpenGL, but not updating your framebuffer every 1/30sec.

But to update it only every 1/5 second. And then use transparency and jit.gl.render @erase_color to get the studder/tracer effect.

Both 2D color images and 3D pointclouds are both matrices. They come out of the colormap outlet -or- the pointcloud outlet.
Both of these matrices can be manipulated to only have pixels/voxels for recognized humans.
- You can can write custom image filtering algorithms.
- You can find 3rd party algorithms
- You can use the depthmap and @align it to the color matrix. Then do some filtering based on depth to remove any background.
- You can use the playermap matrix coming from the 4th outlet to use for filtering with jit.op or your own code.
- You can use the @depthonlyplayers feature of dp.kinect2 to filter your color or pointcloud matrix
- Combine several above...
- etc.

Now with your filtered matrix (pointcloud or flat 2D color image), you want to store it as a snapshot in time to use later.
You could use something like jit.matrixset to store/access a series of them. Maybe one every 1/5 second.
Then you have the artistry of how you merge together several of the point-in-time matrices into a single image matrix.

You could probably also do something similar by rendering everything to OpenGL, but not updating your framebuffer every 1/30sec.
But to update it only every 1/5 second. And then use transparency and jit.gl.render @erase_color to get the studder/tracer effect.


Hi Dale and, first of all, thank you so much for your dpkinect2 plugin!

Then thank you for your help, I really appreciate it as a beginner with Max.

For the storing part, I decided to work with the matrix thru attribute, set at value 0. I tried both with the matrixset and the buffer/erase combination in OpenGl but they weren't that satisfying for the effect I'm looking for.

So I'm storing several snapshots in time to then combine them to the video stream through jit operations.

The problem now is with the image I get from the kinect. I'm now taking the pointcloud outlet and aligning it to the rgb color image, filtering everything with the depthonlyplayers. I render the pointclouds in OpenGl, just as shown in the dpkinect2 help pointcloud patch. The problem is with the quality of the picture. If I draw it with the trid_grid I get a kind of ruined picture, quite pixelated. I'd better keep the points draw mode but then I have no idea of how to remove a sort of grid that appears when using that mode.

I'll attach two pictures to show the effect.

Hi Dale and, first of all, thank you so much for your dpkinect2 plugin!
Then thank you for your help, I really appreciate it as a beginner with Max.

For the storing part, I decided to work with the matrix thru attribute, set at value 0. I tried both with the matrixset and the buffer/erase combination in OpenGl but they weren't that satisfying for the effect I'm looking for.
So I'm storing several snapshots in time to then combine them to the video stream through jit operations.

Looking at your top picture, I have also seen a grid-like pattern in the data. This is inherit in the data coming from the Kinect. I believe it is an artifact that results from the pixel sensors on the camera and Microsoft's math algorithms. The sensor itself is likely laid out in a grid or hex pattern and therefore their data will be also. JPGs, PNG, TIFF...all these data formats are laid out in grids. If you zoom into a picture, you can see the grid pattern. I have also seen a Moiré pattern when I do some processing on the Kinect data. I believe these are all normal things.

Often in the digital world of 0s and 1s we want a continuous analog results. But we will never get that because with computers there are the 0=off and 1=on; a single value stored in a single linear digital memory location. Have you heard the arguments for/against digital music and analog records? This is all the same topic...How do we get an analog "feel/look" when the data is stored in an absolute 0/1 pattern? There are no perfect solutions. However, some might include: dithering, randomness, gradients, aliasing, etc. You can wiki and google all of these and find almost limitless writing on them. Jitter Tutorial 37 has a patch which demonstrates how to add some randomness to OpenGL data.

Looking at your bottom picture, I don't see anything bad myself. However, you have your eyes on the real-world so you can make the comparison. Please remember that when you align, the underlying data does not change.

* Aligning color->depth will take the 1920x1080 color image and align (downscale) it to the 512x424 depth/pointcloud image. Therefore, the result will be a 512x424 image. Some of the color data will be discarded because there are only 1/4th of the pixels in the smaller image.

* Aligning depth->color will take the 512x424 depth/pointcloud image and align (upscale) it to the 1920x1080 color image. This resulting image is 1920x1080 pixels. Some of the color data will be repeated in the result because the underlying 3D mapping data is still 512x424. When the aligning process is done between colordepth, it is affected by that 512x424 resolution because there has to be a "mapping" of colordepth and that depth is still 512x424.

FYI: Both jit.gl.mesh and jit.gl.sketch will usually allow you to control the point size when your drawing method=point. On jit.gl.mesh it is @point_size and it is a floating point number. I have used this before to hide the grid. The grid is still there if you zoom in...but just like JPGs...if you zoom out and look at it...it can create a more blended image.

Looking at your bottom picture, I don't see anything bad myself. However, you have your eyes on the real-world so you can make the comparison. Please remember that when you align, the underlying data does not change.
* Aligning color->depth will take the 1920x1080 color image and align (downscale) it to the 512x424 depth/pointcloud image. Therefore, the result will be a 512x424 image. Some of the color data will be discarded because there are only 1/4th of the pixels in the smaller image.
* Aligning depth->color will take the 512x424 depth/pointcloud image and align (upscale) it to the 1920x1080 color image. This resulting image is 1920x1080 pixels. Some of the color data will be repeated in the result because the underlying 3D mapping data is still 512x424. When the aligning process is done between colordepth, it is affected by that 512x424 resolution because there has to be a "mapping" of colordepth and that depth is still 512x424.

FYI: Both jit.gl.mesh and jit.gl.sketch will usually allow you to control the point size when your drawing method=point. On jit.gl.mesh it is @point_size and it is a floating point number. I have used this before to hide the grid. The grid is still there if you zoom in...but just like JPGs...if you zoom out and look at it...it can create a more blended image.


dp.kinect2 pointcloud