convert matrix - draw pixels on y-axis using brightness-values as new y-position
hello,
i want to convert a matrix in the folliowing way:
i have a 1 plane matrix (float32).
now i want to essentially replace the y-values of the matrix by the birghtness values of the matrix, while keeping the value of the x-position.
the original y-values can be deleted, and each drawn point on y-axis can be just binary white.
please help!
some background on why i want to do this:
i have a kinect depth image that i want to represent in 2d for further analysis (videotracking).
Correct me if I'm wrong but... if you have a 1-dimensional matrix, you have only 1 dimension (either x or y).
In order to modify the "y" you should have a 2 plane matrix (plane 0 for x values, plane 1 vor y values).
Also to fill values onto a matrix, have a look at jit.fill
hi beatriz,
sorry, i was being very unprecise and made errors:
i do have a 320x240 matrix with 1 plane (not dimension, sorry).
it's actually a depth map from a microsoft kinect, so it's in float32, and the values represent the distance of objects.
what i want to do is generate a top-down view area map with this data.
i did achieve this already in 2 ways, but both not satisfactory:
way 1:
do a jit.iter to extract the values, convert to int, and do a setcell on a matrix with x=x and y=value (and value = 255).
works, but is way too slow for 320*240
way 2:
project the matrix onto a mesh, and deform the mesh according to the values.
in an opengl window, make an orthogonal top-down view, and render this into a matrix.
problem with way 2 is, that i loose all the precise depth information.
i'd very much prefer a way where i could semi-automatically generate a new matrix
dim x= x, dim y= value, value = 255
from the initial
dim x= x, dim y= y, value = value
did i express myself better?
maybe jit.expr is the way to go, but i can't seem to really wrap my head around that one...
Hi vogt,
As far as I know (please somebody correct me if I'm wrong), 1-plane matrices are one-dimensional matrices: each cell can store only 1 value - 1 coordinate- per cell.
The fact that your matrix dimensions are 320x240 does not mean that there are 2 dimensions. You could have also a matrix of dimensions 1 76800, and it would be the same: 1 dim only, 1 value per cell.
I don't know if you are looking for something like in this example: x= values from -1. to 1., y = Kinect values, z = values from -1 to 1.
hi beatriz,
thank you very much - that's exactly what i want to do, but i already got that far by the help of dtr in this thread:
https://cycling74.com/forums/open-kinect&page=2
what i want to do is more or less this:
it works, but this is horribly slow!
there must be some more elegant solution than to set each pixel manually. do you have any advice for me?
btw. i realized i can use the 3 planes of dtr's calculation results, as values "r g b" represent the real x y z of points measured by kinect in space (much better than using the x coordinate of the actual image).
still, that doesn't solve my issue:
i need to generate a new matrix from the values of another one.
in the 3-plane case this would be
transform
matrix1 x1 y1 r1 g1 b1
into
matrix2 r1 b1 255 255 255
(i'm using white as i just need to see the "on" pixels -- i want to process this image further with cv.jit)
if you have any advice for me i'd be very very happy!!
thanks,
clemens
ps: attached a screenshot of my status as is - i use clicking on the depth image to locate a point and draw the position of that in the lcd on the right. the blue trapezoid represents the camera angle. i will of course dump this lcd in the end, as i need a 2d matrix with all movement of people visible in it.
(also i'm running in 640*480 in this screenshot)
ps: the problem with the opengl variant is, that the image is correct, but all the precise distance values get lost, and i have to somehow approximate them..
precise matrix -> opengl rendering -> matrix
i can't really adjust the camera view and resolution good enough to use the rendered opengl image as real distance map.. :/
with calculations directly on the matrix i can keep control of the actual position values!
background info: i want to track people's position in a room, and feed them to a real time multi channel ambisoncs audio system.
the 3-plane variant for the supersimple iter thing is
but one frame takes 30 seconds to render that way.
i could of course cut out only a few lines in the middle of the image, this would speed things up.
i guess if there is no other option i have to go with the opengl render -> matrix way.
Sorry vogt, I kind of understand what you want to do, but I cannot figure out any answer ... maybe the more experienced people of the forum can give you a clue.
hi beatriz,
thanks anyway for "listening".
maybe this is just way too complicated to do in realtime anyway.
:(
ps: i wish i could change the topic of the thread; it's probably misleading, and doesn't describe my question very well.
Sounds like you want a Rutt Etra look. Search the forum for some different approaches to making a Rutt Etra like effect.
maybe jit.repos will work for you... have you had a look at that?
@Terry McDermott:
thank you! jit.repos does almost what i was looking for!
but only almost -- i want to draw new pixels in white on a black matrix, not reposition another matrix.
if i eg. feed a totally white matrix into jit.repos, and my "map" into the right inlet, the resulting matrix will still be totally white. i need those areas for which i don't get a value stay black.
any ideas how to achieve that?
@vade: great find, i didn't know about the rutt etra -- but it's not what i'm searchng for. this kind of effect works already. i need a simple 2d top down view of my 3d scene, in matrix form!
i will try to describe in words what i want to do:
an example
let's say i have a matrix with dim 2x2 char and 2 planes
the values are
cell 0/0 - 4; 7
cell 0/1 - 3; 8
cell 1/0 - 1; 2
cell 1/1 - 5; 0
(accoding to cell x/y - plane1 value; plane2 value)
now with the values of the planes i want to fill a new matrix, let's say it's dim 8x8 char with 1 plane.
i want to draw a point at every location specified by the contents of the first matrix.
so i draw white points:
cell 4 / 7 - 255
cell 3 / 8 - 255
cell 1 / 2 - 255
cell 5 /0 - 255
the rest of the pixels stay black.
did i explain my problem in a clearer way?
there has to be a simple solution to this without painting manually with setcell.
please help!
don't know how to this other than with jit.iter and setcell. in my tracking patch i'm also iterating through the perspective corrected vertex matrix to deduct position, volume, etc. i downsample it first for keeping the framerate up.
btw, don't put num boxes in the iteration chain. updating these all the time slows down your system even more.
You'd probably be able to write a shader to do this fairly easily, but you'd have to then read back the resulting texture if you want to analyze the binary image you're outputting. Not sure it would be worth the learning curve for you if you don't already write shaders, as there's no guarantee this would lead to better performance than the technique you're already using. Texture readback is a complex topic...
well writing a shader is beyond my current abilities; and unfortunately i don't have the time for studying that now.
and the setcell method is impossibly slow.
ok so i guess i have to stick to the mesh variant.
i adapted dtr's patch further; by cutting out only a slice of 100 px in the middle i can get rid of floor & ceiling, and rendering that back into a matrix in ortho mode gives me a x-y map of the area in front of the camera.
only trouble is now that i have to use a measuring tape to map the matrix xy values to real distances again. i'm sure i'm loosing a lot of information that way, but whatever, at least it works somehow.
thanks everybody for advice!
That formula above sounds like a job for jit.expr, no?
do you need actual cm distances? couldn't it be enough to have distances relative to the virtual world?
i clip my performance space in a similar way but with 6 instead of 2 clipping planes (1 for each side of the rectangle). then i normalize the position coordinates of the tracked points inside the volume to the xyz dimensions of it. and i use those normalized coordinates as inputs to generative sound (and later visuals too). so i replace the real world distances for virtual world ones, which in my case works fine.
@vade:
well, i was hoping so, but i learned about jit.expr only a week or two ago, and i didn't really manage to achieve what i wanted with it.
do you know how to do that with jit.expr?
now that would be really cool!
@dtr
well i am using a multichannel ambisonics patch, and would have to align the 3d space with my speaker setup anyway; it would make the job way easier and more precise to have the direct coordinates.
right now i'm thinking if i could use the direct output of the perspective correction for tracking, and skipping the mesh projection part, as the image output of your shader already has all the values i need for tracking.
i have to investigate - i will post reults, if i find a good way to do that without using a top-down projection.