Anthony Palomba

So I am trying to come up with an effect that will turn a video into geometry.

I basically want to be able to track an object and transform that object

into geometry and composite it back into the video.

So if it was Brad Pitt on screen, you would see a crude geometric Bizaro Pitt.

It would be nice to be able to control the vertex count so I could make it

more detailed. Vertex depth/height could be controlled by pixel color and luminosity.

Any suggestions on how I might go about implementing this?

So I am trying to come up with an effect that will turn a video into geometry.
I basically want to be able to track an object and transform that object
into geometry and composite it back into the video.

So if it was Brad Pitt on screen, you would see a crude geometric Bizaro Pitt.
It would be nice to be able to control the vertex count so I could make it
more detailed. Vertex depth/height could be controlled by pixel color and luminosity.

Any suggestions on how I might go about implementing this?


converting-video-into-geometric-abstractions

> Vertex depth/height could be controlled by pixel color and luminosity.

There's an example or jitter recipe - don't remember which - that does exactly that.

I looked through the jitter recipes and did not really see anything like what I described.

I looked through the jitter recipes and did not really see anything like what I described.
Can you be a little more specific?


https://cycling74.com/forums/sharing-is-fun-multiples-luminance-z-depth-and-jit-gen-oh-my

and there's this patch, which is from a relatively recent posting, but sorry can't figure which:

there's this thread:
https://cycling74.com/forums/sharing-is-fun-multiples-luminance-z-depth-and-jit-gen-oh-my

and there's this patch, which is from a relatively recent posting, but sorry can't figure which:


Sorry I can't find it. It's similar to jit.gl.nurbs-video-deform.maxpat (Jitter examples in Max program folder) but instead of nurbs it maps the video to vertices.

Sorry I can't find it. It's similar to jit.gl.nurbs-video-deform.maxpat (Jitter examples in Max program folder) but instead of nurbs it maps the video  to vertices.


- run your video matrix through a luminosity converter (don't remember the object name right now)

- run the lumo map into a jit.expr that takes the x and y coordinates of the input cells as output x and y, and takes the lumo (color) value for z

- run that into jit.gl.mesh position inlet and send the original video matrix into the texture inlet

you can also skip the first step and do the lumo calculation from the RGB values in jit.expr.

this reasoning might be a bit buggy but it's the general direction you should be heading for

Though the principal is easy:
- run your video matrix through a luminosity converter (don't remember the object name right now)
- run the lumo map into a jit.expr that takes the x and y coordinates of the input cells as output x and y, and takes the lumo (color) value for z
- run that into jit.gl.mesh position inlet and send the original video matrix into the texture inlet   

you can also skip the first step and do the lumo calculation from the RGB values in jit.expr. 

this reasoning might be a bit buggy but it's the general direction you should be heading for


here's a simple patch to get you started.

it uses jit.gen and jit.gl.pix to handle some of the minor things you need to do, like remap the xyz values, multiply the Z by luminosity, and rotate the texture properly.

here's a simple patch to get you started.
it uses jit.gen and jit.gl.pix to handle some of the minor things you need to do, like remap the xyz values, multiply the Z by luminosity, and rotate the texture properly.

Thanks guys, there are a lot of good ideas here. I will study them.

Converting video into geometric abstractions