frank desben

I would want to create an effect like this one:

https://media.giphy.com/media/d2ZbxnD6YpUsYheU/giphy.gif

Basically directed movement of subparts of an image/video.

It doesn't have to be masked as in this example, the masking I would know how to do.

Perhaps there's also a name for this sort of effect?

That would also help me as I don't really know what to look for :)

I would want to create an effect like this one:
https://media.giphy.com/media/d2ZbxnD6YpUsYheU/giphy.gif
(see attached file).
Basically directed movement of subparts of an image/video.
It doesn't have to be masked as in this example, the masking I would know how to do.

Perhaps there's also a name for this sort of effect?
That would also help me as I don't really know what to look for :)

video-effect-flowing-motion-2

jit.repos is one way to 'move' pixels, but nowadays I'd recommend doing it in jit.gen. One matrix being the source, the other being the movement vector. Then some nice crossfading between them. A step toward that is below, but it's not complete. What makes the gif subtle is that movement only happens in regions that have similar color/texture, which is to say, movement is zero where there are discernible edges. So you need a first analysis of the image to detect edges, and perhaps a distance transform on that to determine the amplitude of movements. Maybe something in cv.jit can determine which is the vector of greatest similarity between neighboring pixels (and then blur that a little). 

Graham's take seems pretty right on, but I'd also add in that this looks like a morph-like effect, which combines spatial distortion with crossfading of the color. The apparent continuity is partially due to fading out the distortion as it reaches the end of its travel distance. So to break this down, I think you'd need:

- distortion - could be done with jit.pix/gl.pix, or jit.repos, using some sort of generated motion map.

- ramp generator - to drive the intensity of the distortion map and also map to the xfade

- masking - to define which parts of the image get distorted ( could use jit.alphablend for this)

- xfade - between distorted and clean image

All of this could be accomplished within a single Gen object with multiple inlets, or otherwise using different Jitter Matrix operators (jit.repos, jit.alphablend, jit.op, jit.xfade, etc)

Graham's take seems pretty right on, but I'd also add in that this looks like a morph-like effect, which combines spatial distortion with crossfading of the color. The apparent continuity is partially due to fading out the distortion as it reaches the end of its travel distance.  So to break this down, I think you'd need:
- distortion - could be done with jit.pix/gl.pix, or jit.repos, using some sort of generated motion map.
- ramp generator - to drive the intensity of the distortion map and also map to the xfade
- masking - to define which parts of the image get distorted ( could use jit.alphablend for this)
- xfade - between distorted and clean image

All of this could be accomplished within a single Gen object with multiple inlets, or otherwise using different Jitter Matrix operators (jit.repos, jit.alphablend, jit.op, jit.xfade, etc)


It also looks like one or more masking layers were used to target the effect.

Basically the fading, masking and so on is not a problem for me,

it's rather the distortion / motion effect -

to apply this to subparts of the image shouldn't be a big problem.

As Andrew wrote: "– distortion – could be done with jit.pix/gl.pix, or jit.repos, using some sort of generated motion map."

Are there any good examples of this online already one could take as a starting point, aside from Graham's already pretty cool gen patch?

Dear all,
thanks for the advice -
Graham - that's a great starting point.
Basically the fading, masking and so on is not a problem for me,
it's rather the distortion / motion effect -
to apply this to subparts of the image shouldn't be a big problem.

As Andrew wrote: "– distortion – could be done with jit.pix/gl.pix, or jit.repos, using some sort of generated motion map."
Are there any good examples of this online already one could take as a starting point, aside from Graham's already pretty cool gen patch?

Are there any good examples of this online already one could take as a starting point, aside from Graham’s already pretty cool gen patch?

There are several WebGL sites that have examples of different effects. Shadertoy is probably the biggest. There are also numerous pdfs and other technical papers out there that have standard equations and algorithms. Not sure about your experience, but most of it can be ported to Gen with little trouble. Here are a few links and an example of how to get it working in [jit.gl.pix].

There are several WebGL sites that have examples of different effects.   Shadertoy is probably the biggest.  There are also numerous pdfs and other technical papers out there that have standard equations and algorithms.   Not sure about your experience, but most of it can be ported to Gen with little trouble.   Here are a few links and an example of how to get it working in [jit.gl.pix].

https://www.shadertoy.com/view/MtXSzs
https://www.shadertoy.com/view/Msl3WX
https://www.shadertoy.com/view/XlS3Wm
https://www.shadertoy.com/view/MdS3RV
https://www.shadertoy.com/view/ltf3W7
https://www.shadertoy.com/view/Xdc3z2

whoops... fixed a minor mistake in the above patch.

That's great Metamax. Thanks for the patch and links. I will have a try with those shaders.

Video Effect: Flowing, Motion