CarlR

I have made a crude patch where I have scissored up an image from a web cam into 12 sections 3 x 4. Then I have put a delay of different amounts on each of them and then glued them back together and sent it to a jit.window. The result is if you move you get cut up into bits then slowly come back together again.

Now in the pursuit of performance I am looking into jit.gl.videoplane ect

My question is, is it possible to split up my video, delay individual sections and re-composite all in GL? or should I just try and texturise all the outputs of the delays to a jit.gl.videoplane. This leaves the scissoring in software.

I have little GL experience as you can probably tell. I fear that the positioning of the sections could be a drag. I intend to make a bigger version of this with many more sections.

Your guidance much appreciated as ever
Carl


live-video-delay-in-gl

Doing video delays in GL is a bit of a pain in the ass, and may not save you as much effort as you'd think.

I made a patch that had a 5x5 grid of sections each of which could be individually delayed different amounts, or zoomed into particular sections of the overall image. 

One method would be to have # frames of delay, and to create # of textures, each associated with that frame. You'd then assign each section of your image a texture corresponding to the amount of delay it needs. You'd be creating as many textures as the maximum number of frames you have.

Another way to do the same thing is to create a single 3d matrix with # frames of delay that you then query for your frame that is going to go into the particular section of your grid.

If you added more frames of delay, you'd need to add more textures to the first model, whereas the second model always uses the same number of textures.

Both models are an improvement over sending individual frames to each section and trying to keep track of content at a particular position per node. It is better to create a single master matrix that can be pulled from, or a single GL set of textures that can be similarly referenced.

I made a patch that had a 5x5 grid of sections each of which could be individually delayed different amounts, or zoomed into particular sections of the overall image. 
   One method would be to have # frames of delay, and to create # of textures, each associated with that frame.  You'd then assign each section of your image a texture corresponding to the amount of delay it needs.   You'd be creating as many textures as the maximum number of frames you have.
25 frames of delay = 25 textures.

Another way to do the same thing is to create a single 3d matrix with # frames of delay that you then query for your frame that is going to go into the particular section of your grid.
5x5 grid = 25 textures.

If you added more frames of delay, you'd need to add more textures to the first model, whereas the second model always uses the same number of textures.  

Both models are an improvement over sending individual frames to each section and trying to keep track of content at a particular position per node.   It is better to create a single master matrix that can be pulled from, or a single GL set of textures that can be similarly referenced.


Probably a bit advanced for me at the moment! I am still trying to work out the relationship of my video input (640 x 480) and the normalised units that are used with the position and scaling messages in gl.I have twelve 160 x 160 sections that come out of a jit.scissors 3 x 4 rows columns but can't figure out what values I need to put them back in the same positions.

I did this fine with jit.glue but performance was not good and had a massive delay even on the video that had no delay

If anybody can be botherd to have a look at my embryonic attempt to stitch my video back together here is an basic example with obviously wrong numbers in the texture messages

This uses a 640 x 480 web cam as the input with jit.qt.grab (Mac)

Thanks Frenzy,
Probably a bit advanced for me at the moment! I am still trying to work out the relationship of my video input (640 x 480) and the normalised units that are used with the position and scaling messages in gl.I have twelve 160 x 160 sections that come out of a jit.scissors 3 x 4 rows columns but can't figure out what values I need to put them back in the same positions. 

I did this fine with jit.glue but performance was not good and had a massive delay even on the video that had no delay 

If anybody can be botherd to have a look at my embryonic attempt to stitch my video back together here is an basic example with obviously wrong numbers in the texture messages 

Try using jit.gl.videoplane @transform_reset 2. This will normalize

On Mon, Dec 8, 2008 at 12:07 PM, Carl Rohumaa wrote:

> Probably a bit advanced for me at the moment! I am still trying to work out the relationship of my video input (640 x 480) and the normalised units that are used with the position and scaling messages in gl.I have twelve 160 x 160 sections that come out of a jit.scissors 3 x 4 rows columns but can't figure out what values I need to put them back in the same positions.

> I did this fine with jit.glue but performance was not good and had a massive delay even on the video that had no delay

> If anybody can be botherd to have a look at my embryonic attempt to stitch my video back together here is an basic example with obviously wrong numbers in the texture messages

> This uses a 640 x 480 web cam as the input with jit.qt.grab (Mac)

> #P message 968 489 205 196617 texture movie12 , position 0.5 -0.5 0 , bang;

> #P message 753 489 210 196617 texture movie11 , position -0.5 -0.5 0 , bang;

> #P message 542 489 205 196617 texture movie10 , position -0.5 0.5 0 , bang;

> #P message 342 489 194 196617 texture movie9 , position 0.5 0.5 0 , bang;

> #P message 937 440 199 196617 texture movie8 , position 0.5 -0.5 0 , bang;

> #P message 728 440 204 196617 texture movie7 , position -0.5 -0.5 0 , bang;

> #P message 524 440 199 196617 texture movie6 , position -0.5 0.5 0 , bang;

> #P message 323 440 194 196617 texture movie5 , position 0.5 0.5 0 , bang;

> #P comment 184 267 124 196617 RE-COMPOSITE SECTIONS to a jit.gl.videoplane;

> #P comment 231 115 100 196617 SPLIT INTO 12 SECTIONS;

> #P newex 342 120 165 196617 jit.scissors @rows 3 @columns 4;

> #P newex 342 86 142 196617 jit.qt.grab 640 480;

> #P comment 228 89 100 196617 GRAB THE VIDEO;

> #P newex 883 317 177 196617 jit.gl.texture comp1 @name movie12;

> #P newex 699 318 177 196617 jit.gl.texture comp1 @name movie11;

> #P newex 342 318 171 196617 jit.gl.texture comp1 @name movie9;

> #P newex 517 318 177 196617 jit.gl.texture comp1 @name movie10;

> #P newex 883 283 171 196617 jit.gl.texture comp1 @name movie8;

> #P newex 699 284 171 196617 jit.gl.texture comp1 @name movie7;

> #P newex 342 284 171 196617 jit.gl.texture comp1 @name movie5;

> #P newex 517 284 171 196617 jit.gl.texture comp1 @name movie6;

> #P newex 106 459 144 196617 jit.window comp1 1680 1050;

> #P bpatcher 106 431 14 11 -27 -24 "fullscreen and hide menu2" 3;

Try using jit.gl.videoplane @transform_reset 2.  This will normalize
coordinates across [-1, 1] x [-1, 1]
wes

On Mon, Dec 8, 2008 at 12:07 PM, Carl Rohumaa  wrote:
>
> Thanks Frenzy,
> Probably a bit advanced for me at the moment! I am still trying to work out the relationship of my video input (640 x 480) and the normalised units that are used with the position and scaling messages in gl.I have twelve 160 x 160 sections that come out of a jit.scissors 3 x 4 rows columns but can't figure out what values I need to put them back in the same positions.
>
> I did this fine with jit.glue but performance was not good and had a massive delay even on the video that had no delay
>
> If anybody can be botherd to have a look at my embryonic attempt to stitch my video back together here is an basic example with obviously wrong numbers in the texture messages
>
> This uses a 640 x 480 web cam as the input with jit.qt.grab (Mac)
>
> Cheers
> Carl
>
> max v2;
> #N vpatcher 311 70 1539 1023;
> #P origin 112 533;
> #P window setfont "Sans Serif" 9.;
> #P window linecount 1;
> #P message 968 489 205 196617 texture movie12 , position 0.5 -0.5 0 , bang;
> #P message 753 489 210 196617 texture movie11 , position -0.5 -0.5 0 , bang;
> #P message 542 489 205 196617 texture movie10 , position -0.5 0.5 0 , bang;
> #P message 342 489 194 196617 texture movie9 , position 0.5 0.5 0 , bang;
> #P message 937 440 199 196617 texture movie8 , position 0.5 -0.5 0 , bang;
> #P message 728 440 204 196617 texture movie7 , position -0.5 -0.5 0 , bang;
> #P message 524 440 199 196617 texture movie6 , position -0.5 0.5 0 , bang;
> #P message 323 440 194 196617 texture movie5 , position 0.5 0.5 0 , bang;
> #P window linecount 2;
> #P comment 184 267 124 196617 RE-COMPOSITE SECTIONS to a jit.gl.videoplane;
> #P comment 231 115 100 196617 SPLIT INTO 12 SECTIONS;
> #P window linecount 1;
> #P newex 342 120 165 196617 jit.scissors @rows 3 @columns 4;
> #P message 358 32 47 196617 close;
> #P toggle 417 31 20 0;
> #P message 304 32 45 196617 open;
> #P newex 342 86 142 196617 jit.qt.grab 640 480;
> #P newex 417 57 65 196617 metro 2;
> #P comment 228 89 100 196617 GRAB THE VIDEO;
> #P newex 883 317 177 196617 jit.gl.texture comp1 @name movie12;
> #P newex 699 318 177 196617 jit.gl.texture comp1 @name movie11;
> #P newex 342 318 171 196617 jit.gl.texture comp1 @name movie9;
> #P newex 517 318 177 196617 jit.gl.texture comp1 @name movie10;
> #P newex 883 283 171 196617 jit.gl.texture comp1 @name movie8;
> #P newex 699 284 171 196617 jit.gl.texture comp1 @name movie7;
> #P newex 342 284 171 196617 jit.gl.texture comp1 @name movie5;
> #P newex 517 284 171 196617 jit.gl.texture comp1 @name movie6;
> #P toggle 89 208 15 0;
> #P newex 106 459 144 196617 jit.window comp1 1680 1050;
> #P user jit.fpsgui 21 437 60 196617 0;
> #P bpatcher 106 431 14 11 -27 -24 "fullscreen and hide menu2" 3;
> #N vpatcher 334 50 1125 572;
> #N comlet fullscreen message;
> #P outlet 51 227 15 0;
> #P window setfont "Sans Serif" 9.;
> #P comment 214 169 134 196617 H to hide menu bar;
> #P toggle 40 96 15 0;
> #P window setfont "Fixedwidth Serif" 10.;
> #P newex 40 70 64 1441802 select 27;
> #P newex 40 45 40 1441802 key;
> #P toggle 235 66 15 0;
> #P newex 235 44 70 1441802 select 104;
> #P newex 235 19 40 1441802 key;
> #P newex 194 95 212 1441802 sel 0 1;
> #P window linecount 2;
> #P message 194 116 98 1441802 ; max showmenubar;
> #P message 425 190 98 1441802 ; max hidemenubar;
> #P window linecount 1;
> #P message 40 115 83 1441802 fullscreen $1;
> #P window setfont "Sans Serif" 9.;
> #P comment 16 169 134 196617 Spacebar for fullscreen;
> #P connect 8 0 9 0;
> #P connect 9 0 10 0;
> #P connect 10 0 1 0;
> #P connect 1 0 12 0;
> #P fasten 7 0 4 0 240 88 199 88;
> #P connect 4 0 3 0;
> #P connect 5 0 6 0;
> #P connect 6 0 7 0;
> #P connect 10 0 7 0;
> #P connect 4 1 2 0;
> #P pop;
> #BP pop 0;
> #P comment 587 671 134 196617 ESC FOR FULLSCREEN;
> #P newex 21 402 210 196617 jit.gl.render comp1 @erase_color 0. 0. 0. 1.;
> #P newex 89 230 51 196617 qmetro 2;
> #P newex 89 264 58 196617 t b b erase;
> #P message 922 403 196 196617 texture movie4 , position 0.5 -0.5 0 , bang;
> #P message 717 403 201 196617 texture movie3 , position -0.5 -0.5 0 , bang;
> #P message 511 403 199 196617 texture movie2 , position -0.3 0.5 0 , bang;
> #P message 299 402 199 196617 texture movie1 , position -0.3 0.5 0 , bang;
> #P newex 156 319 157 196617 t b b b b b b b b b b b b;
> #P window linecount 2;
> #P newex 299 577 330 196617 jit.gl.videoplane comp1 @scale 2. 2. 2. @depth_enable 1 @blend_enable 1 @automatic 1 @dim 2 2 @scale 0.2 0.2 0. @depth_enable 1;
> #P window linecount 1;
> #P newex 883 250 171 196617 jit.gl.texture comp1 @name movie4;
> #P newex 699 251 171 196617 jit.gl.texture comp1 @name movie3;
> #P newex 342 250 171 196617 jit.gl.texture comp1 @name movie1;
> #P newex 517 250 171 196617 jit.gl.texture comp1 @name movie2;
> #P comment 519 123 130 196617 I hope to tap these outputs;
> #P fasten 11 0 13 0 94 380 26 380;
> #P fasten 11 2 13 0 142 358 26 358;
> #P connect 13 0 16 0;
> #P connect 18 0 12 0;
> #P fasten 12 0 11 0 94 246 94 246;
> #P connect 15 0 17 0;
> #P fasten 11 1 6 0 118 295 161 295;
> #P connect 6 11 7 0;
> #P fasten 41 0 5 0 547 512 304 512;
> #P fasten 40 0 5 0 347 512 304 512;
> #P fasten 39 0 5 0 942 483 304 483;
> #P fasten 38 0 5 0 733 483 304 483;
> #P fasten 37 0 5 0 529 483 304 483;
> #P fasten 36 0 5 0 328 483 304 483;
> #P fasten 10 0 5 0 927 465 304 465;
> #P fasten 9 0 5 0 722 465 304 465;
> #P fasten 8 0 5 0 516 465 304 465;
> #P connect 7 0 5 0;
> #P fasten 42 0 5 0 758 540 304 540;
> #P fasten 43 0 5 0 973 540 304 540;
> #P fasten 6 7 36 0 252 370 328 370;
> #P fasten 30 0 29 0 309 68 347 68;
> #P fasten 32 0 29 0 363 60 347 60;
> #P fasten 28 0 29 0 422 78 347 78;
> #P connect 29 0 33 0;
> #P fasten 33 0 2 0 347 195 347 195;
> #P fasten 33 4 20 0 395 211 347 211;
> #P fasten 33 8 24 0 443 224 347 224;
> #P fasten 6 3 40 0 200 390 347 390;
> #P fasten 31 0 28 0 422 50 422 50;
> #P fasten 6 10 8 0 291 351 516 351;
> #P fasten 33 1 1 0 359 183 522 183;
> #P fasten 33 5 19 0 407 194 522 194;
> #P fasten 33 9 23 0 455 161 522 161;
> #P fasten 6 6 37 0 239 370 529 370;
> #P fasten 6 2 41 0 187 390 547 390;
> #P fasten 33 2 3 0 371 172 704 172;
> #P fasten 33 6 21 0 419 202 704 202;
> #P fasten 33 10 25 0 467 152 704 152;
> #P fasten 6 9 9 0 278 351 722 351;
> #P fasten 6 5 38 0 226 370 733 370;
> #P fasten 6 1 42 0 174 390 758 390;
> #P fasten 33 3 4 0 383 234 888 234;
> #P fasten 33 7 22 0 431 212 888 212;
> #P fasten 33 11 26 0 479 144 888 144;
> #P fasten 6 8 10 0 265 351 927 351;
> #P fasten 6 4 39 0 213 370 942 370;
> #P fasten 6 0 43 0 161 390 973 390;
> #P pop;
>


Thanks Wes that really simplified things for me. 

I have also learnt today that 4 divided by 1 is .25! etc

So for others who are as hair-brained as myself if you use @transform_reset 2 then it dose not really matter what the input is as it will be normalised to [-1, 1] x [-1, 1]

So in my patch the twelve chunks of video come in 3 rows and 4 columns, so the scaling on the videoplane needs to be .25 = (4 div 1) wide and .33 = (3 div 1) across

The positions work like this too. -0.75 is the far left 0.75 is far right 0.66 is top 0.0 is middle and -0.66 is bottom taking into account the size of the object

Thanks Wes that really simplified things for me. 
I have also learnt today that 4 divided by 1 is .25! etc

So for others who are as hair-brained as myself if you use @transform_reset 2 then it dose not really matter what the input is as it will be normalised to  [-1, 1] x [-1, 1] 

Live Video delay in GL?