Forums > Jitter

Realtime video buffer with variable speed access

December 17, 2007 | 7:14 pm

Hello,

A question about time. Could it be possible to record live video with
a jit.qt.grab, and store it in real time in a variable access buffer,
which would be read at variable speed rates ?

For example we would read this buffer at the same speed than the
incoming live video (= no delay), then we would read it slower, at
different speeds (=> creating an increasing delay compared with the
realtime), and then for example accelerate again the lecture speed of
the buffer, in order to re-synchronise it with the incoming realtime
video.

I tried to make it with the "video delay" example published by
Jonathan Lee Marcus on the list (thanks again), using matrix
dimensions to store frames read with the srcdimstart and srcdimend
cmds, but I didn’t get it : I can’t read these stored matrix at
different speed rates. Sorry if it’s obvious, I’m not very
comfortable with this method. I post the patch again for reference.

Thanks & best regards,

Thierry Fournier

max v2;
#N vpatcher 31 63 1038 714;
#P origin 66 321;
#P user jit.pwindow 362 345 322 242 0 1 0 0 1 0;
#P window setfont "Sans Serif" 9.;
#P number 377 237 35 9 0 0 0 3 0 0 0 221 221 221 222 222 222 0 0 0;
#P user jit.fpsgui 64 136 60 196617 0;
#P message 671 42 26 196617 500;
#P newex 671 21 45 196617 loadbang;
#P window setfont "Sans Serif" 12.;
#P window linecount 2;
#P comment 711 350 132 196620 simple video delay jonathan lee marcus;
#P toggle 28 38 15 0;
#P window setfont "Sans Serif" 9.;
#P window linecount 1;
#P newex 28 58 64 196617 qmetro 60;
#P message 139 58 38 196617 close;
#P message 96 36 124 196617 open , framerate 60.;
#P newex 28 96 238 196617 jit.qt.grab 320 240 @vmode 1 @unique 1;
#P user jit.pwindow 27 348 322 242 0 1 0 0 1 0;
#P comment 744 221 136 196617 < --less___delay___more-->;
#P message 685 184 42 196617 size $1;
#P newex 377 263 129 196617 expr (($i1+$i2)-$i3)%$i4;
#P newex 671 83 27 196617 + 1;
#P message 379 134 81 196617 dim 320 240 $1;
#P message 671 129 74 196617 max $1 , min 0;
#P comment 705 63 132 196617 < -maximum frames of delay;
#P number 671 61 35 9 0 0 0 3 0 0 0 221 221 221 222 222 222 0 0 0;
#P number 671 150 35 9 0 500 3 3 0 0 0 221 221 221 222 222 222 0 0 0;
#P comment 707 151 136 196617 < -number of frames to delay;
#P newex 377 209 50 196617 r rtdelay;
#P newex 363 307 262 196617 jit.matrix dave 4 char 320 240 @usesrcdim
1 @adapt 0;
#P message 377 286 205 196617 srcdimstart 0 0 $1 , srcdimend 319
239 $1;
#P user hslider 685 203 19 236 501 1 0 0;
#P newex 470 132 50 196617 s rtdelay;
#N counter 0 0 160;
#X flags 0 0;
#P newobj 380 87 82 196617 counter 0 0 160;
#P message 378 166 203 196617 dstdimstart 0 0 $1 , dstdimend 319
239 $1;
#P newex 363 188 283 196617 jit.matrix jack 4 char 320 240 161
@usedstdim 1 @interp 0;
#P newex 363 47 27 196617 t l b;
#P connect 24 0 23 0;
#P fasten 22 0 20 0 144 94 33 94;
#P fasten 21 0 20 0 101 94 33 94;
#P fasten 23 0 20 0 33 94 33 94;
#P connect 20 0 19 0;
#P fasten 20 0 28 0 33 125 69 125;
#P fasten 20 0 0 0 33 125 311 125 311 35 368 35;
#P lcolor 13;
#P fasten 14 0 1 0 384 154 368 154;
#P fasten 2 0 1 0 383 185 368 185;
#P connect 0 0 1 0;
#P lcolor 13;
#P connect 1 0 7 0;
#P lcolor 13;
#P connect 6 0 7 0;
#P connect 7 0 30 0;
#P connect 8 0 29 0;
#P connect 29 0 16 0;
#P connect 16 0 6 0;
#P fasten 3 0 2 0 385 112 374 112 374 160 383 160;
#P fasten 15 0 14 0 676 120 384 120;
#P connect 0 1 3 0;
#P fasten 15 0 16 1 676 114 663 114 663 235 421 235;
#P fasten 11 0 3 4 676 76 457 76;
#P fasten 10 0 16 2 676 245 460 245;
#P fasten 3 0 4 0 385 118 475 118;
#P fasten 15 0 16 3 676 114 660 114 660 224 499 224;
#P connect 26 0 27 0;
#P connect 27 0 11 0;
#P connect 11 0 15 0;
#P fasten 11 0 13 0 676 79 703 79 703 125 676 125;
#P connect 13 0 10 0;
#P fasten 5 0 10 0 690 227 667 227 667 147 676 147;
#P fasten 15 0 17 0 676 106 850 106 850 179 690 179;
#P connect 17 0 5 0;
#P pop;


December 17, 2007 | 9:36 pm


December 17, 2007 | 10:48 pm

For variable speed access, all you have to do is move an index up or
down at different rates. You can play around with a floating point
index and take the 2 closest frames which you would then blend in some
fashion according to the fractional amount of the index.

wes


December 20, 2007 | 9:37 pm


December 20, 2007 | 9:37 pm


December 21, 2007 | 1:01 am


December 21, 2007 | 7:38 pm

Inspired by this discussion I was playing with time buffers a bit, I thought you might like this one.

(save and load to fire the loadbangs)

Mattijs

#P window setfont "Sans Serif" 9.;
#P window linecount 1;
#P comment 48 30 96 196617 2) turn on metro;
#P message 207 233 199 196617 0. 0. 0. 0. 1. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.;
#P newex 207 214 48 196617 loadbang;
#P comment 270 77 96 196617 1) open video input;
#P number 49 185 35 9 0 0 0 3 0 0 0 221 221 221 222 222 222 0 0 0;
#P comment 386 113 65 196617 device input;
#N vpatcher 868 423 1556 1028;
#P window setfont "Sans Serif" 9.;
#P window linecount 1;
#P newex 399 128 31 196617 !/ 1.;
#P newex 399 107 34 196617 / 2.5;
#P newex 141 268 29 196617 t b f;
#P number 446 50 35 9 0 0 0 3 0 0 0 221 221 221 222 222 222 0 0 0;
#P newex 446 83 103 196617 prepend matrixcount;
#P newex 105 340 55 196617 jit.matrix;
#P newex 105 268 27 196617 t b l;
#P newex 105 318 65 196617 jit.op @op *;
#P newex 334 297 75 196617 prepend xshift;
#P newex 105 360 100 196617 jit.resamp @wrap 1;
#P newex 566 83 72 196617 prepend cheby;
#P newex 105 295 112 196617 jit.gradient 4 char 320;
#P newex 256 156 27 196617 – 1;
#P newex 256 125 27 196617 t i i;
#P newex 226 85 30 196617 t b b;
#P newex 256 202 34 196617 % 20;
#P newex 256 182 27 196617 + 0;
#P newex 199 395 48 196617 t clear b;
#P newex 237 415 40 196617 t 0 b 1;
#P newex 237 509 29 196617 gate;
#P newex 50 97 66 196617 jit.rgb2luma;
#P newex 272 482 27 196617 t l l;
#P newex 272 460 64 196617 jit.op @op +;
#P newex 334 275 34 196617 / 20.;
#P newex 226 105 40 196617 uzi 20;
#P newex 50 419 65 196617 jit.op @op *;
#P newex 256 222 107 196617 prepend outputmatrix;
#P newex 67 182 72 196617 prepend index;
#N counter 10;
#X flags 0 0;
#P newobj 67 140 66 196617 counter 10;
#P newex 50 119 27 196617 t l b;
#P newex 50 222 158 196617 jit.matrixset 20 4 char 160 120;
#P inlet 50 77 15 0;
#P inlet 226 65 15 0;
#P inlet 566 30 15 0;
#P inlet 446 30 15 0;
#P outlet 237 531 15 0;
#P outlet 263 342 15 0;
#P window linecount 0;
#P comment 501 532 134 196617 so , where’s the spider? :p;
#P connect 6 0 17 0;
#P connect 17 0 8 0;
#P connect 33 0 7 0;
#P connect 8 0 7 0;
#P connect 10 0 7 0;
#P connect 11 0 7 0;
#P connect 7 0 12 0;
#P connect 8 1 9 0;
#P connect 9 0 10 0;
#P connect 27 0 31 0;
#P connect 31 1 26 0;
#P connect 31 0 26 0;
#P connect 35 0 26 0;
#P connect 23 1 26 0;
#P connect 26 0 30 0;
#P connect 13 0 32 0;
#P connect 30 0 32 0;
#P connect 32 0 28 0;
#P connect 29 0 28 0;
#P connect 28 0 12 1;
#P connect 34 0 9 4;
#P connect 37 0 35 0;
#P connect 35 1 30 1;
#P connect 13 1 20 0;
#P connect 5 0 23 0;
#P connect 23 0 13 0;
#P connect 20 1 19 0;
#P connect 19 0 18 0;
#P connect 19 2 18 0;
#P connect 18 0 2 0;
#P connect 34 0 13 1;
#P connect 13 2 24 0;
#P connect 24 0 25 0;
#P connect 25 0 21 0;
#P connect 21 0 22 0;
#P connect 22 0 11 0;
#P connect 16 0 18 1;
#P connect 26 0 1 0;
#P connect 12 0 15 0;
#P connect 20 0 15 0;
#P connect 19 1 15 0;
#P connect 15 0 16 0;
#P connect 9 0 21 1;
#P connect 34 0 22 1;
#P connect 16 1 15 1;
#P connect 24 1 14 0;
#P connect 14 0 29 0;
#P connect 34 0 14 1;
#P connect 34 0 36 0;
#P connect 36 0 37 0;
#P connect 3 0 34 0;
#P connect 34 0 33 0;
#P connect 4 0 27 0;
#P pop;
#P newobj 13 205 64 196617 p timeripple;
#B color 5;
#P user jit.pwindow 206 287 82 62 0 1 0 0 1 0;
#P toggle 33 279 15 0;
#P newex 33 259 38 196617 sel 27;
#P newex 33 240 40 196617 key;
#P newex 33 297 93 196617 prepend fullscreen;
#P newex 13 318 58 196617 jit.window;
#P newex 49 165 66 196617 loadmess 30;
#P user multiSlider 207 251 102 32 0. 1. 16 2681 47 0 0 2 0 0 1;
#M frgb 0 0 0;
#M brgb 255 255 255;
#M rgb2 127 127 127;
#M rgb3 0 0 0;
#M rgb4 37 52 91;
#M rgb5 74 105 182;
#M rgb6 112 158 18;
#M rgb7 149 211 110;
#M rgb8 187 9 201;
#M rgb9 224 62 37;
#M rgb10 7 114 128;
#P user jit.fpsgui 91 50 60 196617 0;
#P newex 31 71 30 196617 t b b;
#P message 238 77 30 196617 open;
#P toggle 31 30 15 0;
#P newex 31 50 57 196617 qmetro 40;
#N vpatcher 605 381 1180 617;
#P window setfont "Sans Serif" 9.;
#P newex 407 71 48 196617 loadbang;
#P message 407 90 59 196617 getvdevlist;
#P newex 342 90 50 196617 deferlow;
#P newex 195 100 27 196617 t l b;
#P newex 23 99 27 196617 t l b;
#P newex 342 70 27 196617 t b l;
#P message 342 138 59 196617 getinputlist;
#P newex 23 120 25 196617 iter;
#P newex 104 141 38 196617 t clear;
#P newex 273 141 38 196617 t clear;
#P message 484 71 45 196617 input $1;
#P newex 195 141 75 196617 prepend append;
#P newex 195 120 25 196617 iter;
#P message 359 119 58 196617 vdevice $1;
#P newex 23 141 75 196617 prepend append;
#P newex 23 76 112 196617 route vdevlist inputlist;
#P inlet 23 52 15 0;
#P inlet 342 52 15 0;
#P inlet 484 52 15 0;
#P outlet 359 166 15 0;
#P outlet 23 163 15 0;
#P outlet 195 163 15 0;
#P connect 5 0 6 0;
#P connect 6 0 17 0;
#P connect 17 0 14 0;
#P connect 14 0 7 0;
#P connect 7 0 1 0;
#P connect 13 0 1 0;
#P connect 17 1 13 0;
#P connect 6 1 18 0;
#P connect 18 0 9 0;
#P connect 9 0 10 0;
#P connect 10 0 0 0;
#P connect 12 0 0 0;
#P connect 18 1 12 0;
#P connect 4 0 16 0;
#P connect 16 0 19 0;
#P connect 19 0 15 0;
#P connect 16 1 8 0;
#P connect 8 0 2 0;
#P connect 11 0 2 0;
#P connect 20 0 2 0;
#P connect 15 0 2 0;
#P connect 21 0 20 0;
#P connect 3 0 11 0;
#P pop;
#P hidden newobj 238 132 92 196617 p populate devices;
#P user ubumenu 238 113 145 196617 0 1 1 0;
#X add "Built-in iSight";
#X prefix_set 0 0 0;
#P user ubumenu 238 95 145 196617 0 1 1 0;
#X add "DV Video";
#X add "IIDC FireWire Video";
#X add "USB Video Class Video";
#X prefix_set 0 0
0;
#P newex 51 95 98 196617 jit.qt.grab 160 120;
#P comment 386 95 65 196617 device;
#P comment 86 185 341 196617 Time resolution. Watch out with values higher than 50 (on a macbook pro);
#P window linecount 2;
#P comment 311 251 102 196617 make your own time ripple patterns;
#P comment 54 116 115 196617 (you could exchange the qt.grab for a qt.movie);
#P connect 8 0 11 0;
#P connect 8 0 12 0;
#P connect 4 0 21 0;
#P connect 21 0 15 0;
#P connect 16 0 15 0;
#P connect 9 0 8 0;
#P connect 11 0 21 1;
#P connect 17 0 18 0;
#P connect 18 0 19 0;
#P connect 19 0 16 0;
#P connect 14 0 23 0;
#P connect 23 0 21 2;
#P connect 11 1 4 0;
#P hidden connect 7 2 4 0;
#P hidden connect 10 0 4 0;
#P connect 13 0 21 3;
#P connect 25 0 26 0;
#P connect 26 0 13 0;
#P connect 21 1 20 0;
#P hidden connect 7 0 5 0;
#P hidden connect 7 1 6 0;
#P hidden connect 4 1 7 0;
#P hidden connect 5 0 7 1;
#P hidden connect 6 0 7 2;
#P window clipboard copycount 28;


December 22, 2007 | 7:51 am

Hello,

Thank you again, a lot, for these explanations and the example.

>> But theorically I could (in "thru 0") recall the last frame with
>> "matrixoutput 0", or 1 or 2, etc and increase progressively my
>> delay- but apparently these frames can be read only when the
>> entire cycle is filled again. Did I miss someting ?
>>
>
> perhaps i misunderstand, but i think you are confusing the way
> matrixoutput and index works.
> thru should be off, so that we only see what we call for using
>
>
> jit.matrixset stores the incoming matrix at the position you
> specify in the buffer through
> the "index " message.

Yes, that was clear for me.

>
> "matrixoutput 0" outputs zeroth position of the buffer, not
> exactly previous frame as you mention.
> we need to keep track at which frame we just wrote into, so that we
> can think on more or less
> delay depending on that position. that’s why i used the + and %
> operator. it ensures the result
> wraps around the max num frames, while keeping the same distance
> from the frame just written.

But I didn’t get this point – I believed that the last frame was
always at index 0. Though it’s obvious that it can’t be…

I’m going to have a serious look at your patch, at timecube too (I
downloaded it yesterday), and I come again when all these questions
will be absolutely clear for me :-)

Thanks very much again Andre
Best

Thierry


December 22, 2007 | 1:49 pm

hi

>
>>
>> "matrixoutput 0" outputs zeroth position of the buffer, not
>> exactly previous frame as you mention.
>> we need to keep track at which frame we just wrote into, so that
>> we can think on more or less
>> delay depending on that position. that’s why i used the + and %
>> operator. it ensures the result
>> wraps around the max num frames, while keeping the same distance
>> from the frame just written.
>
> But I didn’t get this point – I believed that the last frame was
> always at index 0. Though it’s obvious that it can’t be…
>

this would be true for a fifo like behaviour, but the way the object
works,
it builds an internal array of matrices, that you can write to
position x (index x)
and read from position x-1, which gives you one frame delay (with
message ‘outputmatrix (x-1)’)

because x-1 might fall off the 0-maxframes interval (think on the
case writing to array pos 0,
reading would be pos -1) the + and % are the objs are used to make
sure if you write to pos 0,
you’ll read from pos max, in a 1 frame delay scenario..

it took me a long time to figure out this stuff, but its quite simple
in the end..

look at this in slow motion, it helped me a lot too. remember to use
max num for 0
frames delay, and decrease the number to increase the delay

#P window setfont "Sans Serif" 14.;
#P number 87 209 52 14 0 0 0 3 0 0 0 221 221 221 222 222 222 0 0 0;
#P user multiSlider 354 210 190 33 0. 10. 1 2920 15 0 0 2 0 0 0;
#M frgb 0 0 0;
#M brgb 255 255 255;
#M rgb2 127 127 127;
#M rgb3 0 0 0;
#M rgb4 37 52 91;
#M rgb5 74 105 182;
#M rgb6 112 158 18;
#M rgb7 149 211 110;
#M rgb8 187 9 201;
#M rgb9 224 62 37;
#M rgb10 7 114 128;
#P user multiSlider 354 168 190 33 0. 10. 1 2920 15 0 0 2 0 0 0;
#M frgb 0 0 0;
#M brgb 255 255 255;
#M rgb2 127 127 127;
#M rgb3 0 0 0;
#M rgb4 37 52 91;
#M rgb5 74 105 182;
#M rgb6 112 158 18;
#M rgb7 149 211 110;
#M rgb8 187 9 201;
#M rgb9 224 62 37;
#M rgb10 7 114 128;
#P window setfont "Sans Serif" 9.;
#P window linecount 1;
#N counter 10;
#X flags 0 0;
#P newobj 158 109 68 196617 counter 10;
#P toggle 158 40 15 0;
#P window setfont "Sans Serif" 14.;
#P newex 158 63 92 196622 qmetro 500;
#P number 158 247 52 14 0 0 0 3 0 0 0 221 221 221 222 222 222 0 0 0;
#P newex 158 208 48 196622 % 11;
#P number 158 146 52 14 0 0 0 3 0 0 0 221 221 221 222 222 222 0 0 0;
#P number 220 178 52 14 0 0 0 3 0 0 0 221 221 221 222 222 222 0 0 0;
#P newex 158 176 48 196622 + 0;
#P connect 0 0 10 0;
#P connect 0 0 3 0;
#P connect 3 0 4 0;
#P connect 5 0 7 0;
#P connect 7 0 2 0;
#P connect 4 0 9 0;
#P connect 2 0 0 0;
#P connect 2 0 8 0;
#P connect 1 0 0 1;
#P connect 6 0 5 0;
#P window clipboard copycount 11;

> I’m going to have a serious look at your patch, at timecube too (I
> downloaded it yesterday), and I come again when all these questions
> will be absolutely clear for me :-)
>

have fun!:)

> Thanks very much again Andre
> Best
>

welcome, and thank you Mattijs for a very cool patch!

a

> Thierry
>


December 22, 2007 | 7:06 pm

Hi Mattijs,

Thanks a lot ! The effect is nice… I have to enter the spider now ;-)

Thierry Fournier

http://www.thierryfournier.net


December 22, 2007 | 7:19 pm


February 9, 2008 | 1:43 pm

Hello,

Just a (late) message for thanking Andre Sier, Mattijs Kneppers and
Wesley Smith for your wise advices about realtime video buffer, one
month ago. I worked again on this project after some weeks of
interruption. Everything’s was conceptually clear enough to define
the functions of the patches. Mathieu Chamagne will work with us, on
the project software.

This small installation, which is called "Point d’orgue" ("Fermata"),
will be premiered in Montpellier (France), in November, and I’ll keep
you informed.

Best & thanks again,

Thierry

—————————————-
Thierry Fournier

http://www.thierryfournier.net


August 9, 2011 | 10:55 pm

hey Mattijs, that is a great Patch!


Viewing 13 posts - 1 through 13 (of 13 total)