Chromakey resolutions

May 6, 2006 at 4:17pm

Chromakey resolutions

Hi all,

I’m having a problem with this patch in which a live webcam feed from an
Apple i-sight gets keyed in over a pre-edited video loop.

The problem is that when I run the video loop through the chromakey setup it
gets output at the wring res and looks shocking. I’m guessing this may be to
do with the fact the the video is 720×576 PAL res, whilst the webcam is
320×240. How does the chromakey object handle different resolutions? And is
there a way I can scale the webcam feed up to match the res of the video
feed?

I’ve added Gswitches into my patch for diagnostics. If I bypass the
chromakey, the video plays as well as it does in Quicktime player.

Thanks,
Jonathan

max v2;
#N vpatcher 422 44 1237 983;
#P origin 107 46;
#P user gswitch 484 584 41 32 1 0;
#P user gswitch 409 585 41 32 1 0;
#P window setfont “Sans Serif” 9.;
#P window linecount 1;
#P message 494 159 43 196617 window;
#P window linecount 3;
#P comment 546 159 136 196617 window with no arguments sets it back to
rendering to and outputting a matrix;
#P window linecount 1;
#P message 463 64 92 196617 window fullscreen;
#P window linecount 6;
#P comment 546 65 214 196617 window sets the qt movie to
render directly into the named window , thus able to use hardware
decompression and interpolation as permitted by QuickTime. The window must
have noaccel set to 1 in order for this to work.;
#P window linecount 1;
#P newex 587 845 32 196617 print;
#P message 689 693 50 196617 getname;
#P message 604 694 81 196617 name fullscreen;
#P newex 420 892 32 196617 print;
#P message 318 635 50 196617 getname;
#P message 241 635 73 196617 name preview;
#P comment 245 45 94 196617 FLIP WEBCAM FEED;
#B frgb 231 1 28;
#P toggle 303 79 15 0;
#P toggle 266 78 15 0;
#P newex 241 102 72 196617 pak invert 0 0;
#P newex 76 134 58 196617 jit.dimmap;
#B color 5;
#P window linecount 2;
#P comment 95 33 100 196617 Turn on Live Webcam Feed & Open;
#B frgb 255 9 15;
#P comment 131 626 100 196617 Turn Preview Window On/Off;
#B frgb 255 9 15;
#P user gswitch 86 623 41 32 0 0;
#P button 483 670 15 0;
#P window linecount 1;
#P newex 431 671 48 196617 loadbang;
#P message 528 695 68 196617 fullscreen $1;
#P toggle 511 695 15 0;
#P newex 553 670 35 196617 sel 27;
#P newex 511 670 40 196617 key;
#P newex 418 818 179 196617 jit.window @border 0 @size 320 240;
#B color 5;
#P message 431 697 66 196617 fsmenubar 0;
#P window linecount 2;
#P comment 190 330 34 196617 Select Black;
#B frgb 231 1 28;
#P window linecount 1;
#P comment 171 452 17 196617 1;
#B frgb 231 1 28;
#P newex 75 263 142 196617 jit.matrix 4 float32 320 240;
#P flonum 169 468 108 9 0 0 0 3 0 0 0 221 221 221 222 222 222 0 0 0;
#P flonum 232 485 44 9 0 0 0 3 0 0 0 221 221 221 222 222 222 0 0 0;
#P message 232 505 73 196617 slide_down $1;
#P message 169 505 60 196617 slide_up $1;
#P newex 104 548 43 196617 jit.slide;
#B color 5;
#P newex 75 496 64 196617 jit.op @op ~;
#P hidden message 493 389 21 196617 0.1;
#P hidden message 448 389 21 196617 0.2;
#P hidden newex 448 348 44 196617 loadbang;
#P newex 324 378 93 196617 unpack 0. 0. 0.;
#P newex 324 355 77 196617 vexpr $i1/255.;
#P user swatch 99 362 128 32;
#P user suckah 99 295 80 60;
#P user jit.pwindow 98 294 82 62 0 0 0 0 1 0;
#P message 192 405 41 196617 mode 2;
#P toggle 664 409 15 0;
#P message 664 443 75 196617 alphaignore $1;
#P user jit.fpsgui 18 878 60 196617 0;
#P message 311 231 28 196617 stop;
#P message 147 405 40 196617 mode 1;
#P message 102 405 40 196617 mode 0;
#P flonum 493 409 35 9 0 0 0 3 0 0 0 221 221 221 222 222 222 0 0 0;
#P message 493 443 42 196617 fade $1;
#P flonum 602 409 35 9 0 0 0 3 0 0 0 221 221 221 222 222 222 0 0 0;
#P flonum 545 409 35 9 0 0 0 3 0 0 0 221 221 221 222 222 222 0 0 0;
#P message 602 443 58 196617 maxkey $1;
#P message 545 443 55 196617 minkey $1;
#P flonum 448 410 35 9 0 0 0 3 0 0 0 221 221 221 222 222 222 0 0 0;
#P message 448 444 35 196617 tol $1;
#P flonum 406 410 35 9 0 0 0 3 0 0 0 221 221 221 222 222 222 0 0 0;
#P flonum 365 410 35 9 0 0 0 3 0 0 0 221 221 221 222 222 222 0 0 0;
#P flonum 324 410 35 9 0 0 0 3 0 0 0 221 221 221 222 222 222 0 0 0;
#P flonum 283 410 35 9 0 0 0 3 0 0 0 221 221 221 222 222 222 0 0 0;
#P newex 242 444 177 196617 pak color 0. 0. 0. 0.;
#P user jit.pwindow 85 667 322 242 0 1 0 0 1 0;
#X name u662000175;
#P newex 86 579 71 196617 jit.chromakey;
#B color 5;
#P flonum 343 210 35 9 0 0 0 3 0 0 0 221 221 221 222 222 222 0 0 0;
#P message 343 231 42 196617 rate $1;
#P message 280 231 28 196617 read;
#P newex 269 262 105 196617 jit.qt.movie 320 240;
#P comment 153 229 70 196617 Binary edge;
#B frgb 255 255 255;
#P newex 76 225 70 196617 cv.jit.binedge;
#P window linecount 3;
#P comment 211 146 105 196617 Change thresholding value to obtain a binary
image;
#P number 167 157 35 9 0 0 0 3 0 0 0 221 221 221 222 222 222 0 0 0;
#P window linecount 1;
#P newex 76 191 101 196617 jit.op @op > @val 50;
#P message 150 92 44 196617 settings;
#P message 163 61 31 196617 close;
#P message 132 61 28 196617 open;
#P toggle 76 36 15 0;
#P newex 76 60 49 196617 qmetro 2;
#P newex 76 90 54 196617 jit.qt.grab;
#P user panel 55 29 162 94;
#X brgb 255 253 141;
#X frgb 0 0 0;
#X border 0;
#X rounded 0;
#X shadow 0;
#X done;
#P user panel 165 139 157 49;
#X brgb 198 198 198;
#X frgb 140 140 140;
#X border 1;
#X rounded 0;
#X shadow 0;
#X done;
#P newex 76 167 64 196617 jit.rgb2luma;
#P comment 380 264 100 196617 Background Movie;
#B frgb 251 52 48;
#P comment 280 486 100 196617 Adjust trail effect;
#B frgb 225 59 55;
#P comment 195 487 34 196617 2790;
#B frgb 231 1 28;
#P window linecount 2;
#P comment 455 634 100 196617 Hit ESC in run mode for fullscreen;
#B frgb 255 9 15;
#P window linecount 1;
#P comment 243 64 53 196617 Horizontal;
#P comment 300 64 42 196617 Vertical;
#P window linecount 8;
#P comment 349 43 66 196617 Horizontal: On = mirror mode Off = Opposite
Vertical: On = upside down Off = Correct;
#P connect 26 0 43 0;
#P connect 19 0 61 0;
#P connect 61 0 55 0;
#P connect 12 0 11 0;
#P connect 11 0 10 0;
#P hidden connect 15 0 10 0;
#P fasten 13 0 10 0 137 83 81 83;
#P fasten 14 0 10 0 168 83 81 83;
#P fasten 76 0 75 0 246 127 81 127;
#P connect 10 0 75 0;
#P connect 75 0 7 0;
#P connect 7 0 16 0;
#P fasten 16 0 19 0 81 221 81 221;
#P connect 56 0 25 0;
#P fasten 27 0 25 0 247 430 91 430;
#P fasten 46 0 25 0 197 428 146 428 91 428;
#P fasten 44 0 25 0 669 430 91 430;
#P fasten 32 0 25 0 453 430 91 430;
#P fasten 38 0 25 0 498 430 91 430;
#P fasten 34 0 25 0 550 430 91 430;
#P fasten 35 0 25 0 607 430 91 430;
#P fasten 40 0 25 0 107 428 91 428;
#P fasten 41 0 25 0 152 428 91 428;
#P connect 72 0 26 0;
#P fasten 80 0 26 0 246 660 91 660;
#P fasten 81 0 26 0 323 658 323 661 91 661;
#P fasten 19 0 47 0 81 256 71 256 71 288 104 288;
#P connect 48 0 49 0;
#P fasten 55 0 56 0 80 533 109 533;
#P fasten 57 0 56 0 174 525 109 525;
#P fasten 58 0 56 0 237 525 109 525;
#P fasten 25 0 72 2 91 612 121 612;
#P fasten 21 0 25 1 274 435 152 435;
#P fasten 17 0 16 1 172 186 172 186;
#P connect 60 0 57 0;
#P connect 59 0 58 0;
#P hidden fasten 52 0 80 0 453 374 444 374 444 509 323 509 323 621 246 621;
#P hidden fasten 11 0 21 0 81 148 202 148 202 216 274 216;
#P connect 89 0 21 0;
#P connect 87 0 21 0;
#P fasten 22 0 21 0 285 259 274 259;
#P fasten 23 0 21 0 348 259 274 259;
#P fasten 42 0 21 0 316 259 274 259;
#P fasten 77 0 76 1 271 98 277 98;
#P connect 28 0 27 1;
#P connect 78 0 76 2;
#P hidden fasten 52 0 81 0 453 375 444 375 444 509 323 509;
#P fasten 49 0 50 0 104 399 238 399 238 334 329 334;
#P connect 50 0 51 0;
#P connect 51 0 29 0;
#P connect 29 0 27 2;
#P connect 24 0 23 0;
#P connect 51 1 30 0;
#P connect 30 0 27 3;
#P connect 51 2 31 0;
#P connect 31 0 27 4;
#P connect 91 0 65 0;
#P connect 90 0 65 0;
#P fasten 84 0 65 0 694 716 686 720 423 720;
#P fasten 83 0 65 0 609 720 423 720;
#P fasten 69 0 65 0 533 715 423 715;
#P fasten 64 0 65 0 436 715 423 715;
#P connect 26 1 82 0;
#P connect 25 0 90 1;
#P connect 70 0 64 0;
#P fasten 71 0 64 0 488 691 436 691;
#P hidden connect 52 0 53 0;
#P hidden connect 53 0 33 0;
#P connect 33 0 32 0;
#P hidden connect 52 0 54 0;
#P hidden connect 54 0 39 0;
#P connect 39 0 38 0;
#P connect 21 0 91 1;
#P fasten 67 0 68 0 558 692 516 692;
#P fasten 68 0 69 0 527 710 527 695;
#P connect 36 0 34 0;
#P fasten 66 0 67 0 552 687 552 672;
#P connect 65 1 85 0;
#P connect 37 0 35 0;
#P connect 45 0 44 0;
#P pop;

#25815
May 6, 2006 at 4:50pm

the iSight can do 640 x 480. Might want to bump that up. From there,
one way to match the two sources is to interpolate one of them up or
down before sending to the chromakey, e.g.,

[jit.qt.grab 640 480] –> [jit.matrix foo 720 576]

or

[jit.qt.movie @adapt 1] –> [jit.matrix foo 640 480]

or just

[jit.qt.movie 640 480]

offhand, i don’t know how doing the interpolation in jit.qt.movie
differs from doing it in a jit.matrix.

best,
dan


***
http://danwinckler.com
http://share.dj

http://gunshowimprov.com

#76501
May 6, 2006 at 4:52pm

oops, i think you need to do @interp 1 in those foo matrices.


***
http://danwinckler.com
http://share.dj

http://gunshowimprov.com

#76502
May 6, 2006 at 10:01pm

My problem seems to be more complex than I originally thought. It seems that
all my res issues seem to stem from the branch of data coming into the
chromakey from the webcam feed. Even though the webcam data and the video
loop are kept entirely separate until the chromakey object, the res of the
webcam feed seems to effect the res of my video loop.

If I change the webcam feed res before it arrives at the chromakey to
anything above 320×240 then the video loop becomes stuttery, missing frames
and the picture can’t keep up with the sound. If I leave the webcam feed
as-is (which I don’t mind doing in theory), then the video loop displays at
this same low resolution (even though it should be 720×576), but smoothly. I
can’t figure this out, it’s very frustrating!

Any ideas? My patch is below.

#76503
May 7, 2006 at 10:03am

hello,

few things:

- you’ve limited jit.qt.movie containing your background to 320×240.
initialize it with the matrix size you want(jit.qt.movie 720 576).

- every jitter object having adapt attribute set to 1(@adapt 1) _adapts_ to
the properties of the matrix coming in their leftmost inlet. This means that
if you feed 320×240 matrix into left inlet of chromakey, and 720×576 in
right inlet, jit.chromakey will adapt to 320×240 one.
One solution to this is to resize that matrix just before it goes to
chromakey(after jit.slide) in your case.

you’ve noticed frame drop when you resize matrix coming to jit.chromakey. on
of the reasons this is happening is that your matrices are of the type
float32, which takes 32 bits, instead of 8bits as char type needs, so if we
skip everything else, amount of data coming into jit.chromakey is 4 times
bigger then necessary.
One solution to this is to set the matrix type to char just before it goes
to chromakey(after jit.slide) in your case.

if you do these two solutions, you’ll notice that you’re using float32 just
for jit.slide, but the same effect can be achieved with jit.glop or jit.wake
only with less cpu.

- maybe jit.chromakey is not needed at all: it seems that you want to ‘draw’
edges of incoming web camera stream, which are white pixels. if you want to
‘compose’ something white-with-black-background over something else, you
could use jit.op @op + instead of jit.chromakey. using this trick, you can
avoid two solutions offered above by feeding your properly sized background
movie to left inlet of jit.op, and whatever you want to put over it in the
right inlet(this way jit.op internally scales right-input matrix to
left-input matrix size)

- delete all pwindows:)

hope this helps,
nesa

#76504
May 7, 2006 at 10:56am

Hi Nesa

> – maybe jit.chromakey is not needed at all: it seems that you want to ‘draw’
> edges of incoming web camera stream, which are white pixels. if you want to
> ‘compose’ something white-with-black-background over something else, you
> could use jit.op @op + instead of jit.chromakey. using this trick, you can
> avoid two solutions offered above by feeding your properly sized background
> movie to left inlet of jit.op, and whatever you want to put over it in the
> right inlet(this way jit.op internally scales right-input matrix to
> left-input matrix size)

Thanks so much, this trick has worked wonders! I always thought there must
be a better way round this than to use chromakey, but I hadn’t really got to
grips with understanding jit.op, although it’s now becoming increasingly
apparent to me how important this object is!

> One solution to this is to set the matrix type to char just before it goes
> to chromakey(after jit.slide) in your case.

Thanks for solving another newb error too….now my fan isn’t going into
overdrive when I try to run my patch on my poor little Mac Mini!

Thanks again,
Jonathan

#76505

You must be logged in to reply to this topic.