alpha video layering with motion tracking

Apr 8, 2012 at 1:17pm

alpha video layering with motion tracking

hey guys, i wanna do some video layering stuff based on motion tracking. there are video1&2. video1 is just playing on screen as primary layer. once visitors are on an area tracked by a camera, their body shadow will be tracked and become a secondary layer for playing video2 that directly overlays video1.

to achieve the task, i tried to use background subtraction to get binary video image for controlling alpha channel of B and W area. then, B is filled up by video1, and W is filled up by video2.

but..there are 2 problems:
1) background subtraction seems not the best way to do the stuff… the binary video image is always quite creepy (for example my black hair can’t be “tracked” because its RGB is 0…) i can hardly get a fairly good “white person on black BG image”. so instead, is it workable (i bet it is!) to track the “changed” pixel? which means once there is a change of pixel compared with BG image, it is tracked. what is the approach?

2) i can’t exactly do the video layering….i can just make 2 separated windows, which are either video1/2 layering with either B/W area in binary video, by using jit.alphablend…but i can’t join them together…

would anyone help me please?

#62837
Apr 8, 2012 at 3:43pm

here’s one way to do background subtraction with jit.op @op absdiff

it uses a cv object called cv.dilate, which is not necessary for this technique, but might help.
also look at cv.erode (sort of the inverse operation of dilate).

if you don’t have the cv objects (well, you should) just remove those objects from the patch.

also, i just posted this a few days ago, so make sure you search the forum before posting.

– Pasted Max Patch, click to expand. –
#226844
Apr 8, 2012 at 6:48pm

thanks for the suggestion!

for the video layering, the patch you suggested is to overlay a camera tracked person on a video. then how about overlaying another video on the camera tracked layer (so that another video is supposed to just reveal the part on the tracked layer)?

#226845
Apr 9, 2012 at 3:18am

i don’t really understand your question, however i recommend spending some time understanding what’s going on here. once you understand how the alpha channel is used, you should be able to figure it out any combination of layers and transparency.

post a patch showing what you’ve tried and we’ll be able to offer more suggestions.

#226846
Apr 9, 2012 at 1:52pm

i hope my patch will make things clear! please have a look. many thanks!

– Pasted Max Patch, click to expand. –
#226847
Apr 9, 2012 at 1:56pm

ps: I tried to use 2 jit.alphablend to add 2 video layers as u can see in the patch. but obviously it failed…

#226848
Apr 9, 2012 at 6:02pm

i assume you mean something like the patch below.
please note, your matrix dimensions need to all match up for this to work properly.
you had your grab at 160×120 and your qt.movies at the default of 320×240.

– Pasted Max Patch, click to expand. –
#226849
Apr 10, 2012 at 9:27am

oh yes….thanks so much Robert! you’re my hero!

#226850

You must be logged in to reply to this topic.