How to use webcam as background during motion tracking process?

Jialin Yang's icon

Hi,

I am a uni student and this is my final project are preparing to submit. Actually, I have finished motion tracking part. But I want to make effect such as the using webcam image as background. There is a example of this effect: https://www.youtube.com/watch?v=Q3gT52sHDI4

Do you have any advice on it?

Thanks, Aria

final-project-version-1.maxpat
Max Patch
Jialin Yang's icon
Wil's icon

Hi Aria. I just finished Liubo's 1hour 40min tutorial on blob tracking. I stopped short of the last part where he explains how render the blobs as objects cause I just want to use the x y for something else. But I saw your patch and see that you went through the entire tutorial.

I worked this morning on creating objects so I could look for a solution to what you want to do but having problems getting the blobs to render in the new window. If no one else replies here I might have something in a few days.

I am doing this because well maybe we can share info. I am trying to track the blobs so I can control other max objects like pitch shifting, volume and frequency shifting etc for live performance. but the data coming from the blobs is super erratic and not very accurate. maybe you found a way to make it more precise. for example, i can't find a way to have only one blob... thats is all , just one. and i want to able to control that one blob with a high level of precision so that if i move my hand (or an object in my hand) up/down left /right the numbers precisely change up/down.

Max Patch
Copy patch and select New From Clipboard in Max.

here is everything i have done so far (same as yours), minus the object rendering. i would like that your patch is much cleaner.
`presentation mode as well. I am going to add the image rendering next.

Wil's icon

hi aria. i did some more work on this patch (using it for my work as well). added the video to the background of the blob. also, now the blob stops whenever the motion stops. there is still a weird flickering but i will figure it out.

if anyone else can determine the flickering any help would be appreciated!

blob3.maxpat
Max Patch
blob31.maxpat
Max Patch
blob32.maxpat
Max Patch
Wil's icon

...and how do you remove a previous attachment? this is the right one.

blob33.maxpat
Max Patch
Jialin Yang's icon

Hi Wil,
Sorry for late replay, I just download your patch and will try to find a way to solve a wired flickering. But I guess probably the problems exist in the sub-patch "p-position". I still need some time to figure out that where problem exist. Here is the original patch for "p-position". You can check it first and we can find a way to solve problems together.

Aria

Untitled1.maxpat
Max Patch
Jialin Yang's icon

Hi Wil,
One more question is....How can I color in blobs?

Wil's icon

hi aria. i been working on this patch a lot. but I'm not using the p-position part of it. I did however 'steal' some other code to make this cool trail behind the blob that i really like but not sure if i will ultimately use it. the flickering is gone now and made a presentation mode. i added some smoothing to help keep the blob from jumping around. but its still all over the place. like i said before my patch is controlling audio so there is a simple cycle~ for testing amplitude and pitch. (might want to turn it down a bit)

there is a prepend color in the p-process patch but i don't understand where the color os coming from and each of your blobs is a different color. have to figure that one out. i'll take a look at it.

i have been working with just audio for the last few years so this is new for me working with motion and video. but like yourself, i know exactly what i want it to do so it is just a matter of figuring it out.

here is my new patch. i try to set the setting so there is only one blob. but it changes depending on the lighting. i am mostly working in dark places. it seems for now that you have to reset the size (80 60 menu) before the blob tracks. it also works in presentation mode. all new code that i borrowed and changed is in pink.

maybe you can reuse some of this.

blob10.maxpat
Max Patch
Wil's icon
Max Patch
Copy patch and select New From Clipboard in Max.

hi aria. here is how to change the colors;