cv.jit.blobs.recon: extract/substract people from blob detection

May 21, 2010 at 6:26pm

cv.jit.blobs.recon: extract/substract people from blob detection

hello!

in an ongoing project of mine I’ve “paved” a white wall with black squares (movable via magnets). when moving these squares around, sounds arise (with the help of [cv.jit.blobs.bounds]. of course my sounds should only go along with the squares being shoved around – and NOT be triggered by the mere movement of the people entering the room :)

this is hard to realize just by using [cv.jit.blobs.bounds] – any idea how to extract/substract people from “blob detection”?

I’ve tried working this out with [cv.jit.blobs.recon] – and am far from achieving satisfying/stable results..

grateful for any further hints..!

all the best

-jonas

#50499
May 21, 2010 at 7:19pm

if you have stable light conditions
you could use for the background subtraction maybe only one background picture
and then limit the with cv.jit.blobs.bounds recognized blobs to a certain size and proportion

#181152
May 21, 2010 at 7:32pm

hey xerox! thanks for the quick reply

my light conditions will indeed be stable.. but I didn’t quite understand what you mean with “using only one background picture”..

cheers..!

-jonas

#181153
May 23, 2010 at 3:06am

i mean the background picture for the difference picture
(you take a snapshot of the environment without objects and subtract then current frames from it so you get an picture with only the objects which weren’t there when you took the background picture)
if the light is stable you dont have to update your background picture

#181154
May 24, 2010 at 1:43pm

thanks, zerox! searching the forum via “background” + “substraction” did give me quite a constructive insight..

now I hust have to find out how to predefine/limit the blob sizes to be detected (the answer should lay in modifying cv.jit.blobs.bounds.draw)..?

cheers!

-j

#181155
May 24, 2010 at 9:11pm

so I’ve tried to modify cv.jit.blobs.bounds.draw (turning jean-marc’s abstraction into a subpatcher)

unfortunately I wasn’t able to make cv.jit.blobs.bounds limit its recognized blobs to a certain size and proportion. mainly focused on fooling around with subpatcher prepare_for_lcd’s [jit.iter] output

I’d say I’m back to: HELP.

cheers..

-jonas

#181156
May 24, 2010 at 10:40pm

ok. here is an quick test
the single blobs are not sorted
the first id is always at the upper left corner
it would be nicer to sort them via cv.jit.blobs.sort and grab the single blobs with jit.findbounds
i would prefer to do this dynamically in java
maybe somebody could answer this question ;-) http://cycling74.com/forums/topic.php?id=26704

[mod - removed uncompressed patch]

#181157
May 25, 2010 at 9:25am

hej xerox!

I’ve tried copying ‘n pasting the code a couple of times, without success.. something might be missing at its tail..? (I wouldn’t know for I’m used to the ‘copy compressed’ outlook)

cheers in advance..! :)

-jonas

#181158
May 25, 2010 at 12:10pm

sorry. i didn’t use “copy compressed”

– Pasted Max Patch, click to expand. –
#181159
May 25, 2010 at 1:04pm

:)

thanks, xerox! 2 in 1, nice.. I’m at work right now, will have a closer look at your patch as soon as I get home. and post a version that suits me best (if any modification necessary at all..)

can’t help you with the java part though. (: although I’ve found this neat patch for blob sorting issues recently..

cheers!

– Pasted Max Patch, click to expand. –
#181160
May 29, 2010 at 10:35am

hey xerox!

sorry for the late reply.. I’ve been quite busy this week (haven’t found time to deal with my request until now)..

I’m curious: what’s the [unpack] [pack] bit in your “p drawbounds” for? (+4 and +14)

cheers! :)

-jonas

#181161
May 29, 2010 at 12:39pm

it sets the position of the drawn “write $1″ id number

#181162
May 29, 2010 at 4:09pm

If you are trying to isolate each individual blob recognised I have just achieved something like that successfully.

I’ve been using cv.jit.blobs.centroids to find each blob in the space, and then several instances of cv.jit.label to separate the blobs into their own matrices, making it easy for me to then extract x/y locations from each blob and use them as controllers.

I’m relatively new to Max and this is the way I have found to be very successful in turning objects interpreted by video into trackable controllers for sound.

#181163
May 29, 2010 at 7:17pm

hej orrinward!

nice! :) if I get you right, my approach comes from the opposite direction (bg subtraction).. with the help of cv.jit.blob.bounds I’ll extract every blob that suits my size/proportions. that way I will have quite an amount of blobs detected, but only the important one’s will be given “sound admission” – no people interfering.. but therefore my light has to be absolutely stable

I’m quite new to jitter – wouldn’t call this modified, just a “tidy” version of xerox’s patch:

– Pasted Max Patch, click to expand. –

now I’ll have to make jitter not losen its grip on my detected shapes so one can play around with real objects without destroying one’s audio engineering by his mere presence :)

could you maybe post an example patch? cv.jit.blob.centroids is way appealing for being the easiest way of converting object movement into anything you want (although I’m having the most fun with audio engineering at the moment)..

cheers! & all the best

-jonas

#181164
May 29, 2010 at 7:28pm

Yes of course I can. I’ve been using background subtraction as well, but for that I was using the patches found here – http://cycling74.com/2009/10/26/making-connections-camera-data/

– Pasted Max Patch, click to expand. –

Excuse the junk in there, it’s a rough patch. The small top jit.pwindow shows all recognised centroids. the large left pwindow shows background subtraction (tweaked with the gate and background coefficient)and in the bottom right you see 4 pwindows, showing individual labels.

I’ve also shown an example of converting the matrix data to control a pictslider. This is what I have used in my Sound Social installation, http://www.orrinward.co.uk/soundsocial

This patch is the starting point that has allowed me to quite accurately use people on surveillance footage to control the location of sounds within a networked sound environment. It sounds like your setup is built to have clear contrasting image differences so this could work even better with your setup. It works successfully at tracking individual objects on a low-res video feed with a lot of flicker, so yours should be fine.

My patch was made for PC and I just noticed yours is for Mac, so be sure to change any dx.grab to qt.grab

#181165
May 29, 2010 at 7:49pm

thanks for sharing, orrinward! and nice installation :)

my internet died on me, THEREFORE I’m at a friends, hehe (and wouldn’t have expected to be as lucky as to be given new input/inspiration on my project during my short visit..)

I just now realize how alarmingly uncomfortable I feel without THE WEB. ;)

I’ll have a closer look at all of this.. although I haven’t made much patching myself in this particular case, I’m happy :) progress (even initial observation) in max is so much fun..

and thanks again for further explanation, xerox!

see you soon

-j

#181166
May 30, 2010 at 3:25pm

hey orrin!

I did some combining :)

but it gives me the feeling of inefficiency (cv.jit.blobs.bounds seems very task intensive). and I’m having trouble of labeling my blobs in a proper way (my blob labels should stick to their objects after initial detection, no matter where they move to). the swapping has to be dealt with..

see you soon!

-jonas

– Pasted Max Patch, click to expand. –
#181167
May 31, 2010 at 1:35pm

for anyone who’s interested :) so far so good. my blob labels still tend to swap horizontally though. meaning: blobs having passed detection requirements (size/proportion) change labels after “overtaking” one another, being moved up & down.. I’ll search the forum, this problem should’ve been dealt with already

most annoying though: can it be that cv.jit.blobs.bounds eats alot of CPU? camera feed doesn’t suffer at all – but further patching is quite nerve-wracking (placing cords becoming quite a challenge due extreme latency)..

any help appreciated..!

all the best! -j

– Pasted Max Patch, click to expand. –
#181168

You must be logged in to reply to this topic.