Forums > Jitter

cv.jit & Motion Tracking: SLOWLY adapting the background


d_s
October 11, 2006 | 6:57 am

Hello all,

I am working on an installation that features motion tracking using a background subtraction method and JM Pelletier’s cv.jit. The patch included below is directly from Felix Luque and is referenced in this post: http://tinyurl.com/f4lvo

My question is this: I want to very slowly adapt the background image that is being subtracted in this example, but no matter how high I set the inputs to the jit.slide object the subtracted image adapts in about 2 seconds. My goal is to have a situation where the background image adapts at about 0.05% per frame, or where it takes about 1 minute for a complete recycling of the bg image.

Can anyone help?

Many thanks,
d_s

#P window setfont "Sans Serif" 9.;
#P flonum 172 157 35 9 0 0 0 139 0 0 0 221 221 221 222 222 222 0 0 0;
#P window linecount 1;
#P message 172 183 66 9109513 slide_down $1;
#P user jit.pwindow 38 486 162 122 0 0 0 0 1 0;
#P comment 48 437 272 9109513 average color components and work with greyscale difference;
#P newex 39 457 56 9109513 jit.rgb2luma;
#P message 202 66 45 9109513 settings;
#P message 160 66 32 9109513 close;
#P comment 205 730 100 9109513 mask;
#P comment 183 623 93 9109513 threshold;
#P user jit.pwindow 38 673 162 122 0 0 0 0 1 0;
#P flonum 133 623 28 9 0 0 0 139 0 0 0 221 221 221 222 222 222 0 0 0;
#P newex 39 645 104 9109513 jit.op @op > @val 0.15;
#P window linecount 2;
#P comment 199 93 146 9109513 background averaging coefficient (larger values smoother);
#P flonum 99 158 35 9 0 0 0 139 0 0 0 221 221 221 222 222 222 0 0 0;
#P window linecount 1;
#P message 99 183 55 9109513 slide_up $1;
#P newex 58 207 171 9109513 jit.slide @slide_up 2 @slide_down 2;
#P toggle 58 137 15 0;
#P newex 58 157 27 9109513 gate;
#P newex 39 240 88 9109513 jit.op @op absdiff;
#P message 117 66 29 9109513 open;
#P flonum 73 46 35 9 0.5 0 1 139 0 0 0 221 221 221 222 222 222 0 0 0;
#P toggle 39 46 15 0;
#P newex 39 66 44 9109513 metro 2;
#P newex 39 104 85 9109513 jit.dx.grab 160 120;
#P user jit.pwindow 38 269 162 122 0 0 0 0 1 0;
#P comment 207 280 100 9109513 difference;
#P window linecount 2;
#P comment 89 125 176 9109513 to set background , open gate for some period of time and then close gate;
#P connect 24 0 15 0;
#P fasten 25 0 11 0 177 203 63 203;
#P connect 26 0 25 0;
#P connect 3 0 8 0;
#P fasten 3 0 9 1 44 129 80 129;
#P fasten 4 0 3 0 44 97 44 97;
#P fasten 7 0 3 0 122 92 44 92;
#P fasten 20 0 3 0 165 92 44 92;
#P fasten 21 0 3 0 207 92 44 92;
#P connect 16 0 15 1;
#P fasten 11 0 8 1 63 233 122 233;
#P connect 13 0 12 0;
#P connect 6 0 4 1;
#P fasten 12 0 11 0 104 204 63 204;
#P connect 9 0 11 0;
#P connect 10 0 9 0;
#P connect 15 0 17 0;
#P connect 22 0 24 0;
#P connect 2 0 22 0;
#P connect 8 0 2 0;
#P connect 5 0 4 0;
#P window clipboard copycount 27;


October 11, 2006 | 7:17 am

Try downloading the new cv.jit. I would use cv.jit.ravg.

wes

#P window setfont "Sans Serif" 9.;
#P window linecount 1;
#P newex 51 130 66 196617 jit.rgb2luma;
#P newex 51 185 65 196617 jit.op @op *;
#P newex 51 208 123 196617 cv.jit.ravg @alpha 0.001;
#P message 172 70 32 196617 close;
#P message 129 70 29 196617 open;
#P flonum 85 50 35 9 0.5 0 1 3 0 0 0 221 221 221 222 222 222 0 0 0;
#P toggle 51 50 15 0;
#P newex 51 70 57 196617 qmetro 30;
#P newex 51 108 98 196617 jit.qt.grab 160 120;
#P comment 569 417 87 196617 Motion Mask;
#P comment 329 418 87 196617 Original Image;
#N vpatcher 10 59 221 244;
#P window setfont "Sans Serif" 9.;
#P newex 67 102 103 196617 jit.op @op < @val 13;
#P newex 50 50 27 196617 t l l;
#P newex 67 78 89 196617 jit.op @op absdiff;
#P inlet 50 30 15 0;
#P inlet 160 82 15 0;
#P outlet 67 124 15 0;
#P connect 2 0 4 0;
#P connect 4 1 3 0;
#P connect 3 0 5 0;
#P connect 5 0 0 0;
#P fasten 4 0 3 1 55 72 151 72;
#P connect 1 0 5 1;
#P pop;
#P newobj 184 157 101 196617 p FrameDifferencing;
#P number 275 137 35 9 0 0 0 3 0 0 0 221 221 221 222 222 222 0 0 0;
#P user jit.pwindow 505 433 242 182 0 1 0 0 1 0;
#P user jit.pwindow 261 433 242 182 0 1 0 0 1 0;
#P user jit.pwindow 16 432 242 182 0 1 0 0 1 0;
#P comment 231 121 126 196617 Threshold for Motion Mask;
#P comment 89 417 72 196617 Original Image;
#P connect 17 0 16 0;
#P fasten 17 0 6 0 56 152 189 152;
#P connect 17 0 3 0;
#P connect 15 0 2 0;
#P connect 16 0 15 0;
#P fasten 6 0 16 1 189 179 111 179;
#P fasten 6 0 4 0 189 211 511 211;
#P connect 9 0 17 0;
#P connect 11 0 10 0;
#P fasten 14 0 9 0 177 96 56 96;
#P fasten 13 0 9 0 134 96 56 96;
#P fasten 10 0 9 0 56 101 56 101;
#P connect 12 0 10 1;
#P connect 5 0 6 1;
#P window clipboard copycount 18;


October 11, 2006 | 9:38 am

could you pls explain whats the difference between cv.jit.ravg and
cv.jit.sttdv, and when to use each

2006/10/11, Wesley Smith :
>
> Try downloading the new cv.jit. I would use cv.jit.ravg.
>
> wes
>
> #P window setfont "Sans Serif" 9.;
> #P window linecount 1;
> #P newex 51 130 66 196617 jit.rgb2luma;
> #P newex 51 185 65 196617 jit.op @op *;
> #P newex 51 208 123 196617 cv.jit.ravg @alpha 0.001;
> #P message 172 70 32 196617 close;
> #P message 129 70 29 196617 open;
> #P flonum 85 50 35 9 0.5 0 1 3 0 0 0 221 221 221 222 222 222 0 0 0;
> #P toggle 51 50 15 0;
> #P newex 51 70 57 196617 qmetro 30;
> #P newex 51 108 98 196617 jit.qt.grab 160 120;
> #P comment 569 417 87 196617 Motion Mask;
> #P comment 329 418 87 196617 Original Image;
> #N vpatcher 10 59 221 244;
> #P window setfont "Sans Serif" 9.;
> #P newex 67 102 103 196617 jit.op @op < @val 13;
> #P newex 50 50 27 196617 t l l;
> #P newex 67 78 89 196617 jit.op @op absdiff;
> #P inlet 50 30 15 0;
> #P inlet 160 82 15 0;
> #P outlet 67 124 15 0;
> #P connect 2 0 4 0;
> #P connect 4 1 3 0;
> #P connect 3 0 5 0;
> #P connect 5 0 0 0;
> #P fasten 4 0 3 1 55 72 151 72;
> #P connect 1 0 5 1;
> #P pop;
> #P newobj 184 157 101 196617 p FrameDifferencing;
> #P number 275 137 35 9 0 0 0 3 0 0 0 221 221 221 222 222 222 0 0 0;
> #P user jit.pwindow 505 433 242 182 0 1 0 0 1 0;
> #P user jit.pwindow 261 433 242 182 0 1 0 0 1 0;
> #P user jit.pwindow 16 432 242 182 0 1 0 0 1 0;
> #P comment 231 121 126 196617 Threshold for Motion Mask;
> #P comment 89 417 72 196617 Original Image;
> #P connect 17 0 16 0;
> #P fasten 17 0 6 0 56 152 189 152;
> #P connect 17 0 3 0;
> #P connect 15 0 2 0;
> #P connect 16 0 15 0;
> #P fasten 6 0 16 1 189 179 111 179;
> #P fasten 6 0 4 0 189 211 511 211;
> #P connect 9 0 17 0;
> #P connect 11 0 10 0;
> #P fasten 14 0 9 0 177 96 56 96;
> #P fasten 13 0 9 0 134 96 56 96;
> #P fasten 10 0 9 0 56 101 56 101;
> #P connect 12 0 10 1;
> #P connect 5 0 6 1;
> #P window clipboard copycount 18;
>


October 11, 2006 | 10:03 am

http://en.wikipedia.org/wiki/Standard_deviation

http://en.wikipedia.org/wiki/Average

I believe the cv.jit.ravg object gives you an average image over time, thus
an average image/matrix over time t ( = +- the average pixel value over time
t for each pixel).

The standard deviation over time t gives you a value for how big the changes
are over a time t. So if values of the standard deviation get bigger this
means that the amount of change was bigger.

i’d use cv.jit.ravg for when motion/changes need to be smoothed out as in
Damon’s example. cv.jit.stddev should be used when u need a measure for the
amount of change.

…This is what i believe to be true…
don’t shoot me if a missed a spot :)

gr
d

On 10/11/06, yair reshef wrote:
>
> could you pls explain whats the difference between cv.jit.ravg and
> cv.jit.sttdv, and when to use each
>
> 2006/10/11, Wesley Smith < wesley.hoke@gmail.com>:
> >
> > Try downloading the new cv.jit. I would use cv.jit.ravg.
> >
> > wes
> >
> > #P window setfont "Sans Serif" 9.;
> > #P window linecount 1;
> > #P newex 51 130 66 196617 jit.rgb2luma;
> > #P newex 51 185 65 196617 jit.op @op *;
> > #P newex 51 208 123 196617 cv.jit.ravg @alpha 0.001;
> > #P message 172 70 32 196617 close;
> > #P message 129 70 29 196617 open;
> > #P flonum 85 50 35 9 0.5 0 1 3 0 0 0 221 221 221 222 222 222 0 0 0;
> > #P toggle 51 50 15 0;
> > #P newex 51 70 57 196617 qmetro 30;
> > #P newex 51 108 98 196617 jit.qt.grab 160 120;
> > #P comment 569 417 87 196617 Motion Mask;
> > #P comment 329 418 87 196617 Original Image;
> > #N vpatcher 10 59 221 244;
> > #P window setfont "Sans Serif" 9.;
> > #P newex 67 102 103 196617 jit.op @op < @val 13;
> > #P newex 50 50 27 196617 t l l;
> > #P newex 67 78 89 196617 jit.op @op absdiff;
> > #P inlet 50 30 15 0;
> > #P inlet 160 82 15 0;
> > #P outlet 67 124 15 0;
> > #P connect 2 0 4 0;
> > #P connect 4 1 3 0;
> > #P connect 3 0 5 0;
> > #P connect 5 0 0 0;
> > #P fasten 4 0 3 1 55 72 151 72;
> > #P connect 1 0 5 1;
> > #P pop;
> > #P newobj 184 157 101 196617 p FrameDifferencing;
> > #P number 275 137 35 9 0 0 0 3 0 0 0 221 221 221 222 222 222 0 0 0;
> > #P user jit.pwindow 505 433 242 182 0 1 0 0 1 0;
> > #P user jit.pwindow 261 433 242 182 0 1 0 0 1 0;
> > #P user jit.pwindow 16 432 242 182 0 1 0 0 1 0;
> > #P comment 231 121 126 196617 Threshold for Motion Mask;
> > #P comment 89 417 72 196617 Original Image;
> > #P connect 17 0 16 0;
> > #P fasten 17 0 6 0 56 152 189 152;
> > #P connect 17 0 3 0;
> > #P connect 15 0 2 0;
> > #P connect 16 0 15 0;
> > #P fasten 6 0 16 1 189 179 111 179;
> > #P fasten 6 0 4 0 189 211 511 211;
> > #P connect 9 0 17 0;
> > #P connect 11 0 10 0;
> > #P fasten 14 0 9 0 177 96 56 96;
> > #P fasten 13 0 9 0 134 96 56 96;
> > #P fasten 10 0 9 0 56 101 56 101;
> > #P connect 12 0 10 1;
> > #P connect 5 0 6 1;
> > #P window clipboard copycount 18;
> >
>
>
>
>
>


October 12, 2006 | 2:00 am

Quote: d_s wrote on Wed, 11 October 2006 00:57
—————————————————-

> My question is this: I want to very slowly adapt the background image that is being subtracted in this example, but no matter how high I set the inputs to the jit.slide object the subtracted image adapts in about 2 seconds. My goal is to have a situation where the background image adapts at about 0.05% per frame, or where it takes about 1 minute for a complete recycling of the bg image.

The reason probably has to do with quantization. jit.slide does its calculations on char data. Notice that the formula used is recursive; it uses the old value to compute the new. It works like this:

New average = old average + ((new data – old average)/slide)

Putting some arbitrary numbers in there:

New average = 100 + ((105 – 100)/20)
New average = 100 + (5/20)
New average = 100 + 0.25

Now, because these values are integers, 0.25 gets truncated to 0, so that the average remains at 100, when in fact it should now be 100.25. A small difference, but because the algorithm is recursive, they add up.

cv.jit.ravg essentially duplicates the functionality of jit.slide but carries out its calculations in floating point, regardless of the type of input. In other words, in the example above, like jit.slide it would ouput 100, but it would save 100.25 for the next turn. In practice that means that you can use much longer windows. You can also do the same thing with jit.slide by converting the data to float before processing.

As for standard deviation, think of the following situation: You’re looking at an outdoor parking lot where there is a single tree. Everything is completely still except for the tree’s leaves, which move slightly because of the wind. Ideally, if you wanted to do background subtraction on this scene, you would want to tell the algorithm to ignore the tree’s leaves. The standard deviation helps you do that by telling you, on average, how much the value of each pixel changes. For leaf pixels, that value would be high, for asphalt pixels, low. With this knowledge, you can threshold on a pixel-per-pixel basis.

I hope this wasn’t too confusing!


October 12, 2006 | 2:03 am

Quote: d_s wrote on Wed, 11 October 2006 00:57
—————————————————-

> My goal is to have a situation where the background image adapts at about 0.05% per frame, or where it takes about 1 minute for a complete recycling of the bg image.

I forgot: you can also try lowering the frame rate for the averaging stage. Simply using speedlim should do it.



d_s
October 15, 2006 | 12:42 am

To all,

Thank you so much! The ravg object was indeed just the thing I needed.

Best,
d_s


Viewing 7 posts - 1 through 7 (of 7 total)