Motion Detection Questions
I have gone through the tutorials for jitter and I still feel very new at this whole scene. I am working on an undergraduate research project and I am trying to take live video input, interpret the data through Jitter to get a mean pixel movement value for both the X and Y coordinates in the matrix and then take that list information and input into the Teleo externals for Jitter to move a motor. *phew* So, in effect, someone moving from left to right across the matrix would cause the motor to move ‘x’ direction by the output created by the matrix. I’m clear up until the point when I need to use the information generated in a matrix for another external.
Questions: Should I focus on the mean average threshold values of color and/or light? What is the best way to use the output list generated by a matrix? Am I going about this the wrong way? Would someone smarter than me be willing to create a small basic sample of what I would need to get started on this? (jit.dx.grab or jit.qt.grab are both working now)
take a look at the cv.jit objects, particularly the optical flow
algorithms (cv.jit.lkflow, cv.jit.hsflow). They output several
planes that correspond to a given pixel’s x/y direction (right left,
up down). These might be summed (cv.jit.mass?) to give a general
sense of movement direction in an simple visual scene. These won’t
"track" you per se, but the will give you a sense of right left, up
down in a visual scene.
http://www.iamas.ac.jp/~jovan02/cv/ for more …
I’ve tried cv.jit Optical Flow and I’m happy with the results but my camera is giving me hella problems with pixel noise. It’s interpreting background visual noise as movement. Is there any way to set a detection threshold? Or is there a way to dampen the noise created in a matrix?
What are you feeding into the optical flow algorithms, that is, how
are you treating your video? Are you doing background subtraction or
any other segmentation, smoothing, etc?
No, I’m not, it’s a direct feed from jit.dx.grab into the cv.jit optical flow. Should I smooth it out or do anything else to it, and if so, how? I’m a bit concerned about keeping the number of constantly running processes down as I have to put it onto a computer with a mid-range processor and RAM for the installation, but I’m more worried about actually getting this running.
what do you mean by background visual "noise"?
is this noise due to movement in the background (i.e. people
walking … branches moving) or camera fluctuations (fluctuating
light sources causing auto white balance or auto focus problems) or
is it simply a low resolution camera giving you a generally nasty
image (possibly corrected by increasing the light on the subject)?
If it is a camera resolution problem, you might try smoothing the
grayscale image using jit.fastblur or a median filter (I have one I
can give you and I think Wesley Smith also has one, but I don’t know
if either of us have a windows version compiled yet … Wes?)
If it is a camera fluctuation problem, try to turn off the
"automatic" features of the camera (auto white balance, focus and the
If it is a moving background problem (people or objects are moving in
the background that you are not interested in analyzing) the problem
becomes a little more complex. An algorithm that chooses the
"correct" person to monitor is not a simple problem. Eliminating
"distracting" motions would work here (choosing a static background).
Does any of that apply to your situation?
It’s not movement as visual noise. The camera resolution seems to be fine, the only problem is that I can’t use the unibrain drivers because it makes Jitter crash hard – and it’s hard to set automatic focus and the like with the windows default drivers. I’ll try the grayscale and median options you suggested, thanks.
just a clarification …
I have a optimized spatial median filter (3×3 kernel), not a temporal
median filter (like Wesley’s). While both might be useful to you,
the spatial median filter is good at smoothing the incoming image
while preserving finer grain detail. It is especially good at
removing salt-pepper noise (see http://www.cee.hw.ac.uk/hipr/html/
median.html for more info). It may help get rid of some incoming
Actually, the median object I just posted does both spatial and
temporal median filters depending on whether the matrix is 2D or 3D.
if you create an object like this: "xray.jit.median @size 3" and you
send it a 2D matrix, you will get a 3×3 kernel. I coded it pretty
fast, so I don’t know how optimized it is. In any case, the source
code it in the distro so you can see for yourself.
that’s a cool feature.
thanks for the clarification.
is there a link where i could find your external? (I tried to search max
objects database, but with no results)
Thanks for any info!
search "xray.jit" in google and you will get it
Okay one more question, sort of related.
Essentially what I’d like to do is get a sum of the values of pixel movement within a certain threshold in the x and y coordinates of a matrix and plug it into another external (possible using xray.jit.mean?) Could someone help me out with a sample patch illustrating how this could be done? I can’t seem to find any relevant information in the tutorials for jitter.
process jit.dx.grab -> get sum of movement in matrix -> plug values into another external and have it grab only a certain threshold (whatever would correlate with physical movement across a space consistently, like a person walking from a to b)
Thanks for all the advice already.
> process jit.dx.grab -> get sum of movement in matrix ->
i’ve read through the thread on the forum, and I’m not sure what type
of motion you are trying to translate into motor movement. Is it a
"performer" against a static background? Is it people in the zocalo
(which i reference as an extreme example of people milling about)? Is
it objects in the camera’s view, where different objects need to be
Is your analysis over a certain amount of time, or within a single
For a single body, I’ve had great success with the centroids object in
the cv.jit collection.
you can use jit scissors to break an image into rows/columns grid and
track coordinates in each piece.
jit.op @op absdiff
is your friend.
as is jit.op @op >
> plug values into another external and have it grab only a certain
> threshold (whatever would correlate with physical movement across a
> space consistently, like a person walking from a to b)
when you speak of threshold, I’m imagining a "switch" but it seems that
what you want to do is track coordinates….
with some clarity, I can possibly be of help!
I want to translate recorded movement from a fixed point. So, I set it up in a room and it detects the movement of people walking around against a static background. If I could record [i]both[/i] horizontal and vertical (x,y respectively) values it would be nice but it’s not necessary. Nothing needs to be distinguished and the video isn’t really important as much as the data that is interpreted from it.
After the data is processed into usable numbers or the like I want to then feed it to another external from the matrix window itself. The less complex the better, just so long as it has some sort of solid interger value range so I can set a threshold for the external. So for movement that creates numbers in that range it would trigger the motor using Teleo’s external for x seconds. (i’m using teleo’s 2x2A motor controller module)
I can’t put the html because the message board software truncates the address to fit in the window (nice feature!) but if you google "teleo and max/msp" you’ll find some relevant information.
The reason I’d like to filter all of the numbers out is because I don’t want the motor to be going all of the time at every little movement or camera salt-pepper event.
tap tools has a nice smoothing external tap.smooth.
there are others around as well.
hi, i have a similar problem and i am working it out using difference matte, the principle is like this: if you have an image of your background and you have an image of your subject in front of your background you can subtract the background from the background and subject you get a rough matte but the camera has to be very still and the lights shudn’t fluctuate.
so i made a jitter patch that takes video input and if there is no one in front of the camera i manually capture the input into a matrix averaging the pixels first (using cv.jit.mean) to eliminate noise and then i subtract this image from the live input when there is someone thus isolating the person. i would like my patch to determine automatically if there is someone in front of the camera by analyzing the movement and if there is no one capture a new background image to correct possible light changes. this is my first jitter patch and is for a school project where i have to use motion as a control signal, so something has to happen if someone moves, the professor showed us a director patch that takes video input and draws a square in the places that pixels change and then adds all the areas of the squares to get an "amount of movement", but i would like to go a bit further, work with jitter and to be able to determine not just the movement but the position of the movement. can you fellows give us some pointers on motion analysis? or how to translate this square principle into jitter?
p.d. cv.jit.label works great for eliminating noise on a binary image and just keeping the main "blob".
I’m also working on a similar patch and I’m not too sure how to capture movement of a hand across the camera to control my video output. Aside from that, I can’t seem to find anywhere to download winvdig because the page is done. Any help would be appreciated.
I uploaded windvdig to http://grieffenhagen.de/stuff/WinVDIG_101.exe
Regarding the tracking Problem have a look at the cv.jit. libary at
Am 07.11.2006 um 23:28 schrieb alan:
> I’m also working on a similar patch and I’m not too sure how to
> capture movement of a hand across the camera to control my video
> output. Aside from that, I can’t seem to find anywhere to download
> winvdig because the page is done. Any help would be appreciated.