Motion Detection Questions

    Feb 16 2006 | 9:02 pm
    I have gone through the tutorials for jitter and I still feel very new at this whole scene. I am working on an undergraduate research project and I am trying to take live video input, interpret the data through Jitter to get a mean pixel movement value for both the X and Y coordinates in the matrix and then take that list information and input into the Teleo externals for Jitter to move a motor. *phew* So, in effect, someone moving from left to right across the matrix would cause the motor to move 'x' direction by the output created by the matrix. I'm clear up until the point when I need to use the information generated in a matrix for another external.
    Questions: Should I focus on the mean average threshold values of color and/or light? What is the best way to use the output list generated by a matrix? Am I going about this the wrong way? Would someone smarter than me be willing to create a small basic sample of what I would need to get started on this? (jit.dx.grab or jit.qt.grab are both working now)

    • Feb 16 2006 | 9:34 pm
    • Feb 16 2006 | 9:41 pm
      take a look at the cv.jit objects, particularly the optical flow algorithms (cv.jit.lkflow, cv.jit.hsflow). They output several planes that correspond to a given pixel's x/y direction (right left, up down). These might be summed (cv.jit.mass?) to give a general sense of movement direction in an simple visual scene. These won't "track" you per se, but the will give you a sense of right left, up down in a visual scene.
    • Feb 16 2006 | 10:23 pm
      I've tried cv.jit Optical Flow and I'm happy with the results but my camera is giving me hella problems with pixel noise. It's interpreting background visual noise as movement. Is there any way to set a detection threshold? Or is there a way to dampen the noise created in a matrix?
    • Feb 16 2006 | 11:18 pm
      What are you feeding into the optical flow algorithms, that is, how are you treating your video? Are you doing background subtraction or any other segmentation, smoothing, etc?
    • Feb 17 2006 | 12:57 am
      No, I'm not, it's a direct feed from jit.dx.grab into the cv.jit optical flow. Should I smooth it out or do anything else to it, and if so, how? I'm a bit concerned about keeping the number of constantly running processes down as I have to put it onto a computer with a mid-range processor and RAM for the installation, but I'm more worried about actually getting this running.
    • Feb 17 2006 | 1:23 am
      what do you mean by background visual "noise"?
      is this noise due to movement in the background (i.e. people walking ... branches moving) or camera fluctuations (fluctuating light sources causing auto white balance or auto focus problems) or is it simply a low resolution camera giving you a generally nasty image (possibly corrected by increasing the light on the subject)?
      If it is a camera resolution problem, you might try smoothing the grayscale image using jit.fastblur or a median filter (I have one I can give you and I think Wesley Smith also has one, but I don't know if either of us have a windows version compiled yet ... Wes?)
      If it is a camera fluctuation problem, try to turn off the "automatic" features of the camera (auto white balance, focus and the like).
      If it is a moving background problem (people or objects are moving in the background that you are not interested in analyzing) the problem becomes a little more complex. An algorithm that chooses the "correct" person to monitor is not a simple problem. Eliminating "distracting" motions would work here (choosing a static background).
      Does any of that apply to your situation?
      Cheers, Christopher
    • Feb 17 2006 | 3:42 pm
      It's not movement as visual noise. The camera resolution seems to be fine, the only problem is that I can't use the unibrain drivers because it makes Jitter crash hard - and it's hard to set automatic focus and the like with the windows default drivers. I'll try the grayscale and median options you suggested, thanks.
    • Feb 17 2006 | 7:15 pm
      just a clarification ...
      I have a optimized spatial median filter (3x3 kernel), not a temporal median filter (like Wesley's). While both might be useful to you, the spatial median filter is good at smoothing the incoming image while preserving finer grain detail. It is especially good at removing salt-pepper noise (see median.html for more info). It may help get rid of some incoming visual noise.
      Cheers, Christopher
    • Feb 17 2006 | 7:23 pm
      Actually, the median object I just posted does both spatial and temporal median filters depending on whether the matrix is 2D or 3D. if you create an object like this: "xray.jit.median @size 3" and you send it a 2D matrix, you will get a 3x3 kernel. I coded it pretty fast, so I don't know how optimized it is. In any case, the source code it in the distro so you can see for yourself.
      best, wes
    • Feb 17 2006 | 7:33 pm
      that's a cool feature.
      thanks for the clarification.
      Cheers, Christopher
    • Feb 18 2006 | 9:23 am
      Hi Christopher
      is there a link where i could find your external? (I tried to search max objects database, but with no results)
      Thanks for any info! Marco
    • Feb 18 2006 | 9:35 am
      search "xray.jit" in google and you will get it
    • Feb 28 2006 | 2:23 am
      Okay one more question, sort of related.
      Essentially what I'd like to do is get a sum of the values of pixel movement within a certain threshold in the x and y coordinates of a matrix and plug it into another external (possible using xray.jit.mean?) Could someone help me out with a sample patch illustrating how this could be done? I can't seem to find any relevant information in the tutorials for jitter.
      process jit.dx.grab -> get sum of movement in matrix -> plug values into another external and have it grab only a certain threshold (whatever would correlate with physical movement across a space consistently, like a person walking from a to b)
      Thanks for all the advice already.
    • Feb 28 2006 | 3:19 am
      > process jit.dx.grab -> get sum of movement in matrix ->
      i've read through the thread on the forum, and I'm not sure what type of motion you are trying to translate into motor movement. Is it a "performer" against a static background? Is it people in the zocalo (which i reference as an extreme example of people milling about)? Is it objects in the camera's view, where different objects need to be distinguished? Is your analysis over a certain amount of time, or within a single frame? For a single body, I've had great success with the centroids object in the cv.jit collection. you can use jit scissors to break an image into rows/columns grid and track coordinates in each piece. jit.op @op absdiff is your friend. as is jit.op @op >
      > plug values into another external and have it grab only a certain > threshold (whatever would correlate with physical movement across a > space consistently, like a person walking from a to b)
      when you speak of threshold, I'm imagining a "switch" but it seems that what you want to do is track coordinates....
      with some clarity, I can possibly be of help!
    • Feb 28 2006 | 9:40 pm
      I want to translate recorded movement from a fixed point. So, I set it up in a room and it detects the movement of people walking around against a static background. If I could record [i]both[/i] horizontal and vertical (x,y respectively) values it would be nice but it's not necessary. Nothing needs to be distinguished and the video isn't really important as much as the data that is interpreted from it.
      After the data is processed into usable numbers or the like I want to then feed it to another external from the matrix window itself. The less complex the better, just so long as it has some sort of solid interger value range so I can set a threshold for the external. So for movement that creates numbers in that range it would trigger the motor using Teleo's external for x seconds. (i'm using teleo's 2x2A motor controller module)
      I can't put the html because the message board software truncates the address to fit in the window (nice feature!) but if you google "teleo and max/msp" you'll find some relevant information.
      The reason I'd like to filter all of the numbers out is because I don't want the motor to be going all of the time at every little movement or camera salt-pepper event.
    • Feb 28 2006 | 10:23 pm
      tap tools has a nice smoothing external tap.smooth.
      there are others around as well.
    • Jun 12 2006 | 5:19 am
      hi, i have a similar problem and i am working it out using difference matte, the principle is like this: if you have an image of your background and you have an image of your subject in front of your background you can subtract the background from the background and subject you get a rough matte but the camera has to be very still and the lights shudn't fluctuate.
      so i made a jitter patch that takes video input and if there is no one in front of the camera i manually capture the input into a matrix averaging the pixels first (using cv.jit.mean) to eliminate noise and then i subtract this image from the live input when there is someone thus isolating the person. i would like my patch to determine automatically if there is someone in front of the camera by analyzing the movement and if there is no one capture a new background image to correct possible light changes. this is my first jitter patch and is for a school project where i have to use motion as a control signal, so something has to happen if someone moves, the professor showed us a director patch that takes video input and draws a square in the places that pixels change and then adds all the areas of the squares to get an "amount of movement", but i would like to go a bit further, work with jitter and to be able to determine not just the movement but the position of the movement. can you fellows give us some pointers on motion analysis? or how to translate this square principle into jitter?
      p.d. cv.jit.label works great for eliminating noise on a binary image and just keeping the main "blob".
    • Nov 07 2006 | 10:28 pm
      I'm also working on a similar patch and I'm not too sure how to capture movement of a hand across the camera to control my video output. Aside from that, I can't seem to find anywhere to download winvdig because the page is done. Any help would be appreciated.
    • Nov 07 2006 | 10:47 pm
      I uploaded windvdig to Regarding the tracking Problem have a look at the cv.jit. libary at
      Am 07.11.2006 um 23:28 schrieb alan:
      > > I'm also working on a similar patch and I'm not too sure how to > capture movement of a hand across the camera to control my video > output. Aside from that, I can't seem to find anywhere to download > winvdig because the page is done. Any help would be appreciated. > >