reduce camera noise?


    Sep 11 2007 | 4:43 pm
    Hi all.
    Without resorting to jit.slide on the image itself, does anyone have a
    handy (and not too cpu intensive) means of removing noise from a
    firewire camera image? I am doing some background subtraction, and
    while i can clean up the noise around the person I am looking for, the
    camera itselfis kicking out some stray funky pixels.
    I vaguely remember a discussion on the list from a year or 2 ago about
    this, but I can't find it. Any suggestions?
    Thanks :)
    ~s

    • Sep 11 2007 | 7:27 pm
      On Sep 11, 2007, at 9:43 AM, Scott Fitzgerald wrote:
      > Without resorting to jit.slide on the image itself, does anyone have a
      > handy (and not too cpu intensive) means of removing noise from a
      > firewire camera image?
      The first things to consider before you get to processing your image
      are the standard things for reducing thermal noise:
      1. Use as much light as possible in the space you are filming (so
      that the ratio of light to thermal noise is as high as possible)
      2. Use as large a CCD camera as possible
      3. Try to keep the camera cool somehow (obviously not always possible
      or practical)
      Then, it would make sense to start processing. The following doesn't
      work for all situations, but one thing that I've used successfully
      recently is a threshold based slide implementation. Small amounts of
      difference between frames only affect the output a little bit, but
      larger changes are visibly immediately. I've cleaned up and uploaded
      a CPU and GPU based version below. It uses floating point values so
      that the "slide" is effective, which means that it is slow on the
      CPU, and for a decent GPU version, your card will need to support
      floating point textures.
      You can also use adaptive filtering spatially (perhaps combined with
      temporal analysis/processing). I've included a version which uses a
      kind of multipass adaptive median filter to smooth small change (this
      sort of thing can also be used for things like skin tone smoothing
      filters and such). The shader can easily be modified to be an
      adaptive mean filter if that works better for your material.
      More advanced denoising algorithms also incorporate optical flow in
      the analysis/processing. For this sort of thing, you can perhaps use
      Jean Marc Pelletier's cv.jit objects for. Might be tricky and CPU
      intensive, however.
      Hope this helps.
      -Joshua
    • Sep 11 2007 | 8:11 pm
      here is a gpu based background subtraction i use often. ill try to implement
      your recommendations.
      i use a slide based averaging technique for the background, having control
      over your lighting is a crucial point, more light, less noise. the slide
      routine i use accept a nice range of lighting conditions, i use the infra
      red spectrum, and the sun is a major source for ir. but ir is not good for
      everything.
      patch + a basic luma shader hacked from on of the others.
      On 9/11/07, Joshua Kit Clayton wrote:
      >
      >
      > On Sep 11, 2007, at 9:43 AM, Scott Fitzgerald wrote:
      >
      > > Without resorting to jit.slide on the image itself, does anyone have a
      > > handy (and not too cpu intensive) means of removing noise from a
      > > firewire camera image?
      >
      > The first things to consider before you get to processing your image
      > are the standard things for reducing thermal noise:
      >
      > 1. Use as much light as possible in the space you are filming (so
      > that the ratio of light to thermal noise is as high as possible)
      > 2. Use as large a CCD camera as possible
      > 3. Try to keep the camera cool somehow (obviously not always possible
      > or practical)
      >
      > Then, it would make sense to start processing. The following doesn't
      > work for all situations, but one thing that I've used successfully
      > recently is a threshold based slide implementation. Small amounts of
      > difference between frames only affect the output a little bit, but
      > larger changes are visibly immediately. I've cleaned up and uploaded
      > a CPU and GPU based version below. It uses floating point values so
      > that the "slide" is effective, which means that it is slow on the
      > CPU, and for a decent GPU version, your card will need to support
      > floating point textures.
      >
      > http://www.musorkvision.com/jkc/temp-denoise-example.zip
      >
      > You can also use adaptive filtering spatially (perhaps combined with
      > temporal analysis/processing). I've included a version which uses a
      > kind of multipass adaptive median filter to smooth small change (this
      > sort of thing can also be used for things like skin tone smoothing
      > filters and such). The shader can easily be modified to be an
      > adaptive mean filter if that works better for your material.
      >
      > More advanced denoising algorithms also incorporate optical flow in
      > the analysis/processing. For this sort of thing, you can perhaps use
      > Jean Marc Pelletier's cv.jit objects for. Might be tricky and CPU
      > intensive, however.
      >
      > Hope this helps.
      >
      > -Joshua
      >
    • Sep 11 2007 | 8:12 pm
      On 9/11/07, yair reshef wrote:
      >
      > here is a gpu based background subtraction i use often. ill try to
      > implement your recommendations.
      > i use a slide based averaging technique for the background, having control
      > over your lighting is a crucial point, more light, less noise. the slide
      > routine i use accept a nice range of lighting conditions, i use the infra
      > red spectrum, and the sun is a major source for ir. but ir is not good for
      > everything.
      >
      > patch + a basic luma shader hacked from on of the others.
      >
      >
      >
      > On 9/11/07, Joshua Kit Clayton wrote:
      > >
      > >
      > > On Sep 11, 2007, at 9:43 AM, Scott Fitzgerald wrote:
      > >
      > > > Without resorting to jit.slide on the image itself, does anyone have a
      > > > handy (and not too cpu intensive) means of removing noise from a
      > > > firewire camera image?
      > >
      > > The first things to consider before you get to processing your image
      > > are the standard things for reducing thermal noise:
      > >
      > > 1. Use as much light as possible in the space you are filming (so
      > > that the ratio of light to thermal noise is as high as possible)
      > > 2. Use as large a CCD camera as possible
      > > 3. Try to keep the camera cool somehow (obviously not always possible
      > > or practical)
      > >
      > > Then, it would make sense to start processing. The following doesn't
      > > work for all situations, but one thing that I've used successfully
      > > recently is a threshold based slide implementation. Small amounts of
      > > difference between frames only affect the output a little bit, but
      > > larger changes are visibly immediately. I've cleaned up and uploaded
      > > a CPU and GPU based version below. It uses floating point values so
      > > that the "slide" is effective, which means that it is slow on the
      > > CPU, and for a decent GPU version, your card will need to support
      > > floating point textures.
      > >
      > > http://www.musorkvision.com/jkc/temp-denoise-example.zip
      > >
      > > You can also use adaptive filtering spatially (perhaps combined with
      > > temporal analysis/processing). I've included a version which uses a
      > > kind of multipass adaptive median filter to smooth small change (this
      > > sort of thing can also be used for things like skin tone smoothing
      > > filters and such). The shader can easily be modified to be an
      > > adaptive mean filter if that works better for your material.
      > >
      > > More advanced denoising algorithms also incorporate optical flow in
      > > the analysis/processing. For this sort of thing, you can perhaps use
      > > Jean Marc Pelletier's cv.jit objects for. Might be tricky and CPU
      > > intensive, however.
      > >
      > > Hope this helps.
      > >
      > > -Joshua
      > >
      >
      >
    • Sep 11 2007 | 9:05 pm
      Wow, these are pretty swell, thanks Joshua.
      however, I'm running this on an Intel mac mini, so too much on the gpu
      isn't really going to fly. Right now I'm playing with the camera's
      settings, tryig to finesse it as much as possible, particularly since
      the lighting is fixed (and they didn't exactly listen to me).
      But perhaps the small slide would work well.
      I'm getting the refernce frame with cv.jit.ravg @alpha .1 and doing
      some dilation and erosion on the mask to clean up the sides a bit.
      Obviously works better in some situations than others.
      Thnaks again,
      ~s
      On 9/11/07, Joshua Kit Clayton wrote:
      >
      > On Sep 11, 2007, at 9:43 AM, Scott Fitzgerald wrote:
      >
      > > Without resorting to jit.slide on the image itself, does anyone have a
      > > handy (and not too cpu intensive) means of removing noise from a
      > > firewire camera image?
      >
      > The first things to consider before you get to processing your image
      > are the standard things for reducing thermal noise:
      >
      > 1. Use as much light as possible in the space you are filming (so
      > that the ratio of light to thermal noise is as high as possible)
      > 2. Use as large a CCD camera as possible
      > 3. Try to keep the camera cool somehow (obviously not always possible
      > or practical)
      >
      > Then, it would make sense to start processing. The following doesn't
      > work for all situations, but one thing that I've used successfully
      > recently is a threshold based slide implementation. Small amounts of
      > difference between frames only affect the output a little bit, but
      > larger changes are visibly immediately. I've cleaned up and uploaded
      > a CPU and GPU based version below. It uses floating point values so
      > that the "slide" is effective, which means that it is slow on the
      > CPU, and for a decent GPU version, your card will need to support
      > floating point textures.
      >
      > http://www.musorkvision.com/jkc/temp-denoise-example.zip
      >
      > You can also use adaptive filtering spatially (perhaps combined with
      > temporal analysis/processing). I've included a version which uses a
      > kind of multipass adaptive median filter to smooth small change (this
      > sort of thing can also be used for things like skin tone smoothing
      > filters and such). The shader can easily be modified to be an
      > adaptive mean filter if that works better for your material.
      >
      > More advanced denoising algorithms also incorporate optical flow in
      > the analysis/processing. For this sort of thing, you can perhaps use
      > Jean Marc Pelletier's cv.jit objects for. Might be tricky and CPU
      > intensive, however.
      >
      > Hope this helps.
      >
      > -Joshua
      >
    • Sep 11 2007 | 9:07 pm
      and yair,
      also nice. thanks, i'll see if I can't adapt any of the above
      solutions to this situation.
      ~s
      On 9/11/07, yair reshef wrote:
      >
      >
      > On 9/11/07, yair reshef wrote:
      > > here is a gpu based background subtraction i use often. ill try to
      > implement your recommendations.
      > > i use a slide based averaging technique for the background, having control
      > over your lighting is a crucial point, more light, less noise. the slide
      > routine i use accept a nice range of lighting conditions, i use the infra
      > red spectrum, and the sun is a major source for ir. but ir is not good for
      > everything.
      > >
      > > patch + a basic luma shader hacked from on of the others.
      > >
      > >
      > >
      > >
      > >
      > > On 9/11/07, Joshua Kit Clayton wrote:
      > > >
      > > > On Sep 11, 2007, at 9:43 AM, Scott Fitzgerald wrote:
      > > >
      > > > > Without resorting to jit.slide on the image itself, does anyone have a
      > > > > handy (and not too cpu intensive) means of removing noise from a
      > > > > firewire camera image?
      > > >
      > > > The first things to consider before you get to processing your image
      > > > are the standard things for reducing thermal noise:
      > > >
      > > > 1. Use as much light as possible in the space you are filming (so
      > > > that the ratio of light to thermal noise is as high as possible)
      > > > 2. Use as large a CCD camera as possible
      > > > 3. Try to keep the camera cool somehow (obviously not always possible
      > > > or practical)
      > > >
      > > > Then, it would make sense to start processing. The following doesn't
      > > > work for all situations, but one thing that I've used successfully
      > > > recently is a threshold based slide implementation. Small amounts of
      > > > difference between frames only affect the output a little bit, but
      > > > larger changes are visibly immediately. I've cleaned up and uploaded
      > > > a CPU and GPU based version below. It uses floating point values so
      > > > that the "slide" is effective, which means that it is slow on the
      > > > CPU, and for a decent GPU version, your card will need to support
      > > > floating point textures.
      > > >
      > > >
      > http://www.musorkvision.com/jkc/temp-denoise-example.zip
      > > >
      > > > You can also use adaptive filtering spatially (perhaps combined with
      > > > temporal analysis/processing). I've included a version which uses a
      > > > kind of multipass adaptive median filter to smooth small change (this
      > > > sort of thing can also be used for things like skin tone smoothing
      > > > filters and such). The shader can easily be modified to be an
      > > > adaptive mean filter if that works better for your material.
      > > >
      > > > More advanced denoising algorithms also incorporate optical flow in
      > > > the analysis/processing. For this sort of thing, you can perhaps use
      > > > Jean Marc Pelletier's cv.jit objects for. Might be tricky and CPU
      > > > intensive, however.
      > > >
      > > > Hope this helps.
      > > >
      > > > -Joshua
      > > >
      > >
      > >
      >
      >
      >
      >
      >
    • Sep 13 2007 | 10:58 am
      hi yair,
      would like to try your de-noising-approach, but unfortunately
      i can't see the attachment in the forum...
      could you try to upload it from within there (or whatever helps in cases like this)?
      many thanks,
      jan klug
    • Sep 13 2007 | 3:12 pm
      patch and shader follows
      //shaderTarget luminance
      Luminance based keying
      Luminance coefficients (RGBA)
      />
      On 9/13/07, Jan Klug wrote:
      >
      >
      > hi yair,
      >
      > would like to try your de-noising-approach, but unfortunately
      > i can't see the attachment in the forum...
      >
      > could you try to upload it from within there (or whatever helps in cases
      > like this)?
      >
      > many thanks,
      >
      >
      > jan klug
      >
    • Dec 19 2007 | 12:44 pm
      Hello,
      I would like to try Yair's patch.
      It must be obvious, but I don't understand where is the code or files for the shaders needed in his patch.
      co.luma.jxs
      co.difference.jxs
      co.luma.jxs
      cc.uyvy2rgba.jxs
      Sorry, I'm new to OpenGl ...
      can anyone help me?
      Thanks
      Felix
    • Dec 19 2007 | 10:09 pm
      On Dec 19, 2007 2:44 PM, Felix wrote:
      >
      > Hello,
      > I would like to try Yair's patch.
      > It must be obvious, but I don't understand where is the code or files for
      > the shaders needed in his patch.
      > co.luma.jxs
      > co.difference.jxs
      > co.luma.jxs
      > cc.uyvy2rgba.jxs
      >
      this are included with the jitter original distro.
      the first patch should be copy pasted into a new patch. it will automagicly
      appear.
      the shader should be copy pasted and put somewhere in the max folder. look
      in the max tutorial for more on this.
      have fun and proser.
      >
      > Sorry, I'm new to OpenGl ...
      > can anyone help me?
      > Thanks
      > Felix
      > --
      > Felix Luque
      > http://www.othersounds.net
      > felix@othersounds.net
      >
    • Dec 20 2007 | 1:15 pm
      Thanks Yair,
      the problem is that i was searching for co.luma.jxs, and i don't find it in the max shaders folder. Is it the same shader as co.lumakey.jxs?.
      thanks
      F
    • Dec 20 2007 | 1:45 pm
      i can remember, probebly not as this is just a one liner i took from another
      shader.
      On Dec 20, 2007 5:15 AM, Felix wrote:
      >
      > Thanks Yair,
      > the problem is that i was searching for co.luma.jxs, and i don't find it
      > in the max shaders folder. Is it the same shader as co.lumakey.jxs?.
      > thanks
      > F
      > --
      > Felix Luque
      > http://www.othersounds.net
      > felix@othersounds.net
      >
    • Dec 20 2007 | 3:05 pm
      I don't see the problem. The co.luma.jxs shader is attached earlier in this
      thread. Or am I missing something?
      Thijs
      On Dec 20, 2007 1:45 PM, yair reshef wrote:
      > i can remember, probebly not as this is just a one liner i took from
      > another shader.
      >
      >
      > On Dec 20, 2007 5:15 AM, Felix wrote:
      >
      > >
      > > Thanks Yair,
      > > the problem is that i was searching for co.luma.jxs, and i don't find it
      > > in the max shaders folder. Is it the same shader as co.lumakey.jxs?.
      > > thanks
      > > F
      > > --
      > >
      > >
    • Dec 20 2007 | 3:44 pm
      Hi Thijs,
      I don’t see any attachment, but yes Yair wrote this as the shader:
      //shader
      Luminance based keying
      Target luminance
      Luminance coefficients (RGBA)
      />
      I'm sorry, I don't know much about shader's syntax, but this doesn't look like a shader to me, it looks more like a note of how to make the shader, combining 3 basic shaders. Am I right?. …
      Sorry for my complete lack of knowledge in openGL …
      Felix
    • Dec 20 2007 | 4:31 pm
      ho, i posted this to the list, shader syntax dont play well with the forum.
      here is the shader , save as luma.jxs and put somewhere in max search path
      //shaderTarget luminance
      Luminance based keying
      Luminance coefficients (RGBA)
    • Dec 20 2007 | 4:33 pm
      thanks, ;-)
    • Feb 23 2013 | 12:52 am
      Oops my link above is dead. Haven't looked at this zip file in years, but I'm uploading it here for posterity.
    • Feb 23 2013 | 4:49 am
      cool thanks!
    • Feb 23 2013 | 10:51 am
      but also remember the first piece of advice: Crank up the light while you're recording. Nothing beats actual light.