reduce camera noise?


    Sep 11 2007 | 4:43 pm
    Hi all.
    Without resorting to jit.slide on the image itself, does anyone have a handy (and not too cpu intensive) means of removing noise from a firewire camera image? I am doing some background subtraction, and while i can clean up the noise around the person I am looking for, the camera itselfis kicking out some stray funky pixels.
    I vaguely remember a discussion on the list from a year or 2 ago about this, but I can't find it. Any suggestions?
    Thanks :)
    ~s

    • Sep 11 2007 | 7:27 pm
      On Sep 11, 2007, at 9:43 AM, Scott Fitzgerald wrote:
      > Without resorting to jit.slide on the image itself, does anyone have a > handy (and not too cpu intensive) means of removing noise from a > firewire camera image?
      The first things to consider before you get to processing your image are the standard things for reducing thermal noise:
      1. Use as much light as possible in the space you are filming (so that the ratio of light to thermal noise is as high as possible) 2. Use as large a CCD camera as possible 3. Try to keep the camera cool somehow (obviously not always possible or practical)
      Then, it would make sense to start processing. The following doesn't work for all situations, but one thing that I've used successfully recently is a threshold based slide implementation. Small amounts of difference between frames only affect the output a little bit, but larger changes are visibly immediately. I've cleaned up and uploaded a CPU and GPU based version below. It uses floating point values so that the "slide" is effective, which means that it is slow on the CPU, and for a decent GPU version, your card will need to support floating point textures.
      You can also use adaptive filtering spatially (perhaps combined with temporal analysis/processing). I've included a version which uses a kind of multipass adaptive median filter to smooth small change (this sort of thing can also be used for things like skin tone smoothing filters and such). The shader can easily be modified to be an adaptive mean filter if that works better for your material.
      More advanced denoising algorithms also incorporate optical flow in the analysis/processing. For this sort of thing, you can perhaps use Jean Marc Pelletier's cv.jit objects for. Might be tricky and CPU intensive, however.
      Hope this helps.
      -Joshua
    • Sep 11 2007 | 8:11 pm
      here is a gpu based background subtraction i use often. ill try to implement your recommendations. i use a slide based averaging technique for the background, having control over your lighting is a crucial point, more light, less noise. the slide routine i use accept a nice range of lighting conditions, i use the infra red spectrum, and the sun is a major source for ir. but ir is not good for everything.
      patch + a basic luma shader hacked from on of the others.
      On 9/11/07, Joshua Kit Clayton wrote: > > > On Sep 11, 2007, at 9:43 AM, Scott Fitzgerald wrote: > > > Without resorting to jit.slide on the image itself, does anyone have a > > handy (and not too cpu intensive) means of removing noise from a > > firewire camera image? > > The first things to consider before you get to processing your image > are the standard things for reducing thermal noise: > > 1. Use as much light as possible in the space you are filming (so > that the ratio of light to thermal noise is as high as possible) > 2. Use as large a CCD camera as possible > 3. Try to keep the camera cool somehow (obviously not always possible > or practical) > > Then, it would make sense to start processing. The following doesn't > work for all situations, but one thing that I've used successfully > recently is a threshold based slide implementation. Small amounts of > difference between frames only affect the output a little bit, but > larger changes are visibly immediately. I've cleaned up and uploaded > a CPU and GPU based version below. It uses floating point values so > that the "slide" is effective, which means that it is slow on the > CPU, and for a decent GPU version, your card will need to support > floating point textures. > > http://www.musorkvision.com/jkc/temp-denoise-example.zip > > You can also use adaptive filtering spatially (perhaps combined with > temporal analysis/processing). I've included a version which uses a > kind of multipass adaptive median filter to smooth small change (this > sort of thing can also be used for things like skin tone smoothing > filters and such). The shader can easily be modified to be an > adaptive mean filter if that works better for your material. > > More advanced denoising algorithms also incorporate optical flow in > the analysis/processing. For this sort of thing, you can perhaps use > Jean Marc Pelletier's cv.jit objects for. Might be tricky and CPU > intensive, however. > > Hope this helps. > > -Joshua >
    • Sep 11 2007 | 8:12 pm
      On 9/11/07, yair reshef wrote: > > here is a gpu based background subtraction i use often. ill try to > implement your recommendations. > i use a slide based averaging technique for the background, having control > over your lighting is a crucial point, more light, less noise. the slide > routine i use accept a nice range of lighting conditions, i use the infra > red spectrum, and the sun is a major source for ir. but ir is not good for > everything. > > patch + a basic luma shader hacked from on of the others. > > > > On 9/11/07, Joshua Kit Clayton wrote: > > > > > > On Sep 11, 2007, at 9:43 AM, Scott Fitzgerald wrote: > > > > > Without resorting to jit.slide on the image itself, does anyone have a > > > handy (and not too cpu intensive) means of removing noise from a > > > firewire camera image? > > > > The first things to consider before you get to processing your image > > are the standard things for reducing thermal noise: > > > > 1. Use as much light as possible in the space you are filming (so > > that the ratio of light to thermal noise is as high as possible) > > 2. Use as large a CCD camera as possible > > 3. Try to keep the camera cool somehow (obviously not always possible > > or practical) > > > > Then, it would make sense to start processing. The following doesn't > > work for all situations, but one thing that I've used successfully > > recently is a threshold based slide implementation. Small amounts of > > difference between frames only affect the output a little bit, but > > larger changes are visibly immediately. I've cleaned up and uploaded > > a CPU and GPU based version below. It uses floating point values so > > that the "slide" is effective, which means that it is slow on the > > CPU, and for a decent GPU version, your card will need to support > > floating point textures. > > > > http://www.musorkvision.com/jkc/temp-denoise-example.zip > > > > You can also use adaptive filtering spatially (perhaps combined with > > temporal analysis/processing). I've included a version which uses a > > kind of multipass adaptive median filter to smooth small change (this > > sort of thing can also be used for things like skin tone smoothing > > filters and such). The shader can easily be modified to be an > > adaptive mean filter if that works better for your material. > > > > More advanced denoising algorithms also incorporate optical flow in > > the analysis/processing. For this sort of thing, you can perhaps use > > Jean Marc Pelletier's cv.jit objects for. Might be tricky and CPU > > intensive, however. > > > > Hope this helps. > > > > -Joshua > > > >
    • Sep 11 2007 | 9:05 pm
      Wow, these are pretty swell, thanks Joshua.
      however, I'm running this on an Intel mac mini, so too much on the gpu isn't really going to fly. Right now I'm playing with the camera's settings, tryig to finesse it as much as possible, particularly since the lighting is fixed (and they didn't exactly listen to me).
      But perhaps the small slide would work well.
      I'm getting the refernce frame with cv.jit.ravg @alpha .1 and doing some dilation and erosion on the mask to clean up the sides a bit. Obviously works better in some situations than others.
      Thnaks again,
      ~s
      On 9/11/07, Joshua Kit Clayton wrote: > > On Sep 11, 2007, at 9:43 AM, Scott Fitzgerald wrote: > > > Without resorting to jit.slide on the image itself, does anyone have a > > handy (and not too cpu intensive) means of removing noise from a > > firewire camera image? > > The first things to consider before you get to processing your image > are the standard things for reducing thermal noise: > > 1. Use as much light as possible in the space you are filming (so > that the ratio of light to thermal noise is as high as possible) > 2. Use as large a CCD camera as possible > 3. Try to keep the camera cool somehow (obviously not always possible > or practical) > > Then, it would make sense to start processing. The following doesn't > work for all situations, but one thing that I've used successfully > recently is a threshold based slide implementation. Small amounts of > difference between frames only affect the output a little bit, but > larger changes are visibly immediately. I've cleaned up and uploaded > a CPU and GPU based version below. It uses floating point values so > that the "slide" is effective, which means that it is slow on the > CPU, and for a decent GPU version, your card will need to support > floating point textures. > > http://www.musorkvision.com/jkc/temp-denoise-example.zip > > You can also use adaptive filtering spatially (perhaps combined with > temporal analysis/processing). I've included a version which uses a > kind of multipass adaptive median filter to smooth small change (this > sort of thing can also be used for things like skin tone smoothing > filters and such). The shader can easily be modified to be an > adaptive mean filter if that works better for your material. > > More advanced denoising algorithms also incorporate optical flow in > the analysis/processing. For this sort of thing, you can perhaps use > Jean Marc Pelletier's cv.jit objects for. Might be tricky and CPU > intensive, however. > > Hope this helps. > > -Joshua >
    • Sep 11 2007 | 9:07 pm
      and yair,
      also nice. thanks, i'll see if I can't adapt any of the above solutions to this situation.
      ~s
      On 9/11/07, yair reshef wrote: > > > On 9/11/07, yair reshef wrote: > > here is a gpu based background subtraction i use often. ill try to > implement your recommendations. > > i use a slide based averaging technique for the background, having control > over your lighting is a crucial point, more light, less noise. the slide > routine i use accept a nice range of lighting conditions, i use the infra > red spectrum, and the sun is a major source for ir. but ir is not good for > everything. > > > > patch + a basic luma shader hacked from on of the others. > > > > > > > > > > > > On 9/11/07, Joshua Kit Clayton wrote: > > > > > > On Sep 11, 2007, at 9:43 AM, Scott Fitzgerald wrote: > > > > > > > Without resorting to jit.slide on the image itself, does anyone have a > > > > handy (and not too cpu intensive) means of removing noise from a > > > > firewire camera image? > > > > > > The first things to consider before you get to processing your image > > > are the standard things for reducing thermal noise: > > > > > > 1. Use as much light as possible in the space you are filming (so > > > that the ratio of light to thermal noise is as high as possible) > > > 2. Use as large a CCD camera as possible > > > 3. Try to keep the camera cool somehow (obviously not always possible > > > or practical) > > > > > > Then, it would make sense to start processing. The following doesn't > > > work for all situations, but one thing that I've used successfully > > > recently is a threshold based slide implementation. Small amounts of > > > difference between frames only affect the output a little bit, but > > > larger changes are visibly immediately. I've cleaned up and uploaded > > > a CPU and GPU based version below. It uses floating point values so > > > that the "slide" is effective, which means that it is slow on the > > > CPU, and for a decent GPU version, your card will need to support > > > floating point textures. > > > > > > > http://www.musorkvision.com/jkc/temp-denoise-example.zip > > > > > > You can also use adaptive filtering spatially (perhaps combined with > > > temporal analysis/processing). I've included a version which uses a > > > kind of multipass adaptive median filter to smooth small change (this > > > sort of thing can also be used for things like skin tone smoothing > > > filters and such). The shader can easily be modified to be an > > > adaptive mean filter if that works better for your material. > > > > > > More advanced denoising algorithms also incorporate optical flow in > > > the analysis/processing. For this sort of thing, you can perhaps use > > > Jean Marc Pelletier's cv.jit objects for. Might be tricky and CPU > > > intensive, however. > > > > > > Hope this helps. > > > > > > -Joshua > > > > > > > > > > > >
    • Sep 13 2007 | 10:58 am
      hi yair,
      would like to try your de-noising-approach, but unfortunately i can't see the attachment in the forum...
      could you try to upload it from within there (or whatever helps in cases like this)?
      many thanks,
      jan klug
    • Sep 13 2007 | 3:12 pm
      patch and shader follows
      //shader
      Luminance based keying Target luminance Luminance coefficients (RGBA)
      />
      On 9/13/07, Jan Klug wrote: > > > hi yair, > > would like to try your de-noising-approach, but unfortunately > i can't see the attachment in the forum... > > could you try to upload it from within there (or whatever helps in cases > like this)? > > many thanks, > > > jan klug >
    • Dec 19 2007 | 12:44 pm
      Hello, I would like to try Yair's patch. It must be obvious, but I don't understand where is the code or files for the shaders needed in his patch. co.luma.jxs co.difference.jxs co.luma.jxs cc.uyvy2rgba.jxs
      Sorry, I'm new to OpenGl ... can anyone help me? Thanks Felix
    • Dec 19 2007 | 10:09 pm
      On Dec 19, 2007 2:44 PM, Felix wrote:
      > > Hello, > I would like to try Yair's patch. > It must be obvious, but I don't understand where is the code or files for > the shaders needed in his patch. > co.luma.jxs > co.difference.jxs > co.luma.jxs > cc.uyvy2rgba.jxs > this are included with the jitter original distro. the first patch should be copy pasted into a new patch. it will automagicly appear. the shader should be copy pasted and put somewhere in the max folder. look in the max tutorial for more on this. have fun and proser.
      > > Sorry, I'm new to OpenGl ... > can anyone help me? > Thanks > Felix > -- > Felix Luque > http://www.othersounds.net > felix@othersounds.net >
    • Dec 20 2007 | 1:15 pm
      Thanks Yair, the problem is that i was searching for co.luma.jxs, and i don't find it in the max shaders folder. Is it the same shader as co.lumakey.jxs?. thanks F
    • Dec 20 2007 | 1:45 pm
      i can remember, probebly not as this is just a one liner i took from another shader.
      On Dec 20, 2007 5:15 AM, Felix wrote:
      > > Thanks Yair, > the problem is that i was searching for co.luma.jxs, and i don't find it > in the max shaders folder. Is it the same shader as co.lumakey.jxs?. > thanks > F > -- > Felix Luque > http://www.othersounds.net > felix@othersounds.net >
    • Dec 20 2007 | 3:05 pm
      I don't see the problem. The co.luma.jxs shader is attached earlier in this thread. Or am I missing something?
      Thijs
      On Dec 20, 2007 1:45 PM, yair reshef wrote:
      > i can remember, probebly not as this is just a one liner i took from > another shader. > > > On Dec 20, 2007 5:15 AM, Felix wrote: > > > > > Thanks Yair, > > the problem is that i was searching for co.luma.jxs, and i don't find it > > in the max shaders folder. Is it the same shader as co.lumakey.jxs?. > > thanks > > F > > -- > > > >
    • Dec 20 2007 | 3:44 pm
      Hi Thijs, I don’t see any attachment, but yes Yair wrote this as the shader: //shader Luminance based keying Target luminance Luminance coefficients (RGBA) /> I'm sorry, I don't know much about shader's syntax, but this doesn't look like a shader to me, it looks more like a note of how to make the shader, combining 3 basic shaders. Am I right?. … Sorry for my complete lack of knowledge in openGL … Felix
    • Dec 20 2007 | 4:31 pm
      ho, i posted this to the list, shader syntax dont play well with the forum.
      here is the shader , save as luma.jxs and put somewhere in max search path
      //shader
      Luminance based keying Target luminance Luminance coefficients (RGBA)
    • Dec 20 2007 | 4:33 pm
      thanks, ;-)
    • Feb 23 2013 | 12:52 am
      Oops my link above is dead. Haven't looked at this zip file in years, but I'm uploading it here for posterity.
    • Feb 23 2013 | 4:49 am
      cool thanks!
    • Feb 23 2013 | 10:51 am
      but also remember the first piece of advice: Crank up the light while you're recording. Nothing beats actual light.