reduce camera noise?

On Sep 11, 2007, at 9:43 AM, Scott Fitzgerald wrote:

> Without resorting to jit.slide on the image itself, does anyone have a
> handy (and not too cpu intensive) means of removing noise from a
> firewire camera image?

The first things to consider before you get to processing your image
are the standard things for reducing thermal noise:

1. Use as much light as possible in the space you are filming (so
that the ratio of light to thermal noise is as high as possible)
2. Use as large a CCD camera as possible
3. Try to keep the camera cool somehow (obviously not always possible
or practical)

Then, it would make sense to start processing. The following doesn't
work for all situations, but one thing that I've used successfully
recently is a threshold based slide implementation. Small amounts of
difference between frames only affect the output a little bit, but
larger changes are visibly immediately. I've cleaned up and uploaded
a CPU and GPU based version below. It uses floating point values so
that the "slide" is effective, which means that it is slow on the
CPU, and for a decent GPU version, your card will need to support
floating point textures.

http://www.musorkvision.com/jkc/temp-denoise-example.zip

You can also use adaptive filtering spatially (perhaps combined with
temporal analysis/processing). I've included a version which uses a
kind of multipass adaptive median filter to smooth small change (this
sort of thing can also be used for things like skin tone smoothing
filters and such). The shader can easily be modified to be an
adaptive mean filter if that works better for your material.

More advanced denoising algorithms also incorporate optical flow in
the analysis/processing. For this sort of thing, you can perhaps use
Jean Marc Pelletier's cv.jit objects for. Might be tricky and CPU
intensive, however.

Hope this helps.

-Joshua

here is a gpu based background subtraction i use often. ill try to implement
your recommendations.
i use a slide based averaging technique for the background, having control
over your lighting is a crucial point, more light, less noise. the slide
routine i use accept a nice range of lighting conditions, i use the infra
red spectrum, and the sun is a major source for ir. but ir is not good for
everything.

patch + a basic luma shader hacked from on of the others.

On 9/11/07, Joshua Kit Clayton wrote:
>
>
> On Sep 11, 2007, at 9:43 AM, Scott Fitzgerald wrote:
>
> > Without resorting to jit.slide on the image itself, does anyone have a
> > handy (and not too cpu intensive) means of removing noise from a
> > firewire camera image?
>
> The first things to consider before you get to processing your image
> are the standard things for reducing thermal noise:
>
> 1. Use as much light as possible in the space you are filming (so
> that the ratio of light to thermal noise is as high as possible)
> 2. Use as large a CCD camera as possible
> 3. Try to keep the camera cool somehow (obviously not always possible
> or practical)
>
> Then, it would make sense to start processing. The following doesn't
> work for all situations, but one thing that I've used successfully
> recently is a threshold based slide implementation. Small amounts of
> difference between frames only affect the output a little bit, but
> larger changes are visibly immediately. I've cleaned up and uploaded
> a CPU and GPU based version below. It uses floating point values so
> that the "slide" is effective, which means that it is slow on the
> CPU, and for a decent GPU version, your card will need to support
> floating point textures.
>
> http://www.musorkvision.com/jkc/temp-denoise-example.zip
>
> You can also use adaptive filtering spatially (perhaps combined with
> temporal analysis/processing). I've included a version which uses a
> kind of multipass adaptive median filter to smooth small change (this
> sort of thing can also be used for things like skin tone smoothing
> filters and such). The shader can easily be modified to be an
> adaptive mean filter if that works better for your material.
>
> More advanced denoising algorithms also incorporate optical flow in
> the analysis/processing. For this sort of thing, you can perhaps use
> Jean Marc Pelletier's cv.jit objects for. Might be tricky and CPU
> intensive, however.
>
> Hope this helps.
>
> -Joshua
>

On 9/11/07, yair reshef wrote:
>
> here is a gpu based background subtraction i use often. ill try to
> implement your recommendations.
> i use a slide based averaging technique for the background, having control
> over your lighting is a crucial point, more light, less noise. the slide
> routine i use accept a nice range of lighting conditions, i use the infra
> red spectrum, and the sun is a major source for ir. but ir is not good for
> everything.
>
> patch + a basic luma shader hacked from on of the others.
>
>
>
> On 9/11/07, Joshua Kit Clayton wrote:
> >
> >
> > On Sep 11, 2007, at 9:43 AM, Scott Fitzgerald wrote:
> >
> > > Without resorting to jit.slide on the image itself, does anyone have a
> > > handy (and not too cpu intensive) means of removing noise from a
> > > firewire camera image?
> >
> > The first things to consider before you get to processing your image
> > are the standard things for reducing thermal noise:
> >
> > 1. Use as much light as possible in the space you are filming (so
> > that the ratio of light to thermal noise is as high as possible)
> > 2. Use as large a CCD camera as possible
> > 3. Try to keep the camera cool somehow (obviously not always possible
> > or practical)
> >
> > Then, it would make sense to start processing. The following doesn't
> > work for all situations, but one thing that I've used successfully
> > recently is a threshold based slide implementation. Small amounts of
> > difference between frames only affect the output a little bit, but
> > larger changes are visibly immediately. I've cleaned up and uploaded
> > a CPU and GPU based version below. It uses floating point values so
> > that the "slide" is effective, which means that it is slow on the
> > CPU, and for a decent GPU version, your card will need to support
> > floating point textures.
> >
> > http://www.musorkvision.com/jkc/temp-denoise-example.zip
> >
> > You can also use adaptive filtering spatially (perhaps combined with
> > temporal analysis/processing). I've included a version which uses a
> > kind of multipass adaptive median filter to smooth small change (this
> > sort of thing can also be used for things like skin tone smoothing
> > filters and such). The shader can easily be modified to be an
> > adaptive mean filter if that works better for your material.
> >
> > More advanced denoising algorithms also incorporate optical flow in
> > the analysis/processing. For this sort of thing, you can perhaps use
> > Jean Marc Pelletier's cv.jit objects for. Might be tricky and CPU
> > intensive, however.
> >
> > Hope this helps.
> >
> > -Joshua
> >
>
>

Wow, these are pretty swell, thanks Joshua.

however, I'm running this on an Intel mac mini, so too much on the gpu
isn't really going to fly. Right now I'm playing with the camera's
settings, tryig to finesse it as much as possible, particularly since
the lighting is fixed (and they didn't exactly listen to me).

But perhaps the small slide would work well.

I'm getting the refernce frame with cv.jit.ravg @alpha .1 and doing
some dilation and erosion on the mask to clean up the sides a bit.
Obviously works better in some situations than others.

Thnaks again,

~s

On 9/11/07, Joshua Kit Clayton wrote:
>
> On Sep 11, 2007, at 9:43 AM, Scott Fitzgerald wrote:
>
> > Without resorting to jit.slide on the image itself, does anyone have a
> > handy (and not too cpu intensive) means of removing noise from a
> > firewire camera image?
>
> The first things to consider before you get to processing your image
> are the standard things for reducing thermal noise:
>
> 1. Use as much light as possible in the space you are filming (so
> that the ratio of light to thermal noise is as high as possible)
> 2. Use as large a CCD camera as possible
> 3. Try to keep the camera cool somehow (obviously not always possible
> or practical)
>
> Then, it would make sense to start processing. The following doesn't
> work for all situations, but one thing that I've used successfully
> recently is a threshold based slide implementation. Small amounts of
> difference between frames only affect the output a little bit, but
> larger changes are visibly immediately. I've cleaned up and uploaded
> a CPU and GPU based version below. It uses floating point values so
> that the "slide" is effective, which means that it is slow on the
> CPU, and for a decent GPU version, your card will need to support
> floating point textures.
>
> http://www.musorkvision.com/jkc/temp-denoise-example.zip
>
> You can also use adaptive filtering spatially (perhaps combined with
> temporal analysis/processing). I've included a version which uses a
> kind of multipass adaptive median filter to smooth small change (this
> sort of thing can also be used for things like skin tone smoothing
> filters and such). The shader can easily be modified to be an
> adaptive mean filter if that works better for your material.
>
> More advanced denoising algorithms also incorporate optical flow in
> the analysis/processing. For this sort of thing, you can perhaps use
> Jean Marc Pelletier's cv.jit objects for. Might be tricky and CPU
> intensive, however.
>
> Hope this helps.
>
> -Joshua
>

and yair,

also nice. thanks, i'll see if I can't adapt any of the above
solutions to this situation.

~s

On 9/11/07, yair reshef wrote:
>
>
> On 9/11/07, yair reshef wrote:
> > here is a gpu based background subtraction i use often. ill try to
> implement your recommendations.
> > i use a slide based averaging technique for the background, having control
> over your lighting is a crucial point, more light, less noise. the slide
> routine i use accept a nice range of lighting conditions, i use the infra
> red spectrum, and the sun is a major source for ir. but ir is not good for
> everything.
> >
> > patch + a basic luma shader hacked from on of the others.
> >
> >
> >
> >
> >
> > On 9/11/07, Joshua Kit Clayton wrote:
> > >
> > > On Sep 11, 2007, at 9:43 AM, Scott Fitzgerald wrote:
> > >
> > > > Without resorting to jit.slide on the image itself, does anyone have a
> > > > handy (and not too cpu intensive) means of removing noise from a
> > > > firewire camera image?
> > >
> > > The first things to consider before you get to processing your image
> > > are the standard things for reducing thermal noise:
> > >
> > > 1. Use as much light as possible in the space you are filming (so
> > > that the ratio of light to thermal noise is as high as possible)
> > > 2. Use as large a CCD camera as possible
> > > 3. Try to keep the camera cool somehow (obviously not always possible
> > > or practical)
> > >
> > > Then, it would make sense to start processing. The following doesn't
> > > work for all situations, but one thing that I've used successfully
> > > recently is a threshold based slide implementation. Small amounts of
> > > difference between frames only affect the output a little bit, but
> > > larger changes are visibly immediately. I've cleaned up and uploaded
> > > a CPU and GPU based version below. It uses floating point values so
> > > that the "slide" is effective, which means that it is slow on the
> > > CPU, and for a decent GPU version, your card will need to support
> > > floating point textures.
> > >
> > >
> http://www.musorkvision.com/jkc/temp-denoise-example.zip
> > >
> > > You can also use adaptive filtering spatially (perhaps combined with
> > > temporal analysis/processing). I've included a version which uses a
> > > kind of multipass adaptive median filter to smooth small change (this
> > > sort of thing can also be used for things like skin tone smoothing
> > > filters and such). The shader can easily be modified to be an
> > > adaptive mean filter if that works better for your material.
> > >
> > > More advanced denoising algorithms also incorporate optical flow in
> > > the analysis/processing. For this sort of thing, you can perhaps use
> > > Jean Marc Pelletier's cv.jit objects for. Might be tricky and CPU
> > > intensive, however.
> > >
> > > Hope this helps.
> > >
> > > -Joshua
> > >
> >
> >
>
>
>
>
>

hi yair,

would like to try your de-noising-approach, but unfortunately
i can't see the attachment in the forum...

could you try to upload it from within there (or whatever helps in cases like this)?

many thanks,

jan klug

patch and shader follows

//shader

Luminance based keying
Target luminance
Luminance coefficients (RGBA)

/>

On 9/13/07, Jan Klug wrote:
>
>
> hi yair,
>
> would like to try your de-noising-approach, but unfortunately
> i can't see the attachment in the forum...
>
> could you try to upload it from within there (or whatever helps in cases
> like this)?
>
> many thanks,
>
>
> jan klug
>

Hello,
I would like to try Yair's patch.
It must be obvious, but I don't understand where is the code or files for the shaders needed in his patch.
co.luma.jxs
co.difference.jxs
co.luma.jxs
cc.uyvy2rgba.jxs

Sorry, I'm new to OpenGl ...
can anyone help me?
Thanks
Felix

On Dec 19, 2007 2:44 PM, Felix wrote:

>
> Hello,
> I would like to try Yair's patch.
> It must be obvious, but I don't understand where is the code or files for
> the shaders needed in his patch.
> co.luma.jxs
> co.difference.jxs
> co.luma.jxs
> cc.uyvy2rgba.jxs
>
this are included with the jitter original distro.
the first patch should be copy pasted into a new patch. it will automagicly
appear.
the shader should be copy pasted and put somewhere in the max folder. look
in the max tutorial for more on this.
have fun and proser.

>
> Sorry, I'm new to OpenGl ...
> can anyone help me?
> Thanks
> Felix
> --
> Felix Luque
> http://www.othersounds.net
> felix@othersounds.net
>