wippen

Has anyone gathered any experience using a CMUcam? Specifically: if used as a sensor for the production of MIDI-data, is there any hidden latency which is not documented in the pertinent documents?

For the uninitiated: the CMUcam2 claims to do frame differencing at 26fps and color tracking at up to 50fps. Both of these framerates are fine for using the CMUcam as a sensor (although the 50fps is a very promising value, even at a low resolution as a zone sensor). I am worried that there is additional latency somewhere in the design which would exceed values sufficiently low to be used for music.

Has anyone gathered any experience using a CMUcam?  Specifically: if used as a sensor for the production of MIDI-data, is there any hidden latency which is not documented in the pertinent documents?

For the uninitiated: the CMUcam2 claims to do frame differencing at 26fps and color tracking at up to 50fps.  Both of these framerates are fine for using the CMUcam as a sensor (although the 50fps is a very promising value, even at a low resolution as a zone sensor).  I am worried that there is additional latency somewhere in the design which would exceed values sufficiently low to be used for music.  

ot-cmucam-as-sensor

Here's a partial answer to my own question:

The CMUcam2 can track colors at 50fps (a very respectable rate, more than low enough for musical timing) if the resolution is set low enough; meaning a vertical resolution of 65 (I assume this means horizontally 85). If the thing being tracked is, say, the musician himself, then the left, top and bottom-most extensions of the person in the field of view create a rectangular area, which is what is being sent as data and discrete zones are possible only in a limited manner. Although this is not a very good resolution for MIDI, it can be interpolated with [slide] to emulate a better resolution.

Frame differencing is possible at 26fps at a resolution of 8x8.

I could not find any information leading me to believe that there is any extra latency in the system; the baud rates available are more than enough to use for a good MIDI translation.

I've gotten used to precise definition of discrete zones and responses using 320x240 resolution with Jitter and various motion-tracking schemes, so it seems a shame to limit the possibilities, but it would be nice to have a stand-alone vision-sensor which can send out MIDI and leave the computers to do other tasks.

The CMUcam2 can track colors at 50fps (a very respectable rate, more than low enough for musical timing) if the resolution is set low enough; meaning a vertical resolution of 65 (I assume this means horizontally 85).  If the thing being tracked is, say, the musician himself, then the left, top and bottom-most extensions of the person in the field of view create a rectangular area, which is what is being sent as data and discrete zones are possible only in a limited manner.  Although this is not a very good resolution for MIDI, it can be interpolated with [slide] to emulate a better resolution.

I've gotten used to precise definition of discrete zones and responses using 320x240 resolution with Jitter and various motion-tracking schemes, so it seems a shame to limit the possibilities, but it would be nice to have a stand-alone vision-sensor which can send out MIDI and leave the computers to do other tasks.


I now see that it IS possible to do line-mode analysis, so discrete zones CAN be set up. Interesting; this could be quite a useful device...

One more thing:
I now see that it IS possible to do line-mode analysis, so discrete zones CAN be set up.  Interesting; this could be quite a useful device...


dont you love when you write the whole thread yourself.

Sounds like a cool device, im gonna check it out.

> The CMUcam2 can track colors at 50fps (a very respectable rate, more

This is 20ms jitter, unacceptable for rhythms, but fine for controller 

20ms latency could be handled by a musician, but not 20ms of jitter...

Dayton schrieb:
> The CMUcam2 can track colors at 50fps (a very respectable rate, more
> than low enough for musical timing)

This is 20ms jitter, unacceptable for rhythms, but fine for controller 
data...
20ms latency could be handled by a musician, but not 20ms of jitter...

-- 
Stefan Tiedje------------x-------
--_____-----------|--------------
--(_|_ ----|-----|-----()-------
-- _|_)----|-----()--------------
----------()--------www.ccmix.com


OT CMUcam as sensor