Syphon -> cv.jit: Alternatives to Texture-to-Matrix Conversion to Avoid Slowdown
I am using Syphon-fed video feeds from 3 different cameras (PS3 EYEs) in combination with cv.jit.blobs to track IR sources in a stage performance.
I am currently simply connecting jit.gl.syphonclient to jit.matrix in order to get something that would work with cv.jit, which does not accept textures but only matrices. This causes a serious slowdown (due to software rendering), reducing the incoming video (upwards of 40 FPS) to a crawling 20FPS, which makes blob tracking much harder.
I also tried rendering each texture to its own videoplane, placing them side-to-side in 3d space, and rendering the scene as a whole to a matrix (later to be scissored into 3), but there is no lesser slowdown with that setup.
Is there any way to do this that's more efficient? Am I not doing texture-to-matrix conversion in the best way possible?
It would truly be a life saver if the answer to those questions was 'yes'; we are very late in the production cycle and time is running out.
A thousand thanks in advance!
I sadly don't have an answer but i'd be curious to know what software you're using for the ps3 eyes, are the 3 connected to the same computer ?
well, i don't think there's gonna be a magic bullet for this.
the first suggestion is obviously, to downsample as much as possible. i'm not sure if it's more efficient to downsample from texture to texture or from matrix to matrix. you'll have to run some comparisons.
downsampling to texture would be something like sending the output of syphon to a jit.gl.texture @adapt 0 @dim 80 60
the other suggestion is use jit.gl.asyncread, instead of rendering to a matrix.
again, keep your resolution as low as possible.
Definitely recommend downsampling on GPU before moving to CPU for CV techniques.
Thank you very much for your help!
@nat: I'm using the 'macam' USB webcam driver on OS X Mountain Lion in order to get the cameras detected as hardware, and then this little gem:
It's written in openFrameworks and allows Syphoning multiple PS3 Eyes (at high framerates) anywhere. Also, built-in extensive GUI, including shutter/gain manipulation.