qudaparcs

I've pieced together a patch that (sometimes accurately) tracks a pair of eyes by first recognizing most prominent face captured via a webcam and utilizing cv.jit.faces.largest & cv.jit.faces.eyes.img objects (produces two matrixes, each zoomed in on one eye).

Have spent the better part of this weekend trying to figure out a means of time-delaying and then overlaying the two eye matrixes back onto the live-feed for output / projection... Basically I wish to replace a person's eye movements with those that took place several seconds prior.

The data produced by the cv.jit objects would have to be used to 'glue' the time-delayed matrixes onto any face recognized by the patch, and also to get the time-delayed eyes to follow the face while it moves in-frame. I can't seem to figure out how to do this....

I have xyxy box dimensions for a face. I've got xy data streams for the pair of eyes that will be delayed. I have a subpatch that will do the delaying. And I found reference in the forums on overlaying planes onto a matrix with jit.gl.videoplane objects but... By now I'd have to say that I've lost m'way. Or the will? Flummoxed! ;)

Anyone have feedback / advice / suggestions on this front?

I've pieced together a patch that (sometimes accurately) tracks a pair of eyes by first recognizing most prominent face captured via a webcam and  utilizing cv.jit.faces.largest & cv.jit.faces.eyes.img objects (produces two matrixes, each zoomed in on one eye).

Anyone have feedback / advice / suggestions on this front?


delay-new-feed-follow-eye-tracking-wcv-jit

Edited this in order to replace patcher contents (that I had previously pasted in) with a zip folder containing just the functional sections... In case it helps to contextualize the problem I attempt to explain above. Hope this helps!

Poo. Folder too large. Back to copy/pasting...

The code doesn't run. Can't you compress it and copy and paste that? (select everything in patch window, edit, copy compressed, paste here).

Am compress-copying the patch I'm trying to problem solve in - working just with facial recognition (not the eye tracking) - so as to simplify for the time being.

- Left side of patch produces centre xy data of faces.

- cv.jit.faces / draw is set up to add a photo (.png attached), instead of the delayed video, so that you can see an example of the aim and problem.

- gl objects were set up just a possible alternate solution to layering delayed video onto live feed but my efforts to get it to follow along tracking paths froze up Max upon patch launches; so those have been removed.

- Right side of patch produces the delayed video that, in concept, would take the place of the photo and could the follow / replace real-time face. My thought was that the centre xy data would provide accuracy but I guess I just can't find a way to incorporate it.

I'm working on Mac - frequently freezes on the macbook but not on the desktop - and with Max 6.

- gl objects were set up just a possible alternate solution to layering  delayed video onto live feed but my efforts to get it to follow along tracking paths froze up Max upon patch launches; so those have been removed.

Thank-you, in advance, for any insights!


Delay new feed & follow eye tracking (w/cv.jit)?