Delay new feed & follow eye tracking (w/cv.jit)?

Nov 17, 2013 at 5:22pm

Delay new feed & follow eye tracking (w/cv.jit)?

Hi forum friends,

The aim:

I’ve pieced together a patch that (sometimes accurately) tracks a pair of eyes by first recognizing most prominent face captured via a webcam and utilizing cv.jit.faces.largest & cv.jit.faces.eyes.img objects (produces two matrixes, each zoomed in on one eye).

Have spent the better part of this weekend trying to figure out a means of time-delaying and then overlaying the two eye matrixes back onto the live-feed for output / projection… Basically I wish to replace a person’s eye movements with those that took place several seconds prior.

The problem:

The data produced by the cv.jit objects would have to be used to ‘glue’ the time-delayed matrixes onto any face recognized by the patch, and also to get the time-delayed eyes to follow the face while it moves in-frame. I can’t seem to figure out how to do this….

I have xyxy box dimensions for a face. I’ve got xy data streams for the pair of eyes that will be delayed. I have a subpatch that will do the delaying. And I found reference in the forums on overlaying planes onto a matrix with jit.gl.videoplane objects but… By now I’d have to say that I’ve lost m’way. Or the will? Flummoxed! ;)

Anyone have feedback / advice / suggestions on this front?

#271488
Nov 18, 2013 at 9:50am

Edited this in order to replace patcher contents (that I had previously pasted in) with a zip folder containing just the functional sections… In case it helps to contextualize the problem I attempt to explain above. Hope this helps!

#271574
Nov 18, 2013 at 1:48pm

Poo. Folder too large. Back to copy/pasting…

#271612
Nov 22, 2013 at 8:20pm

The code doesn’t run. Can’t you compress it and copy and paste that? (select everything in patch window, edit, copy compressed, paste here).

#273536
Nov 23, 2013 at 5:27am

Hi Whorl,

Am compress-copying the patch I’m trying to problem solve in – working just with facial recognition (not the eye tracking) – so as to simplify for the time being.

- Left side of patch produces centre xy data of faces.

- cv.jit.faces / draw is set up to add a photo (.png attached), instead of the delayed video, so that you can see an example of the aim and problem.

- gl objects were set up just a possible alternate solution to layering delayed video onto live feed but my efforts to get it to follow along tracking paths froze up Max upon patch launches; so those have been removed.

- Right side of patch produces the delayed video that, in concept, would take the place of the photo and could the follow / replace real-time face. My thought was that the centre xy data would provide accuracy but I guess I just can’t find a way to incorporate it.

I’m working on Mac – frequently freezes on the macbook but not on the desktop – and with Max 6.

Thank-you, in advance, for any insights!

<code>

– Pasted Max Patch, click to expand. –

</code>

Attachments:
  1. lenseye1
  2. lenseye
#273557

You must be logged in to reply to this topic.