I’m trying to use the CV library to determine facial tilt. I have eye tracking and face tracking working (from the examples), but it’s hard to discern the titling of the face from eye tracking or face tracking alone.

Using head tracking together with eye tracking could give you the relative location of the eyes in comparison to the head, but there doesn’t seem to be any information on how to achieve head tracking using cv.jit.

Does anyone have any experience tracking heads in cv.jit?

Thanks
Ramona

I did this by tracking the points near the relative location of the
eyes, and then calculating the angle between them(between the line
they’re forming and edge of the matrix).

pasted below is an old tryout i dug up, hope it helps…

best,
nesa

#P window setfont “Sans Serif” 9.;
#P window linecount 1;
#P newex 542 495 33 196617 !- 90;
#P user dial 542 524 60 60 360 1 0 0 159 359 1 1. 170 170 170 221 221
221 120 120 120 225 225 225 0 0 0 0 0 0;
#P button 196 272 15 0;
#P message 220 266 33 196617 clear;
#P newex 542 460 46 196617 * -180.;
#P newex 542 439 63 196617 / 3.141593;
#P flonum 579 482 35 9 0 0 0 3 0 0 0 221 221 221 222 222 222 0 0 0;
#P comment 616 483 44 196617 degrees;
#P newex 507 403 45 196617 cartopol;
#P newex 507 374 80 196617 vexpr $f2 -$f1;
#P newex 519 284 27 196617 + 1;
#P newex 486 326 119 196617 gate 6;
#P newex 504 233 40 196617 jit.iter;
#P newex 335 266 29 196617 sel 1;
#P toggle 304 305 15 0;
#P newex 304 267 27 196617 < 6.;
#P newex 304 242 52 196617 cv.jit.sum;
#P newex 304 218 106 196617 jit.unpack 1 @offset 2;
#P newex 17 143 123 196617 jit.dimmap @invert 1 0;
#P newex 61 264 60 196617 jit.op @op +;
#P user jit.fpsgui 174 80 60 196617 0;
#P newex 17 219 124 196617 jit.matrix 4 char 320 240;
#P newex 165 226 135 196617 cv.jit.track.draw 160 120;
#P user jit.pwindow 24 330 322 242 0 1 0 0 1 0;
#P message 368 134 14 196617 1;
#P window setfont “Sans Serif” 12.;
#N vpatcher 846 87 1647 714;
#P origin 0 36;
#P window setfont “Sans Serif” 9.;
#P user jit.fpsgui 108 174 60 196617 0;
#P flonum 21 115 35 9 0 0 0 3 0 0 0 221 221 221 222 222 222 0 0 0;
#P message 21 135 50 196617 bscale $1;
#P flonum 21 75 35 9 0 0 0 3 0 0 0 221 221 221 222 222 222 0 0 0;
#P message 21 95 50 196617 gscale $1;
#P flonum 21 37 35 9 0 0 0 3 0 0 0 221 221 221 222 222 222 0 0 0;
#P message 21 57 50 196617 rscale $1;
#P newex 302 227 100 196617 cv.jit.faces.draw;
#P user jit.pwindow 405 83 82 62 0 0 0 0 1 0;
#P comment 214 376 66 196617 face found;
#P comment 298 376 123 196617 still looking for face…;
#P button 281 375 15 0;
#P button 199 375 15 0;
#P newex 96 56 61 196617 jit.rgb2luma;
#N vpatcher 331 161 646 432;
#P window setfont “Sans Serif” 9.;
#P window linecount 1;
#P newex 72 114 124 196617 jit.matrix 1 char 160 120;
#P inlet 72 83 15 0;
#P window linecount 0;
#P newex 72 144 50 196617 jit.change;
#P outlet 72 194 15 0;
#P connect 2 0 3 0;
#P connect 3 0 1 0;
#P connect 1 0 0 0;
#P pop;
#P newobj 96 122 84 196617 p prepare matrix;
#N vpatcher 141 144 491 388;
#P inlet 89 59 15 0;
#P window setfont “Sans Serif” 9.;
#P newex 89 92 156 196617 jit.matrix 1 char 80 60 @adapt 0;
#P newex 89 122 50 196617 jit.change;
#P outlet 89 159 15 0;
#P connect 3 0 2 0;
#P connect 2 0 1 0;
#P connect 1 0 0 0;
#P pop;
#P newobj 188 189 84 196617 p prepare matrix;
#P comment 346 306 180 196617 4: 80×60 -> 320×240(coordinate fix);
#P comment 345 287 180 196617 2: 80×60 -> 160×120(coordinate fix);
#P newex 188 285 108 196617 jit.op @op * @val 2.;
#P newex 188 222 103 196617 cv.jit.faces @model 2;
#N vpatcher 501 83 1090 533;
#P window setfont “Sans Serif” 9.;
#P window linecount 1;
#P newex 307 244 40 196617 t i i;
#P window linecount 0;
#P newex 75 289 27 196617 !- 1;
#P window linecount 1;
#P newex 75 314 50 196617 gate;
#N comlet face coordinates;
#P outlet 75 365 15 0;
#P comment 133 315 149 196617 face found , pass track points;
#P window linecount 0;
#P newex 115 124 103 196617 t l l;
#P window linecount 2;
#P comment 231 205 73 196617 check face size…;
#P window linecount 1;
#P newex 307 208 32 196617 < = 0.;
#P newex 208 171 110 196617 unpack 0. 0. 0. 0.;
#P window linecount 2;
#P comment 347 317 113 196617 1 – face not found , try again in next
frame;
#P inlet 115 72 15 0;
#N comlet bang if face not found;
#P outlet 337 381 15 0;
#P connect 11 0 10 0;
#P connect 10 0 9 0;
#P connect 9 0 8 0;
#P connect 1 0 6 0;
#P connect 6 0 9 1;
#P connect 6 1 3 0;
#P connect 3 3 4 0;
#P connect 4 0 11 0;
#P connect 11 1 0 0;
#P pop;
#P newobj 188 338 103 196617 p is the face detected;
#P outlet 96 546 15 0;
#N vpatcher 516 141 829 459;
#P window setfont “Sans Serif” 9.;
#P newex 55 142 45 196617 gate 1 1;
#N comlet gate;
#P inlet 114 27 15 0;
#N comlet incoming matrix;
#P inlet 50 30 15 0;
#P outlet 55 215 15 0;
#P connect 2 0 3 0;
#P connect 3 0 0 0;
#P connect 1 0 3 1;
#P pop;
#P newobj 188 162 75 196617 p inverted gate;
#N comlet bang to find face/get track points;
#P inlet 253 77 15 0;
#P newex 96 86 102 196617 t l l;
#P newex 96 517 227 196617 cv.jit.track @npoints 6 @radius 3;
#N comlet video input;
#P inlet 96 27 15 0;
#P newex 188 402 414 196617 t l l l l l l;
#N vpatcher 25 70 490 371;
#P window setfont “Sans Serif” 9.;
#P window linecount 1;
#P newex 49 193 69 196617 prepend set 5;
#P window linecount 0;
#P newex 50 154 52 196617 pack 0. 0.;
#P newex 87 126 27 196617 + 0.;
#P newex 50 127 27 196617 + 0.;
#P newex 173 82 37 196617 * 0.76;
#P newex 124 82 37 196617 * 0.66;
#P newex 50 50 122 196617 unpack 0. 0. 0. 0.;
#P inlet 50 30 15 0;
#P outlet 51 241 15 0;
#P connect 7 0 8 0;
#P connect 1 0 2 0;
#P connect 2 0 5 0;
#P connect 5 0 7 0;
#P connect 8 0 0 0;
#P connect 3 0 5 1;
#P connect 2 1 6 0;
#P connect 6 0 7 1;
#P connect 4 0 6 1;
#P connect 2 2 3 0;
#P connect 2 3 4 0;
#P pop;
#P newobj 588 433 57 196617 p lips_right;
#N vpatcher 25 70 490 371;
#P window setfont “Sans Serif” 9.;
#P window linecount 1;
#P newex 48 188 69 196617 prepend set 4;
#P window linecount 0;
#P newex 50 154 52 196617 pack 0. 0.;
#P newex 87 126 27 196617 + 0.;
#P newex 50 127 27 196617 + 0.;
#P newex 173 82 37 196617 * 0.76;
#P newex 124 82 43 196617 * 0.355;
#P newex 50 50 122 196617 unpack 0. 0. 0. 0.;
#P inlet 50 30 15 0;
#P outlet 50 229 15 0;
#P connect 7 0 8 0;
#P connect 1 0 2 0;
#P connect 2 0 5 0;
#P connect 5 0 7 0;
#P connect 8 0 0 0;
#P connect 3 0 5 1;
#P connect 2 1 6 0;
#P connect 6 0 7 1;
#P connect 4 0 6 1;
#P connect 2 2 3 0;
#P connect 2 3 4 0;
#P pop;
#P newobj 508 433 51 196617 p lips_left;
#N vpatcher 25 70 330 368;
#P window setfont “Sans Serif” 9.;
#P window linecount 1;
#P newex 50 187 69 196617 prepend set 3;
#P window linecount 0;
#P newex 50 154 52 196617 pack 0. 0.;
#P newex 87 126 27 196617 + 0.;
#P newex 50 127 27 196617 + 0.;
#P newex 167 82 27 196617 * 1.;
#P newex 124 82 37 196617 * 0.5;
#P newex 50 50 122 196617 unpack 0. 0. 0. 0.;
#P inlet 50 30 15 0;
#P outlet 51 220 15 0;
#P connect 1 0 2 0;
#P connect 2 0 5 0;
#P connect 5 0 7 0;
#P connect 7 0 8 0;
#P connect 8 0 0 0;
#P connect 3 0 5 1;
#P connect 2 1 6 0;
#P connect 6 0 7 1;
#P connect 4 0 6 1;
#P connect 2 2 3 0;
#P connect 2 3 4 0;
#P pop;
#P newobj 428 433 33 196617 p chin;
#N vpatcher 186 137 651 438;
#P window setfont “Sans Serif” 9.;
#P window linecount 1;
#P newex 51 186 69 196617 prepend set 2;
#P window linecount 0;
#P newex 50 154 52 196617 pack 0. 0.;
#P newex 87 126 27 196617 + 0.;
#P newex 50 127 27 196617 + 0.;
#P newex 167 82 43 196617 * 0.355;
#P newex 124 82 37 196617 * 0.73;
#P newex 50 50 122 196617 unpack 0. 0. 0. 0.;
#P inlet 50 30 15 0;
#P outlet 50 217 15 0;
#P connect 1 0 2 0;
#P connect 2 0 5 0;
#P connect 5 0 7 0;
#P connect 8 0 0 0;
#P connect 7 0 8 0;
#P connect 3 0 5 1;
#P connect 2 1 6 0;
#P connect 6 0 7 1;
#P connect 4 0 6 1;
#P connect 2 2 3 0;
#P connect 2 3 4 0;
#P pop;
#P newobj 348 433 59 196617 p right_eye;
#N vpatcher 25 70 490 371;
#P window setfont “Sans Serif” 9.;
#P newex 51 180 69 196617 prepend set 1;
#P newex 50 154 52 196617 pack 0. 0.;
#P newex 87 126 27 196617 + 0.;
#P newex 50 127 27 196617 + 0.;
#P newex 167 82 43 196617 * 0.355;
#P newex 124 82 37 196617 * 0.33;
#P newex 50 50 122 196617 unpack 0. 0. 0. 0.;
#P inlet 50 30 15 0;
#P outlet 50 209 15 0;
#P connect 1 0 2 0;
#P connect 2 0 5 0;
#P connect 5 0 7 0;
#P connect 8 0 0 0;
#P connect 7 0 8 0;
#P connect 3 0 5 1;
#P connect 2 1 6 0;
#P connect 6 0 7 1;
#P connect 4 0 6 1;
#P connect 2 2 3 0;
#P connect 2 3 4 0;
#P pop;
#P newobj 268 433 53 196617 p left_eye;
#N vpatcher 100 275 565 576;
#P window setfont “Sans Serif” 9.;
#P newex 87 126 27 196617 + 0.;
#P newex 50 127 27 196617 + 0.;
#P newex 161 82 37 196617 * 0.61;
#P newex 124 82 31 196617 * 0.5;
#P newex 50 50 122 196617 unpack 0. 0. 0. 0.;
#P inlet 50 30 15 0;
#P outlet 50 214 15 0;
#P newex 50 180 69 196617 prepend set 0;
#P newex 50 154 52 196617 pack 0. 0.;
#P connect 3 0 4 0;
#P connect 4 0 7 0;
#P connect 7 0 0 0;
#P connect 0 0 1 0;
#P connect 1 0 2 0;
#P connect 5 0 7 1;
#P connect 4 1 8 0;
#P connect 8 0 0 1;
#P connect 6 0 8 1;
#P connect 4 2 5 0;
#P connect 4 3 6 0;
#P pop;
#P newobj 188 433 36 196617 p nose;
#P newex 188 315 40 196617 jit.iter;
#P newex 188 255 404 196617 jit.expr @expr “in[0].p[0]” “in[0].p[1]”
“in[0].p[2]-in[0].p[0]” “in[0].p[3]-in[0].p[1]“;
#P comment 598 257 100 196617 x , y , width , height;
#P connect 31 0 30 0;
#P connect 33 0 32 0;
#P connect 35 0 34 0;
#P connect 10 0 23 0;
#P connect 30 0 23 0;
#P connect 34 0 23 0;
#P connect 32 0 23 0;
#P connect 23 0 12 0;
#P connect 12 0 22 0;
#P connect 22 0 11 0;
#P fasten 3 0 11 0 193 466 101 466;
#P fasten 4 0 11 0 273 466 101 466;
#P fasten 5 0 11 0 353 466 101 466;
#P fasten 6 0 11 0 433 466 101 466;
#P fasten 7 0 11 0 513 466 101 466;
#P fasten 8 0 11 0 593 466 101 466;
#P connect 11 0 15 0;
#P fasten 22 0 36 0 101 157 113 157;
#P connect 12 1 14 0;
#P connect 14 0 21 0;
#P connect 21 0 17 0;
#P connect 17 0 1 0;
#P connect 1 0 18 0;
#P connect 18 0 2 0;
#P connect 2 0 16 0;
#P connect 16 0 9 0;
#P connect 9 0 3 0;
#P connect 16 0 24 0;
#P fasten 16 1 14 1 286 361 756 361 756 150 258 150;
#P connect 13 0 14 1;
#P connect 9 1 4 0;
#P connect 16 1 25 0;
#P connect 17 0 29 0;
#P connect 9 2 5 0;
#P connect 21 0 29 1;
#P connect 29 0 28 0;
#P connect 9 3 6 0;
#P connect 9 4 7 0;
#P connect 9 5 8 0;
#P pop;
#P newobj 165 174 213 196620 p find face and track its points;
#P window setfont “Sans Serif” 9.;
#P newex 62 116 51 196617 s grabber;
#P toggle 17 21 15 0;
#P message 120 51 30 196617 close;
#P message 82 49 31 196617 open;
#P newex 17 44 55 196617 qmetro 20;
#P newex 17 91 146 196617 jit.qt.grab 320 240 @unique 0;
#P comment 324 306 114 196617 are some points lost?;
#P connect 28 0 32 0;
#P connect 28 0 26 0;
#P connect 27 0 28 0;
#P connect 32 0 31 0;
#P connect 10 0 13 1;
#P connect 10 0 30 0;
#P connect 30 0 29 0;
#P connect 7 0 10 0;
#P fasten 7 0 15 0 170 202 309 202;
#P fasten 7 0 20 0 170 200 509 200;
#P connect 29 0 10 1;
#P connect 24 1 27 0;
#P connect 23 0 24 0;
#P connect 21 1 23 0;
#P connect 21 2 23 1;
#P connect 2 0 1 0;
#P connect 2 0 12 0;
#P connect 1 0 14 0;
#P connect 1 0 6 0;
#P connect 4 0 1 0;
#P connect 3 0 1 0;
#P fasten 20 0 21 1 509 321 600 321;
#P connect 20 1 22 0;
#P connect 22 0 21 0;
#P connect 8 0 7 1;
#P fasten 19 0 8 0 340 291 398 291 398 126 373 126;
#P connect 17 0 19 0;
#P connect 17 0 18 0;
#P connect 16 0 17 0;
#P connect 15 0 16 0;
#P connect 14 0 11 0;
#P connect 14 0 7 0;
#P connect 11 0 13 0;
#P connect 13 0 9 0;
#P connect 5 0 2 0;
#P window clipboard copycount 33;