Kinect V1 360° rotation tracking

G's icon

Hi all,

I'm building a one-person spatial sound installation (8 speakers, mc.ambipanning~) where the visitor's body rotation steers the sound. I am using a Kinect V1 but I'm having trouble with making a stable 0-360 heading.

My current setup is: Kinect v1 -> TouchDesigner (sending skeleton and face tracking) on a microsoft laptop -> OSC -> Max on a macbook. I compute the angle of the shoulder line with cartopol, and a small [js] unwraps it (picks 'angle' or 'angle +180, whichever is closest to the previous value). face:tracked is used as a front/back anchor.

Problem: it still flips by 180, especially when the visitor stands side-on to the kinect or turns quickly. The kinect always assumes you're facing it?

Question:

  1. Any robust ways to get a full 360° heading from kinect skeleton / face tracking data?

  2. Better front/back cues than face tracking

I prefer not to put any sensors on the visitors.

Thanks!

Rei Kurosawa's icon

Could you share a short OSC trace covering one of the flips, before the values enter your unwrap script? Please include the raw left and right shoulder positions, their tracking status and `face:tracked` for the same few frames. Seeing those alongside the angle before and after your unwrap would help pin down where the 180° jump first appears.

TFL's icon

Probably the perfect job for the new Vision package available in the Package Manager. [jit.vis.body] provides skeleton estimation based on simple camera image (no need for a kinect). I just tested and the skeleton doesn't flip to keep facing the camera (it can rotate 360°).

Face tracking can be done with [jit.vis.face] or [jit.vis.facemesh]

This approach would even allow you to get rid of the Windows computer with Touch Designer.