dp.kinect2: actively loose user?
hi,
i have a question regarding the amazing dp.kinect2 external (which i use for face tracking):
whenever the user currently being tracked moves to the outer limits of the kinect's viewing field, the kinect can't track him/her anymore (no tracking data is sent) but still recognizes the user (lost_user is not sent yet). now, the problem with this is that since the kinect ceases to send tracking data, every time this happens, my patch appears to be stalled (although actually everything is fine).
hence my question: is it possible to actively tell the kinect to loose the current user as soon as it can't track him/her any more (and look for a new one)?
and another small question: i only want to track a single user at a time - how do i do this most efficiently? i reckon i could apply a dynamic filter based on the playerID to all data but i wonder wether there is an easier way (like a single-user-mode or something)...?
thanks for any help!
k
Hi! I am not behind my work computer but maybe I can help you from memory and the help of the web...
About the face going to the edges:
When using skeleton tracking there are messages for when a person is to close to an edge. Maybe this also works for face tracking?
From the documentation on: https://github.com/diablodale/dp.kinect2/wiki/Message-based-Data
"User Location
User location messages, also known as center of mass, are generated when @skeleton is enabled. Once a person is identified, you will begin receiving user location messages. It is not necessary to have full skeletal tracking as indicated by a calib_success message. This enables you to roughly track a person's location when there is interference or because the Kinect was already skeletal tracking its maximum number of people.
OSC Max route
--------------------------------------------------
/user/playerID x y z q user playerID x y z q
- userid is an long integer and is the same unique playerID from the matching user identification message
- x, y, z are floating point numbers representing the real-world location of the person's center of mass relative to the Kinect sensor. Their values are influenced by the attributes @distmeter, @flipx, @rotatemethod, and @translate.
- q is an integer bit field that indicates the "quality" of the person's location. It is possible to have no, one, or multiple quality flags at the same time.
Quality bitflag values:
1 Part of the player's body is out of frame to the camera's right.
2 Part of the player's body is out of frame to the camera's left.
4 Part of the player's body is out of frame above the camera's field of view.
8 Part of the player's body is out of frame below the camera's field of view.
Example: quality = 5 indicates the player's body is out of frame to the camera's right and above.
-----------------
Regarding you second question:
For the skel tracking you have the data with a user id, but you can also just zslice the user id off and you are left with general data. The advantage is that your patch is user agnostic, the disadvantage is that multiple people going in at the same time can throw your patch off. This depends on your situation.
If you need to avoid users getting mixed then you need to add some code that detects which user comes in first and assign that user as your USER1 and free the slot when that userID is lost. All the other ID will be ignored in the mean time....
Hope this helps...
hi mrmaarten,
thanks for your reply! i did read the documentation - actually the detection of this status (user is still recognized but not trackable any more) is not even the problem since if i run dp.kinect2 with @unique 1 i do very easily detect when my problem occurs since the kinect ceases to output data.
actually i can even anticipate this happening by calculating the viewing angle of the viewer and comparing this to the max fov of the kinect.
i am pretty positive i could build some sort of 'translation layer' to all kinect data and eg. treat the output of the kinect in this state as "lost_user" and then treat the reappearance of the user (or more precise the user getting trackable again) as a new "calib_success" but all this (especially if i need to combine this with filtering every output regarding to playerID) will result in a rather messy process which could be completely bypassed by simply being able to tell the kinect when to loose the user... hence the question...
k
ok, i got down to sort this out with what i have and unlike i imagined it turned out to be pretty easy - sorry for making a stir.
if anyone should have similar problems some additional info which might help:
- this 'stalling' actually only occurs with face tracking since the underlying problem is, that the skeleton is still being tracked but not the face.
- the stopping of output/tracking data can be easily detected with the help of the delay object - which, very handily, keeps resetting itself with no output as long as it continuosly recieves bangs. this works so well i now solely use this to detect wether an active user is being tracked or not.
- in terms of separating the users the approach of mrmaarten works very well, zl slice and gate are your friends... (active user comes out of calib_success outlet).
ok... i cheered to early. what i tried above should be simple but to get it to work solidly is almost impossible since the kinect sometimes does not work as expected and as well all sorts of other problems pop up. so i am back at my original question:
is it possible to actively tell the kinect to "loose" the current user (and look for a new one)?
thanks for any help!
k
Hi Karl,
I get now more what you mean!
I don't think you can tell the kinect to loose a user. To make sure you could ask the developer? But also the reason why your patch stalls/ doesn't lose the user make sense: the skeleton is still tracked, it is just the head that is off.
So I do think the way to do it is with a slice and route: make a Designated_User and only send data to it when it is a user and when the head is tracked (which you can see). If you work with multiple users you also have to decide which gets designated.
I was busy with the 'code', I had one version working with a patch but it didn't work 100% well (I wanted to track 2 users). I had the idea it would be easier in Javascript with if statements, but I didn't get around it. I'll post it when it is done....
If anybody has other ideas....
mhm... actually i have met the developer before, i will ask him.
apart: what you suggest is pretty much what i'm currently trying to do - only it does not work too solidly... regardless, here is the code if anyone should be interested:
I'm the dev of dp.kinect2. Deep in the c++ code, I do have additional state information on which I might be able to explore more information/state/feature to Max.
In v1.0 of dp.kinect2 (it was recently released...moving out of the long-term beta) the Max patch receives body presence/tracking and face tracking. Like in the above posts, they could be used in combination to detect cases where the user has moved to the edge of the frame, the body is tracked (at a poorer confidence) and the face is not being tracked.
The state of face tracking is a bit tricky because the Kinect looses face tracking all the time. Even when in the center of the camera, the Kinect looses face tracking. It is usually very brief; maybe 1-2 frames where each is 1/30sec. Because of this, it is not safe to always assume that if face tracking is lost (for 1/30sec) to report it as a lost user.
The Kinect v2 tracks up to six people. The current v2.0 SDK does not allow the selection of *which* six to track. It is automatic.
https://social.msdn.microsoft.com/Forums/en-US/666a50a2-c9a8-4981-ba8e-463ff47f152d/kinect-system-engagement-in-c?forum=kinectv2sdk
What is possible with the already released v1.0 dp.kinect2 to create your own application logic for tracking. The Kinect body tracking will always search for 6. However, the patch doesn't have to react to all 6. Perhaps it only reacts to the body closest to the kinect and in full frame. And if this body moves partially out of the frame or is fully lost, then the patch chooses the next closest body on which to react. Yes, there is a case where a person is partially out of frame and then they move back into the frame. However, this case always exists because users move in/out of the frame constantly and your patch needs to have logic in which to manage this...similar logic could handle the partial-out-of-frame case.
If during this forum thread anyone can distinguish additional data/state that they want dp.kinect2 to provide, please open a new issue at https://github.com/diablodale/dp.kinect2/issues and describe the feature that you want and please include examples. I can better track feature requests and bugs there.
thanks so much for claryfying!
i'm currently trying to work it out in max and slowly i do make some progress, i'll post my result as soon as it's presentable.
k
if anyone should be interested, attached is my "single-user"-logic (update of patch posted above).
it's quite simple and not very thorougly debugged because i have some difficulties testing it since my office mates get tired to play the "test people" but that said, it seems to work ok.
what it does:
- stay with the first tracked user (and only output this user's tracking data) until lost - regardless of other users being found or lost afterwards.
- if the currently tracked user gets lost, select a next one randomly (or more precise: the one with the lowest player id - which is pretty random though).
- output a "kuseractive" variable to know wether the kinect is currently tracking any user (1) or completely idle (0).
since the logic is only based on calib_success and user_lost messages it should work with other uses than face tracking too. apart: the blue subpatcher on the very top is for monitoring purposes only (so you can see whats happening in big on a second screen).
best
k
hi there
is it possible to track lets say the xyz coordinates of two users at the same time ?
@beit, yes. dp.kinect2 and the Kinect v2 sensor can track up to six persons. You can get the general user position, or specific joints. https://github.com/diablodale/dp.kinect2/wiki/Message-based-Data#user-location