Skeleton Tracking with PoseNet and Node for Max

Sam Tarakajian's icon

Hi all,

If you're interested in working with skeletal position data, there are a few ways of getting pose estimation data into Max. However, I wanted to point out another one that you might want to try. This one uses PoseNet, which makes use of an neural net trained on human poses to estimate face and skeletal data from video. The working example is in the n4m-examples repository, in the posenet directory.

Even if you've never used Node for Max or github before, it hopefully won't be too hard to use this example to get pose data flowing into Max.

Good luck and happy patching,
-S

maaark's icon

This is so amazinggggg! Thanks Sam!

John Daniel's icon

Hello Sam!
I have about 1000 uses for this - sooooo happy

One problem -
I keep getting a console error when installing dependencies - "NPM exited with non-zero code "1"at /Volumes/Macintosh HD/Users/(me)/documents/n4m-examples-master/posenet"

I am on a slow cellphone hotspot for my internet since the hurricane hit - Am betting the hotpsot is where to place the blame....

The readme inside the archive says "1. Install npm dependencies by clicking the indicated button. Since Electron's kind of big in size, this make take a while depending on your network environment. "

Finally now to the actual question - Will I have to wait til I can find a bigger internet pipe?

The demo I found on the posenet site works fab !
https://storage.googleapis.com/tfjs-models/demos/posenet/camera.html

*Really* looking forward to exploring with this.. : )
cheers
jd

Sam Tarakajian's icon

I am on a slow cellphone hotspot for my internet since the hurricane hit

Ouch! Sorry to hear that. Yeah, this could definitely be the issue. Maybe a request is timing out when trying to download Electron, or possibly the connection is being interrupted. There are a couple of things you could try to get some more information:

1. Check inside the posenet/ folder for any .log files. Maybe there's something in there that can give us a clue what's going on.
2. Try running `npm install` from the command line and see if there's any more information logged there. To do this you'd either need to install npm on your machine, or you'd need to use the npm binary that ships with Node for Max. On OS X your session might look like:

cd path/to/n4m-examples/posenet
/Applications/Max.app/Contents/Resources/C74/packages/Node\ For\ Max/source/bin/npm/bin/npm install

I'd be curious what prints to the console there. Let me know how it goes!

John Daniel's icon

Hey Sam ! thanks so much for your help with this...
Inside the posenet folder there are "shorty" log files that all say this:

*********
npm ERR! Unexpected end of JSON input while parsing near '...-spigot":"~2.0.0"},"d'
npm ERR! A complete log of this run can be found in:
npm ERR! /Users/PubMini2/.npm/_logs/2018-11-28T23_13_09_985Z-debug.log
*********


The complete log file mentioned above ends like this.....
- I am picking it up right before the errors start - Can send you the full log file if you need.
no errors up to this point.
****************
11862 silly saveTree | | | `-- negotiator@0.6.1
11862 silly saveTree | | +-- base64id@1.0.0
11862 silly saveTree | | +-- cookie@0.3.1
11862 silly saveTree | | +-- debug@3.1.0
11862 silly saveTree | | `-- ws@3.3.3
11862 silly saveTree | `-- socket.io-adapter@1.1.1
11862 silly saveTree `-- stats.js@0.17.0
11863 verbose type SyntaxError
11864 verbose stack SyntaxError: Unexpected end of JSON input while parsing near '...-spigot":"~2.0.0"},"d'
11864 verbose stack at JSON.parse (<anonymous>)
11864 verbose stack at parseJson (/Applications/Max.app/Contents/Resources/C74/packages/Node For Max/source/bin/npm/node_modules/json-parse-better-errors/index.js:7:17)
11864 verbose stack at consumeBody.call.then.buffer (/Applications/Max.app/Contents/Resources/C74/packages/Node For Max/source/bin/npm/node_modules/node-fetch-npm/src/body.js:96:50)
11864 verbose stack at <anonymous>
11864 verbose stack at process._tickCallback (internal/process/next_tick.js:188:7)
11865 verbose cwd /Users/PubMini2/Documents/max quicky stuff/n4m-examples-master/posenet
11866 verbose Darwin 17.6.0
11867 verbose argv "/Applications/Max.app/Contents/Resources/C74/packages/Node For Max/source/bin/osx/node" "/Applications/Max.app/Contents/Resources/C74/packages/Node For Max/source/bin/npm/bin/npm-cli.js" "--scripts-prepend-node-path=true" "install"
11868 verbose node v8.11.3
11869 verbose npm v5.6.0
11870 error Unexpected end of JSON input while parsing near '...-spigot":"~2.0.0"},"d'
11871 verbose exit [ 1, true ]

Sam Tarakajian's icon

Weird. Yeah could be a network error---a JSON file doesn't download completely and so can't be parsed correctly. Not sure what to do really other than wait for a better connection.

you might also try npm install from the command line, like I suggested. Worth a shot anyway.

John Daniel's icon

Hey Sam! I tried your suggestion above, here's what came back

Baffling - the Posenet demo web page works perfectly...
thanks much for your help
jd
*******************************
Last login: Wed Dec 12 16:47:29 on ttys000
Mac-mini:~ PubMini2$ cd Documents
Mac-mini:Documents PubMini2$ cd Max\ 8/
Mac-mini:Max 8 PubMini2$ cd Library
Mac-mini:Library PubMini2$ cd n4m-examples
Mac-mini:n4m-examples PubMini2$ cd posenet
Mac-mini:posenet PubMini2$ /Applications/Max.app/Contents/Resources/C74/packages/Node\ For\ Max/source/bin/npm/bin/npm install
internal/modules/cjs/loader.js:605
throw err;
^

Error: Cannot find module '../lib/utils/unsupported.js'
at Function.Module._resolveFilename (internal/modules/cjs/loader.js:603:15)
at Function.Module._load (internal/modules/cjs/loader.js:529:25)
at Module.require (internal/modules/cjs/loader.js:658:17)
at require (internal/modules/cjs/helpers.js:22:18)
at /Applications/Max.app/Contents/Resources/C74/packages/Node For Max/source/bin/npm/bin/node_modules/npm/bin/npm-cli.js:19:21
at Object.<anonymous> (/Applications/Max.app/Contents/Resources/C74/packages/Node For Max/source/bin/npm/bin/node_modules/npm/bin/npm-cli.js:92:3)
at Module._compile (internal/modules/cjs/loader.js:722:30)
at Object.Module._extensions..js (internal/modules/cjs/loader.js:733:10)
at Module.load (internal/modules/cjs/loader.js:620:32)
at tryModuleLoad (internal/modules/cjs/loader.js:560:12)
Mac-mini:posenet PubMini2$

John Daniel's icon

A note to add - i have noticed that when I hit the "install dependencies" in the maxpat,
inside the posenet folder it creates the folder "node_modules/.staging"

Inside of the .staging folder are many many subfolders, then after a few seconds all of the folders delete and the node_module folder is then left empty.

Log generated in the posenet folder is here:

Other projects in the n4m-examples folder seem to work and install dependencies correctly

npm-debug-u113624431.log.zip
application/zip 1.91 KB
zipped logfile


John Daniel's icon

Hey Sam!
It's working now ! : )

I deleted the Max8 app and reinstalled and all is good now. I can get data from the dict and the tracking is pretty good.
I THINK my problem was a bad internet connection at first, further compounded by clueless operator-error flailing with dependencies.


Now that I am up and running (THIS IS SOOO COOL) I have run across one small issue -

******** The issue:
IF I am not connected to internet when running the "main" patch - Start/Stop Posenet only opens up the Electron window with the text "Loading the Model" and does not proceed any further...

IF I am connected to the net, "Loading the model" comes up for a few seconds and then the Posenet window starts working properly. I can get all the pose data from the dict and all is well.

I can now disconnect the internet and close/restart the patch, or Quit/Restart Max and all works perfectly UNTIL the next reboot of my machine - then it starts all over....

***** the question:
I am assuming there is some file/data being brought down from the net that survives a Max restart but not a computer restart... Is there anyway to make this truly "Offline"?

Thanks so much for your help
cheers!
jd

stefano's icon

update...
openpose (and other community examples) has been move to:
https://github.com/Cycling74/n4m-community

Maaike Adema's icon

Hey Sam! I'm trying to download the working example, but the zip file I downloaded from the link doesn't contain a posenet folder. Is there another way to download the correct file? I'm really excited to try this out and it would help me out a lot.

Yuichi Yogo's icon

Hi Maaike, I've moved the posenet repo to a separate repo.
https://github.com/yuichkun/n4m-posenet
You can download the latest version there.

naaman's icon

great stuff here :)

Arthur Parmentier's icon

Hi everyone,

Posenet v2.0 is out but n4m is so outdated that I am unable to update the example to the new tensorflow package. Unfortunately, the max-js-bundler package does not install correctly on my system...

Yoann's icon

I'm trying this great tool. I had some problem, but doing npm install solved it.
I'm interested in the multi-pose mode, but when I select it I get the error "dict.view: extra arguments for message "dictionary"
Is there a way to parse the different poses?

thanks

jack white's icon

why in n4m-examples i can not see node patch with pose net?

Sam Tarakajian's icon

Hey Jack White, this and other examples have been moved to https://github.com/Cycling74/n4m-community

jack white's icon

Sam, thanks to the availability!😊

DarioC's icon

what are you using on MAc OS X Kinect v1 or v2? I just bought a realsense d455 I could use it according to you with these settings

michel8's icon

Hi, any hint to read out multi-pose from the dictionary? The yuichkun/n4m-posenet send multi-pose in sequence with no clear begin and end of the person ID. If there are two person, I can alternate the dict and see two diffrent person. But if sudenly there are three person, I can't keep them apart. I have to know first there are three person detected.
Would be glad to get and split the diffrent tracked persons clearly!
Thanks 8]

michel8's icon

Here an update about multi-pose in dictionary
https://cycling74.com/forums/posenet-into-dict/replies/1#reply-5fd770d2b2a7ee5f9fbcfb79

Simon Rycroft's icon

I have been enjoying using this awesome tool for a few months now using the camera input, but now have a project requiring pose estimation from pre-recorded videos. Is there a way to feed the algorithm with a jitter matrix instead of the camera? Cheers, Simon