Skeleton Tracking with PoseNet and Node for Max


    Nov 23 2018 | 7:41 pm
    Hi all,
    If you're interested in working with skeletal position data, there are a few ways of getting pose estimation data into Max. However, I wanted to point out another one that you might want to try. This one uses PoseNet, which makes use of an neural net trained on human poses to estimate face and skeletal data from video. The working example is in the n4m-examples repository, in the posenet directory.
    Even if you've never used Node for Max or github before, it hopefully won't be too hard to use this example to get pose data flowing into Max.
    Good luck and happy patching, -S

    • Nov 26 2018 | 3:16 pm
      This is so amazinggggg! Thanks Sam!
    • Nov 27 2018 | 4:53 pm
      Hello Sam! I have about 1000 uses for this - sooooo happy One problem - I keep getting a console error when installing dependencies - "NPM exited with non-zero code "1"at /Volumes/Macintosh HD/Users/(me)/documents/n4m-examples-master/posenet"
      I am on a slow cellphone hotspot for my internet since the hurricane hit - Am betting the hotpsot is where to place the blame....
      The readme inside the archive says "1. Install npm dependencies by clicking the indicated button. Since Electron's kind of big in size, this make take a while depending on your network environment. "
      Finally now to the actual question - Will I have to wait til I can find a bigger internet pipe? The demo I found on the posenet site works fab ! https://storage.googleapis.com/tfjs-models/demos/posenet/camera.html
      *Really* looking forward to exploring with this.. : ) cheers jd
    • Nov 28 2018 | 9:09 pm
      I am on a slow cellphone hotspot for my internet since the hurricane hit
      Ouch! Sorry to hear that. Yeah, this could definitely be the issue. Maybe a request is timing out when trying to download Electron, or possibly the connection is being interrupted. There are a couple of things you could try to get some more information:
      1. Check inside the posenet/ folder for any .log files. Maybe there's something in there that can give us a clue what's going on. 2. Try running `npm install` from the command line and see if there's any more information logged there. To do this you'd either need to install npm on your machine, or you'd need to use the npm binary that ships with Node for Max. On OS X your session might look like:
      cd path/to/n4m-examples/posenet
      /Applications/Max.app/Contents/Resources/C74/packages/Node\ For\ Max/source/bin/npm/bin/npm install
      I'd be curious what prints to the console there. Let me know how it goes!
    • Nov 28 2018 | 11:27 pm
      Hey Sam ! thanks so much for your help with this... Inside the posenet folder there are "shorty" log files that all say this:
      ********* npm ERR! Unexpected end of JSON input while parsing near '...-spigot":"~2.0.0"},"d' npm ERR! A complete log of this run can be found in: npm ERR! /Users/PubMini2/.npm/_logs/2018-11-28T23_13_09_985Z-debug.log *********
      The complete log file mentioned above ends like this..... - I am picking it up right before the errors start - Can send you the full log file if you need. no errors up to this point. **************** 11862 silly saveTree | | | `-- negotiator@0.6.1 11862 silly saveTree | | +-- base64id@1.0.0 11862 silly saveTree | | +-- cookie@0.3.1 11862 silly saveTree | | +-- debug@3.1.0 11862 silly saveTree | | `-- ws@3.3.3 11862 silly saveTree | `-- socket.io-adapter@1.1.1 11862 silly saveTree `-- stats.js@0.17.0 11863 verbose type SyntaxError 11864 verbose stack SyntaxError: Unexpected end of JSON input while parsing near '...-spigot":"~2.0.0"},"d' 11864 verbose stack at JSON.parse (<anonymous>) 11864 verbose stack at parseJson (/Applications/Max.app/Contents/Resources/C74/packages/Node For Max/source/bin/npm/node_modules/json-parse-better-errors/index.js:7:17) 11864 verbose stack at consumeBody.call.then.buffer (/Applications/Max.app/Contents/Resources/C74/packages/Node For Max/source/bin/npm/node_modules/node-fetch-npm/src/body.js:96:50) 11864 verbose stack at <anonymous> 11864 verbose stack at process._tickCallback (internal/process/next_tick.js:188:7) 11865 verbose cwd /Users/PubMini2/Documents/max quicky stuff/n4m-examples-master/posenet 11866 verbose Darwin 17.6.0 11867 verbose argv "/Applications/Max.app/Contents/Resources/C74/packages/Node For Max/source/bin/osx/node" "/Applications/Max.app/Contents/Resources/C74/packages/Node For Max/source/bin/npm/bin/npm-cli.js" "--scripts-prepend-node-path=true" "install" 11868 verbose node v8.11.3 11869 verbose npm v5.6.0 11870 error Unexpected end of JSON input while parsing near '...-spigot":"~2.0.0"},"d' 11871 verbose exit [ 1, true ]
    • Nov 29 2018 | 7:09 pm
      Weird. Yeah could be a network error---a JSON file doesn't download completely and so can't be parsed correctly. Not sure what to do really other than wait for a better connection.
      you might also try npm install from the command line, like I suggested. Worth a shot anyway.
    • Dec 12 2018 | 10:52 pm
      Hey Sam! I tried your suggestion above, here's what came back Baffling - the Posenet demo web page works perfectly... thanks much for your help jd ******************************* Last login: Wed Dec 12 16:47:29 on ttys000 Mac-mini:~ PubMini2$ cd Documents Mac-mini:Documents PubMini2$ cd Max\ 8/ Mac-mini:Max 8 PubMini2$ cd Library Mac-mini:Library PubMini2$ cd n4m-examples Mac-mini:n4m-examples PubMini2$ cd posenet Mac-mini:posenet PubMini2$ /Applications/Max.app/Contents/Resources/C74/packages/Node\ For\ Max/source/bin/npm/bin/npm install internal/modules/cjs/loader.js:605 throw err; ^
      Error: Cannot find module '../lib/utils/unsupported.js' at Function.Module._resolveFilename (internal/modules/cjs/loader.js:603:15) at Function.Module._load (internal/modules/cjs/loader.js:529:25) at Module.require (internal/modules/cjs/loader.js:658:17) at require (internal/modules/cjs/helpers.js:22:18) at /Applications/Max.app/Contents/Resources/C74/packages/Node For Max/source/bin/npm/bin/node_modules/npm/bin/npm-cli.js:19:21 at Object.<anonymous> (/Applications/Max.app/Contents/Resources/C74/packages/Node For Max/source/bin/npm/bin/node_modules/npm/bin/npm-cli.js:92:3) at Module._compile (internal/modules/cjs/loader.js:722:30) at Object.Module._extensions..js (internal/modules/cjs/loader.js:733:10) at Module.load (internal/modules/cjs/loader.js:620:32) at tryModuleLoad (internal/modules/cjs/loader.js:560:12) Mac-mini:posenet PubMini2$
    • Dec 13 2018 | 10:12 pm
      A note to add - i have noticed that when I hit the "install dependencies" in the maxpat, inside the posenet folder it creates the folder "node_modules/.staging"
      Inside of the .staging folder are many many subfolders, then after a few seconds all of the folders delete and the node_module folder is then left empty. Log generated in the posenet folder is here:
      Other projects in the n4m-examples folder seem to work and install dependencies correctly
    • Jan 09 2019 | 1:14 am
      Hey Sam! It's working now ! : )
      I deleted the Max8 app and reinstalled and all is good now. I can get data from the dict and the tracking is pretty good. I THINK my problem was a bad internet connection at first, further compounded by clueless operator-error flailing with dependencies.
      Now that I am up and running (THIS IS SOOO COOL) I have run across one small issue - ******** The issue: IF I am not connected to internet when running the "main" patch - Start/Stop Posenet only opens up the Electron window with the text "Loading the Model" and does not proceed any further...
      IF I am connected to the net, "Loading the model" comes up for a few seconds and then the Posenet window starts working properly. I can get all the pose data from the dict and all is well.
      I can now disconnect the internet and close/restart the patch, or Quit/Restart Max and all works perfectly UNTIL the next reboot of my machine - then it starts all over.... ***** the question: I am assuming there is some file/data being brought down from the net that survives a Max restart but not a computer restart... Is there anyway to make this truly "Offline"?
      Thanks so much for your help cheers! jd
    • Jan 27 2019 | 7:43 am
      update... openpose (and other community examples) has been move to: https://github.com/Cycling74/n4m-community
    • May 08 2019 | 10:38 am
      Hey Sam! I'm trying to download the working example, but the zip file I downloaded from the link doesn't contain a posenet folder. Is there another way to download the correct file? I'm really excited to try this out and it would help me out a lot.
    • Jun 22 2019 | 11:48 am
      Hi Maaike, I've moved the posenet repo to a separate repo. https://github.com/yuichkun/n4m-posenet You can download the latest version there.
    • Dec 26 2019 | 6:45 am
      great stuff here :)
    • Feb 28 2020 | 3:38 pm
      Hi everyone, Posenet v2.0 is out but n4m is so outdated that I am unable to update the example to the new tensorflow package. Unfortunately, the max-js-bundler package does not install correctly on my system...
    • May 17 2020 | 11:51 am
      I'm trying this great tool. I had some problem, but doing npm install solved it. I'm interested in the multi-pose mode, but when I select it I get the error "dict.view: extra arguments for message "dictionary" Is there a way to parse the different poses?
      thanks
    • Jun 04 2020 | 11:00 am
      why in n4m-examples i can not see node patch with pose net?
    • Jun 04 2020 | 11:38 am
      Hey Jack White, this and other examples have been moved to https://github.com/Cycling74/n4m-community
    • Jun 04 2020 | 3:25 pm
      Sam, thanks to the availability!😊
    • Oct 12 2020 | 6:25 pm
      what are you using on MAc OS X Kinect v1 or v2? I just bought a realsense d455 I could use it according to you with these settings
    • Oct 17 2020 | 7:26 pm
      Hi, any hint to read out multi-pose from the dictionary? The yuichkun/n4m-posenet send multi-pose in sequence with no clear begin and end of the person ID. If there are two person, I can alternate the dict and see two diffrent person. But if sudenly there are three person, I can't keep them apart. I have to know first there are three person detected. Would be glad to get and split the diffrent tracked persons clearly! Thanks 8]