Holiday notice: Cycling '74 will be closed on Dec 25th, 26th, and Jan 1st. Manual authorization, support cases, and manual order processing will be delayed.
openpose for Max --- Skeleton Tracking based on Artificial Intelligence on 2D images (no Kinect)
- Sep 02 2017 | 5:22 pmincredible.
- Sep 03 2017 | 11:39 amawesome
- Sep 03 2017 | 4:27 pmthanks a lot for sharing HOLLYHOOK
- Nov 01 2017 | 8:10 pmThanks for sharing ! :) Im really stoked to try it. I would have one question , what do i have to add specificly to the Path environment variable to get the extrenal to access the DLLs ?I get error 126 when trying to load the external.Thanks Again
- Feb 04 2018 | 5:21 pmHow did you manage to use it? Is anyone able to help with the installment of the dependecies for running openPose? Thank you!! I tried following what it says on the github, but it is too much for a noob like me
- Feb 04 2018 | 8:17 pmnever got it to work , the external is broken
- Feb 05 2018 | 9:31 amLook amazing but never got it to work too...
- Feb 05 2018 | 9:46 amSame here, I would love to test but, I get error 126 ...
- Feb 05 2018 | 10:16 amMy guess is that it is not broken. You just have to install all the dependencies. From what I figured from the installation page (https://github.com/CMU-Perceptual-Computing-Lab/openpose/blob/master/doc/installation.md), it says you need to install openCV and Caffe (I don't know why it says Ubuntu only. My guess is that we need to install these as well). Then install CMake GUI and Visual Studio. Andonly after that, you build OpenPose and test it. And if everything works, my guess is that the external will work as well.But that is easier said then done. So I ask again. :) Can someone help us noobs with a for-noobs tutorial on how to install all that? Hollyhook? Please!! At least can you confirm what I wrote above? Thank you so much!! ygrPS: Did you guys manage to install all the dependencies?
- Feb 05 2018 | 7:28 pmThis sounds interesting. I'll give it a try and if I can get it to work, I'll make up a tutorial for how to get it working that will be not so developer-centric ;)
- Feb 05 2018 | 7:58 pmThe openpose developer had a openpose demo, which came with pre-build libraries for openpose and some 3rd party DLLs. The idea was, if you are able to run the demo (with cuda etc.), you can take all the dlls it comes with, put them on the PC somewhere so that windows can find them (in PATH) and the external runs. I can't find the openpose demo anymore, and they also have improved the build system. So i should re-build the external again, on the base of the new openpose... I'm quite busy right now, but will do it as soon as i find time. To be honest, I didn't expect that anyone uses it, because the feedback was kind of thin.. so didn't care about updating.Btw, if anybody would like to try to build it, I'm happy to give the sources of the external away. Btw2, Seen this? https://twitter.com/NataliaNeverova/status/959530358613176320 "code coming soon" \o/ could also be candidate for wrapping into a max external, most likely with the same limitations (PC only, Nvidia, DLL mess)
- Feb 06 2018 | 1:14 amThanks for that link, it looks much more interesting, surpassing Kinect instead of catching up to it.Hopefully it works in realtime, will check the paper.
- Feb 07 2018 | 8:11 amThank you, Hollyhook, for you answer. My first observation regarding you mentioning that there wasn't such great interest in this object is the fact that people don't know about it existing. OpenPose, I mean. BTW, can you modify the title to something like: "OpenPose: Kinect Improved Alternative to Skeleton Tracking based on Artificial Intelligence". And I am adding a tag, Machine Learning. :) This way many maxers will find out about it for sure and interest will grow.Now regarding what you said, maybe you can link a download to the build in case someone wants and knows how to fix it. But hopefully you have a bit of time to look into it and fix it. That would be really great. I can barely wait to try it!I will start looking for that demo now. :) My guess is that maybe if I use the same code as you did, with the same demo, it might work, right? I might have to ask you other things about this later on. :)And your link with the people masking with the DensePose is genius! Hope they manage to put the code out and someone manages to import it in Max as well. :)) Like I said, the only programming language I know is Max. Unfortunately! I already have great ideas for some projects involving this!!! Thank you so much! And again, if anyone manages to make this work, please help the whole community and write all the steps you took in detail. Thank you so much!
- Feb 07 2018 | 9:18 amwaw that's pretty rad ! hopefully the densepose framework can be extended to non windows machines as well (it's a little early to speculate further on that however).
- Feb 08 2018 | 2:39 pmThank you @YGREQ. Looks like i was a little enthusiastic when writing the post. Your suggestion about changing the title is good. Once I get the external to run again, i will change it accordingly. It is a bit of a hens-egg problem. I'm aware of the fact that this is is not easy to setup, and not targeting the "typical max user" (if that is actually existing). So it was for me more an experiment, to find out if it makes sense to release stuff like that at all, and i didn't care much about proper description. With the result that it was too nerdy apparently, and not just me got discouraged.@VICHUG, it is actually not a windows/non-windows issue, it is that the algorithms for Machine Learning can run on many kernels in parallel, so you can get real-time results if you make use of a decent graphics card. "thank you, Gamers" seems to be a running joke in the ML community because graphic cards developed for games are great and affordable. The projects are typically developed with Linux, not windows.Also I want to mention that i think there is so much potential with externals (densepose is only one example) that i don't understand the strategy of cycling74 here. The API for externals has layers of different ages, where you understand the latest only if you dig through down to the older ones, which is way too time consuming. And I would wish for a platform supporting the developers like unity asset store, if you want to release something as a max developer you're really left on your own, no infrastructure, no platform... this way expanding the capabilities of max is so much slower that it could be if there would be more support for the community. Max is loosing ground to vvvv - just my subjective impression is that in germany it already lost, young people don't chose max anymore.
- Feb 11 2018 | 9:19 amI found this page on github that talks about the demo and how to use it: https://github.com/CMU-Perceptual-Computing-Lab/openpose/blob/master/doc/demo_overview.mdIf anyone manages to install, please let us know how exactly. For newbies, please! Thank you!
- Mar 09 2018 | 2:15 pmJust discovered your work on Patreon ; thanks for sharing ! I would really really love to give a try to this external, but even after installing all the libs, and adding the bin folder to the Path environment variable, I still have the same problem (error 126...) (The demo DO work fine on my machine) Please, help us running this amazing piece of code ! Thanks
- Mar 11 2018 | 11:15 amHey Mathieu! Where did you find the demo? Because I didn't. And can you provide us noobs (that don't know python) with a few basic steps? Thank you!
- May 13 2018 | 3:27 pmI re-build the max external with openpose 1.3.0. Anybody would like to test if it works? This is difficult for me, without having access to a second PC. It still requires the openpose windows portable demo as a first step of the installation procedure (https://github.com/CMU-Perceptual-Computing-Lab/openpose/blob/master/doc/installation.md#windows-portable-demo)
- May 14 2018 | 2:47 amThanks so much , will see if I can get it running and let you know . thx again
- May 14 2018 | 6:43 amThank you. The external is for the time being here: http://aiton.de/jit.openpose%20for%20openpose%20v1.3.0.zip Instructions are in the demo patch.hope it works!
- May 19 2018 | 7:15 pmThank you, Hollyhook!! I will try as well. Trying to install all the prerequisites. I cannot find Visual Basic 2015. Do you know if 2017 Community is ok? I am asking because last time I tried installing, some of my apps stopped working. Thank you!
- May 19 2018 | 8:04 pmActually for the windows portable demo, you don't need to compile anything, so no need to install Visual. Admittedly the installation instructions are a bit confusing, so I should make more clear what is needed and what not.The windows portable demo is shortly described here: https://github.com/CMU-Perceptual-Computing-Lab/openpose/blob/master/doc/installation.md#windows-portable-demo. See the download link to the prebuild binaries. Base for the external is the openpose 1.3.0 running on GPU. So you need the binaries openpose-1.3.0-win64-gpu-binaries.zip For the requirements (https://github.com/CMU-Perceptual-Computing-Lab/openpose/blob/master/doc/installation.md#requirements) the section for the NVidia GPU is important.Unfortunately I can't say if the separate installation of CUDA & CuDNN is absolutely necessary or not. The portable demo comes with some cuda dlls, so maybe that is already sufficient. Perhaps you could just try if it just works without installing cuda?last but not least you need to download the COCO model. The openpose portable demo comes with a little script for downloading the right files.Please keep me updated, so I can improve the installation instructions. Thank you!
- May 19 2018 | 10:08 pmSo I installed CMake win64-x64 msi installer, Installed Visual Basic 2017, downloaded openpose-1.3.0-win64-gpu-binariews, went to openpose-1.3.0-win64-gpu-binaries\models\getModels.bat and got the models and everything else automatically. Opened the demo via openpose-1.3.0-win64-gpu-binaries\bin\OpenPoseDemo.exe. It gave me "OpenPoseDemo.exe has stopped responding".I installed Cuda as well. It gives me the same error. Copied the CUDNN files over the Cuda folder like adviced. Same error.hmm. I don't know what to do right now.
- May 19 2018 | 10:16 pmI am trying to figure out what the following means as I just run OpenPoseDemo.exe normally. Should I add something after the .exe? I remember there was this possibility to run exe with extra commands. Is this it?Running on Video
# Ubuntu ./build/examples/openpose/openpose.bin --video examples/media/video.avi # With face and hands ./build/examples/openpose/openpose.bin --video examples/media/video.avi --face --hand:: Windows - Portable Demo bin\OpenPoseDemo.exe --video examples\media\video.avi :: With face and hands bin\OpenPoseDemo.exe --video examples\media\video.avi --face --hand:: Windows - Library build\x64\Release\OpenPoseDemo.exe --video examples\media\video.avi :: With face and hands build\x64\Release\OpenPoseDemo.exe --video examples\media\video.avi --face --hand - May 20 2018 | 8:47 amAh, good you ask. The demo is supposed to be run from the command line, like this
- In windows, press Windows key + R
- In the little dialog that opens, type "cmd" to start the windows command window
- In the command window, type "cd openpose-folder", where openpose-folder is the directory where you extracted the openpose.zip into. On my PC, it is C:\dev\openpose\openpose-1.3.0-win64-gpu-binaries
- Now you can type the entire command like they instructed, e.g. bin\OpenPoseDemo.exe --video examples\media\video.avi to run it on a little demo video that comes with the zip file. Or, if you run omit the parameter, and just run it by typing bin\OpenPoseDemo.exe, it will use the web cam of the PC.
- Adding -- face will enable detection of 70 key points in the faces, --hand is for detecting keypoints at joints of the fingers, both in addition to the pose (body) detection
- May 20 2018 | 11:56 amThank you, Hollyhook. I feel like a child, but I have to say I still have problems. I managed to play the demo with a video.I copy/pasted the folder you gave in the Max 7/Packages folder. I copied all the other dlls from the openpose-1.3.0-win64-gpu-binaries\bin folder to the Max 7\Packages\jit.openpose folder except the dll that was already there. Like you said in the patcher comments.The object jit.openpose.pose does not work. It gives me an error <<Error 126 loading external jit.openpose.pose>> It is not recognized.Am I still missing something? And thank you again for getting me this far.
- May 20 2018 | 12:24 pmI think you are almost there. error 126 means windows cannot find or load the openpose DLLs. You have to tell windows where they are. This can be done by means of an environment variable called PATH. There are guidelines how to do this, e.g. https://stackoverflow.com/questions/44272416/how-to-add-a-folder-to-path-environment-variable-in-windows-10-with-screensho Add the directory name where DLLs are to the path. restart max, and ... ???
- May 20 2018 | 12:56 pmwait a second. OpenPose found the dlls. OpenPose works by itself. I mean the demo. And Max has the dlls copied inside the Max 7 folder.And the question is which one needs to find the dlls? Max or something else?
- May 20 2018 | 12:59 pmIf they are in the same folder than the executable, windows will find the dlls. However, the Max executable is somewhere else (you could in theory move all the dlls into the max exe directory and it should work also)
- May 20 2018 | 2:06 pmYou're a genius!Now how did you manage to put slashes inside of the openpose object? Otherwise, I will back once I figure this out. Again. It's one of those things in Max that I always forget. :)
- May 20 2018 | 2:24 pmI managed to open via opendialog fold..And it works!!!! 17 fps here on a GeForce 1060 with the Tai Chi in People's Park (Shenzhen).mp4 fileUhuuuuu!!! Thank you! Thank you! Thank you!What next?
- May 20 2018 | 2:47 pmFrom my understanding, this only sees the skeleton, right? Without the face tracking.. Am I right?
- May 20 2018 | 6:53 pmHappy it works! yeah, whats next.... First of all, updating the instructions, your questions give me the direction where improvement is needed, so thank you for asking. The face tracking is another external, it is already running here. I decided to split face detection and pose detection into two externals, in order to have best performance for each. Face is tracking 70 keypoints in a face, right now the external runs with 10fps (at least better than the demo). Will have to see if an how to move on, openpose has some more interesting things to offer: hands, 3D reconstruction with multiple cameras, and tracking of persons over frames...
- May 21 2018 | 2:23 pmtotally amazing ! it works here @22 fps with a live video feed, on a GTX1070 :-) thousands thanks !!!looking forward all the "whats next" !!
- May 22 2018 | 11:18 pmWait, I did not understand about the face detection? Is it integrated into the max patch?
- May 23 2018 | 12:17 ammalade! thx hollyhook! you’re a beast
- May 23 2018 | 7:44 pmThanks guys, really appreciate your feedback. @Ygreq, face tracking is not included. Btw: https://storage.googleapis.com/tfjs-models/demos/posenet/camera.html is amazing. With the new tensorflow interface in javascript, access to ML models will be much easier in future, so much will come.
- May 23 2018 | 9:58 pmThanks!
- May 24 2018 | 11:30 am@hollyhook is there a tensorflow object available for max yet?
- May 25 2018 | 8:26 amThere is no max object, but as max is good to connect with other programing languages, one still can use tensorflow. In my little side project deeploops (happy to drop the link here) I use tensorflow via the java API, and a mxj. Recently tensorflow.js was released, so this would probably the better way.Openpose uses custom build caffe instead of tensorflow, and it has no java or js API, therefore the wrapper needs to be an external.
- Jun 03 2018 | 6:40 pmHey Fantastic ! It works here too :-) Thank you very much HOLLYHOOK !
- Jun 03 2018 | 7:14 pmthanks, glad to hear that!I found a bug 🤦 If you swap the red and blue channels like in the picture, the detection quality improves quite a bit.
- Jun 06 2018 | 8:58 amGreat! Thank you, Hollyhook!
- Jul 29 2018 | 11:14 amupdate: external for face detectionit requires the same openpose dlls, means if you have jit.openpose.pose running, this patch works out-of-the-box.
- Jul 30 2018 | 8:26 am:OWill try out soon! Thank you so much, Hollyhook!
- Nov 09 2018 | 7:56 pmHi Hollyhook! It seems I cannot use the patch properly because the IDing keeps on switching. I mean user 1 and user 2 switch between themselves. So for example I would not be able to put a black circle on user 1's face as that circle would switch to user 2's face and back. A bit randomly.BTW, it seems the system does the IDing as such. If one person enters the scene, it is tagged as user 1. If another person enters the scene, it is tagged as user 2 and so on. But if user 1 goes out of the frame, user 2 becomes user 1.Any ideas on how to solve this, dear Hollyhook? Anyone else? Thank you so much!
- Nov 09 2018 | 9:03 pmHi yreg, there is no tracking in openpose v1.3, so IDs can jump between persons anytime. In the latest openpose version there is some experimental single-person tracking, not sure if that would be a help if you wish to detect multiple users... I once created a simple tracking module in js, which works like 90% of the time. I can provide it, if you wish, but while writing the module, i understood that tracking in 2D is really not a simple task For the face detection external, I implemented a filter which is supposed to output the largest face only, determined by comparing the length of the faces in y direction, like distance between eyebrow and chin.
- Nov 23 2018 | 2:25 pmHi Hollyhook! Thank you for your reply. I wasn't able to reply on time because someone spammed this thread and the commenting was blocked. https://cycling74.com/forums/i-am-not-able-to-comment-on-this-thread-does-this-happen-to-you-as-wellI talked to the admins to open it up again. Anyway..That is unfortunate. Like I was saying, that limits the possibilities of OpenPose a lot. May I ask to see the tracking app you mentioned? Thank you!! BTW, is it hard to implement the 1 person detection option?Thank you very much!
- Nov 25 2018 | 5:06 pmAttaching the simple js based person ID tracking. It simply calculates the distance between newly received skeletons to the old (saved) ones, and assigns the new budy to the nearest person ID accordingly - as I said, a 90% solution.The single person tracking in openpose is not released yet, so there is no demo standalone yet, making installation more difficult than it already is. A quick search didn't bring up any release date for openpose 1.5, so right now I'd rather wait. Anybody checked if PoseNet is capable of person tracking?
- Dec 15 2018 | 6:17 am
HI HOLLYHOOK,I have some problem with ID describe too. I use Message to see the array in jit.pose.pose object. However, no matter how many people I detect , I only see the 56 array length in it.( see the pic). In my thought it should be more data when I detect more than one person? I have tried the post you share but I don't familiar with javascript. It seems have two JS in it but I don't know how to replace it to the patch you share in patreon.And I read the stickmen.js , It says include "logging.js" and "body.js" , but I don't know how to include and I cannot find the "logging.js" either.Could you tell me some solution? thank you! - Dec 15 2018 | 7:36 pmHello WU JIAJIUN thanks for your question. The left outlet of jit.openpose.pose sends 56 numbers for each detected person. If there are several person, it sends them very quickly one after another through the same outlet. The fist number is the ID of the person. If you wish to separate the data for each person, you could use the route or routepass object, like shown in this picture:
I hope this already clarifies the issue. If you wish to try out the patch witch person tracking js, let me know and I can upload a variant of the patch with the person ID tracking integrated. - Dec 16 2018 | 12:11 pmPlease do that, Hollyhook. I couldn't use what you already put here unfortunately. This: STICKMEN_WITH_PERSON_TRACKING.ZIP
- Dec 16 2018 | 12:52 pmThank you HOLLYHOOK! It solve my problem. Then I can make visual through using max/msp object! Still, if you share the patch with tracking, I would love to try !
