Machine Learning in Max (Hallelujah)
hello max list,
it's been years.
i write with an announcement and an invitation:
ANNOUNCEMENT:ml.lib:
ml-lib is a library of machine learning externals for Max and Pure Data. ml-lib is primarily based on the Gesture Recognition Toolkit by Nick Gillian ml-lib is designed to work on a variety of platforms including OS X, Windows, Linux, on Intel and ARM architectures.
in other words: we've ported a range of machine learning algorithms (including svm, mlp, knn, hmm, hmm, dtw, ...) to Max and Pd (including ARM/RPi). these libraries allow you to work with complex mappings in a dramatically more powerful ways than scaling and interpolation approaches.
you'll find some example videos here:
http://artfab.art.cmu.edu/?p=2747
you'll find an uber powerful example here:
http://artfab.art.cmu.edu/?p=2732
INVITATION:
documentation for this library is week; the svm, dtw and mlp help patches have some useful things in them; many others don't. i'm calling out to those who are interested in mapping, appreciate machine learning, and are up for helping us help the max community step up how we deal with "meaningless numbers"--as David Z put it years back--in Max.
Please use the github issue tracker for specific bug reports or requests. Please don't hesitate to contact me if you have any questions or requests (batchku AT gee mail point com)
Exciting times ahead.
UPDATE (June 5, 2015):
-we presented a paper on ml.lib at the 2015 New Interfaces for Musical Expression Conference; Check it out for more details
-please post all bug reports on our GitHub Issue Tracker
-we now have a discussion list for ml.lib here
Warm regards,
ali
Hi, This looks like an exciting library and i want to try it. I downloaded the folder, but I can't seem to find any max externals other than the help files. Are they hidden in some folder that I'm missing?
thanks, Nick
Download the latest "release" which includes binaries:
this is super exciting. won't be able to delve in too deep until the semester is over but after that, it's on.
fantastic act of generosity to the max community; opens up a whole new world of possibilities for max users;
I have to chime in too; while I also have limited time in which to explore and learn from this, the video of Makoto Ono using this with Duplo blocks is astounding. Great work and thanks for sharing.
yeah this is looking really great.
Alas, I am absolutely too dumb to use this without parental assistance, hehe.
Finally getting around to looking at this and it seems quite powerful, but being a noob to machine learning/neural network stuff, I have no idea which algorithms to use where.
For example, if I want to be able to recall gestures made with an x/y analog control on a gamepad, what algorithm should I look at? I had a look at the videos in the link above, but all of what I saw was based around the recognition of static positions, and nothing time based. I'd want to be able to train gestures like (doing a full circle or half circle), which happen in the time domain.
hello rodrigo,
thanks for looking at these.
nick gillian (author of the 'gesture recognition toolkit') which we've ported to max/pd has a very extensive documentation website; look at this hyperlink list of algorithms:
for each algorithm, he actually lists advantages and disadvantages in very readable language. for example:
in these pages, he addresses the critical and valid questions of "i have this" and "i want this" pretty well. jaime and i are currently working on a paper on ml.lib intended for submission to the NIME conference which will also pull together additional references and documentation; this should also help.
in the mean time, for your specific case of "an x/y analog control on a gamepad", you should look at the Dynamic Time Warping algorithm (in our world, it's ml.dtw). luckily it's one of the ones that actually has a decent help file (look in the "test" patcher for an example). it should give you a pretty good classification of various gestures performed if you send your x-y as a 2d feature vector. note that dtw will give you a classification when it has one; as you proceed thru the gesture, that classification may change (two gestures may have similar starts but become more distinguishable later). also, dtw will do a very good job of classifying correctly even if the gesture is performed more quickly or slowly in subsequent takes.
i hope this helps and give us a shout if it doesn't.
ali
Hi Ali,
good work and thanks for sharing it.
However I tried to install the binaries and Max crashes as soon as I try to declare any ml. object in my patch.
I'm running OSX 10.7.5 and Max 6.1.8. Is my system too outdated to run your library? If yes which are the minimum system requirements?
I've tried to re build the binaries but it did not work (mostly extern C include files not found). I tried to dig into it (quickly updating all the header path for my system) but it did not work. On my system I've installed MaxSDK-6.1.1, while your project file uses MaxSDK-6.1.4. Do you think that could be the cause?
Below the crash report (pretty identical for all ml. objects)
Cheers
Process: Max [1377]
Path: /Applications/Max 6.1/Max.app/Contents/MacOS/Max
Identifier: com.cycling74.Max
Version: 6.1.8 [9fafea6] (6.1.8)
Code Type: X86 (Native)
Parent Process: launchd [254]
Date/Time: 2015-01-05 11:49:53.993 +0800
OS Version: Mac OS X 10.7.5 (11G63)
Report Version: 9
Interval Since Last Report: 908043 sec
Crashes Since Last Report: 29
Per-App Interval Since Last Report: 262077 sec
Per-App Crashes Since Last Report: 13
Anonymous UUID: C60E9FF5-8D9B-45E7-A7FB-A92D9A45B2A8
Crashed Thread: 0 Dispatch queue: com.apple.main-thread
Exception Type: EXC_BAD_ACCESS (SIGSEGV)
Exception Codes: KERN_INVALID_ADDRESS at 0x000000005723d2e0
VM Regions Near 0x5723d2e0:
__LINKEDIT 000000000fe98000-000000000fef1000 [ 356K] r--/rwx SM=COW /Applications/Max 6.1/*/*.dtw
-->
__TEXT 000000008fe26000-000000008fe59000 [ 204K] r-x/rwx SM=COW /usr/lib/dyld
Application Specific Information:
objc[1377]: garbage collection is OFF
Thread 0 Crashed:: Dispatch queue: com.apple.main-thread
0 com.cycling74.Max 0x00034400 c74_debug_malloc_footer_test + 131
1 com.cycling74.Max 0x00034461 sysmem_freeptr_imp + 54
2 com.cycling74.MaxAPI 0x00b7e6b0 sysmem_freeptr + 32
3 edu.cmu.ml.dtw 0x0fe2a690 flext_root_single_d::operator delete(void*) + 208
4 edu.cmu.ml.dtw 0x0fe05245 operator delete(void*) + 21
5 edu.cmu.ml.dtw 0x0fe58f23 GRT::Log::setProceedingText(std::__1::basic_string, std::__1::allocator > const&) + 99
6 edu.cmu.ml.dtw 0x0fe66f5b GRT::DebugLog::DebugLog(std::__1::basic_string, std::__1::allocator >) + 155
7 edu.cmu.ml.dtw 0x0fe7150f GRT::UnlabelledData::UnlabelledData(unsigned int, std::__1::basic_string, std::__1::allocator >, std::__1::basic_string, std::__1::allocator >) + 207
8 edu.cmu.ml.dtw 0x0fe0ccb0 ml::ml::ml() + 448
9 edu.cmu.ml.dtw 0x0fe01627 ml::ml_classification::ml_classification() + 39
10 edu.cmu.ml.dtw 0x0fe0c5a7 ml::ml_dtw::ml_dtw() + 39
11 edu.cmu.ml.dtw 0x0fe0b867 ml::ml_dtw::ml_dtw() + 23
12 edu.cmu.ml.dtw 0x0fe0b533 ml::ml_dtw::__init__(int, atom*) + 51
13 edu.cmu.ml.dtw 0x0fe1ced5 flext_obj_single_d::obj_new(symbol const*, short, atom*) + 1829
14 com.cycling74.Max 0x0001f9ea typedmess_fun + 70
15 com.cycling74.Max 0x000caf22 object_new_typed_flags + 399
16 com.cycling74.Max 0x000caf5c object_new_typed + 46
17 com.cycling74.Max 0x0000db99 auxtable_search + 128
18 com.cycling74.Max 0x0000f454 newload_internal + 102
19 com.cycling74.Max 0x0000f85c newload + 47
20 com.cycling74.Max 0x0001f9ea typedmess_fun + 70
21 com.cycling74.Max 0x0001fcc1 typedmess + 68
22 com.cycling74.Max 0x00020af5 aeval + 1437
23 com.cycling74.Max 0x00004d31 atombuf_eval + 138
24 com.cycling74.Max 0x0018e1eb jnewobj_new + 2633
25 com.cycling74.Max 0x0001f9ea typedmess_fun + 70
26 com.cycling74.Max 0x000caed1 object_new_typed_flags + 318
27 com.cycling74.Max 0x000caf5c object_new_typed + 46
28 com.cycling74.Max 0x001acde1 jpatcher_newobject + 394
29 com.cycling74.Max 0x001af553 jpatcher_pastefromdictionary + 805
30 com.cycling74.Max 0x001afd01 jpatcher_replace + 1593
31 com.cycling74.Max 0x000c89e8 object_method_typedfun + 132
32 com.cycling74.Max 0x000c8b5b object_method_typed + 177
33 com.cycling74.Max 0x00134f92 jbox_swapme + 125
34 com.cycling74.Max 0x0018d4bb jnewobj_enter + 1530
35 com.cycling74.Max 0x000c8fd7 object_method_imp + 821
36 com.cycling74.Max 0x000c9270 object_method + 143
37 com.cycling74.Max 0x001c1108 jpatchercontroller_entertext_imp + 249
38 com.cycling74.Max 0x001c12ab jpatchercontroller_entertext + 213
39 com.cycling74.Max 0x0011c12b TextfieldComponent::endEdit() + 407
40 com.cycling74.Max 0x0011c2fc BoxComponent::endEdit() + 112
41 com.cycling74.Max 0x0011df57 TextfieldTextEditor::keyPressed(juce::KeyPress const&) + 949
42 com.cycling74.Max 0x003e06db juce::ComponentPeer::handleKeyPress(int, wchar_t, bool, int) + 323
43 com.cycling74.Max 0x0046b971 juce::NSViewComponentPeer::handleKeyEvent(NSEvent*, bool) + 461
44 com.cycling74.Max 0x0046bacf juce::NSViewComponentPeer::redirectKeyDown(NSEvent*) + 57
45 com.cycling74.Max 0x004657cf -[JuceNSView_1_52_105_3 keyDown:] + 157
46 com.apple.AppKit 0x9af40aeb -[NSWindow sendEvent:] + 10891
47 com.apple.AppKit 0x9aed8f77 -[NSApplication sendEvent:] + 4788
48 com.apple.AppKit 0x9ae6ab21 -[NSApplication run] + 1007
49 com.cycling74.Max 0x004689e7 juce::MessageManager::runDispatchLoop() + 731
50 com.cycling74.Max 0x0032a8e4 juce::JUCEApplication::main(juce::StringArray const&) + 68
51 com.cycling74.Max 0x0032aa07 juce::JUCEApplication::main(int, char const**) + 73
52 com.cycling74.Max 0x002525a6 main + 486
53 com.cycling74.Max 0x000025c9 _start + 208
54 com.cycling74.Max 0x000024f8 start + 40
Hello
On Monday, January 5, 2015, Cycling 74 wrote:
Stefano Fasciani wrote:
Hi Ali,
good work and thanks for sharing it.
However I tried to install the binaries and Max crashes as soon as I try to declare any ml. object in my patch.
I'm running OSX 10.7.5 and Max 6.1.8. Is my system too outdated to run your library?
Likely.
If yes which are the minimum system requirements?
They have only been tested on 10.8 and above.
I would recommend 10.10
I've tried to re build the binaries but it did not work (mostly extern C include files not found). I tried to dig into it (quickly updating all the header path for my system) but it did not work. On my system I've installed MaxSDK-6.1.1, while your project file uses MaxSDK-6.1.4. Do you think that could be the cause?
Likely.
Also, you need flext.
I have tested the objects with Max 7 as well; they load but I noticed something wonky in the performance; I only tested ml.svm and very briefly; I noticed that while it worked, it slowed down the GUI terribly. Disconcerting, but as max 7 is so fresh, we haven't looked into it further.
Ali
The crashlog indicates that the GRT lib is doing something to trash memory, apparently in the DebugLog or Log constructor while calling setProceedingText() [sic]. How exactly flext's delete operator is being called is a little hard to determine, though, so the callstack might not be accurate.
The videos are very exciting!
Jeremy
Someone fwd this library to Autechre. I want to hear what they can do with it.
Hey guys, congrats on the release. It looks really awesome! I've not had any problems so far, except a little bit of trouble with dtw (0.16.0-alpha, Max7, OSX 10.10.1).
It seems the maxhelp file is giving slightly misleading instructions.
maxhelp suggests:
0. clear
1. Record on
2. Sample ONE
3. Sample TWO
4. Train
5. Record OFF
6. Test ONE
7. Test TWO
That doesn't work. What does work is:
0. Clear
1. Record Gesture ONE:
* Record ON
* Sample ONE
* Record OFF
2. Record Gesture TWO:
* Record ON
* Sample TWO
* Record OFF
3. Train
4. Test Gesture ONE:
* Record ON
* Test ONE
* Record OFF
5. Test Gesture TWO:
* Record ON
* Test TWO
* Record OFF
When I do the above, it detects the different signals correctly, which is great. I tried to train it with phone accelerometer (3d) signals, e.g. rotating clockwise vs counterclockwise around various different axis. And that works too - but only if I start the 'record' action immediately before the gesture and 'stop' it immediately after. Which I can kind of understand why (I guess it needs to know exactly which buffer / sequence to look at and map). But that makes it a bit impractical for gesture detection in the long run. Is there an intelligent way to automagically start and stop the record so that I can keep doing gestures and it will keep detecting them without me manually start/stopping the record? (e.g. I could use a timer, but that isn't very smart. Or Maybe if it detects a gesture with a very high confidence, or has extremely low confidence, it could reset?)
Hi there!
Noticed you just got a paper published on this work at NIME 2015 - congrats, that's awesome.
Just thought you might like to know that your externals are already being used at the core of a project my company started which is a VR flying simulator called Airflow. We're using the SVM to take in software joint data from an incomplete IMU-based full body motion capture control rig whose payload is being piped into Max as a blob of OSC floats.
Check it out:
https://www.youtube.com/watch?v=KJyNSaZMWNI // More info available on my website here.
I was also wondering if you could give me an ETA on a Windows build for the Max externals? The rig running this ride is currently three computers, two Windows machines and a Mac. The only reason we're still using the Mac is because the Windows versions of ml-lib don't exist yet... we could collapse it down to just one VR rendering machine and one machine that does everything else, once that's in place.
I'm thinking of giving it a stab myself, building against the Max 6 SDK in VS Express, over the next couple of weeks. I'll fork and give you a link if that happens. Since we're all Max users here, I know that things happen in opposite land, so new additions get built in OS X first and then Windows much later... so I get it if it's not a priority until you reach 1.0.
Hello there,
thank you for this wonderful reference.
the paper was presented and is now on the NIME site: https://nime2015.lsu.edu/proceedings/201/0201-paper.pdf
We'll be putting a small site together to keep track of projects made with ml.lib for others to see.
thanks for sharing!
ali
That's cool, thanks for including our project (eventually)!
I edited my post quite a lot after you read it I think, can you get back to me re: Windows etc?
Thanks :)
Is it me or are all pages (except the github) down?
I'm able to find this page via home to the artfab homepage, but there's next to nothing on it.
http://artfab.art.cmu.edu/?page_id=2520
Hello Rodrigo,
I just tried the github link on the artfab page and it works fine for me:
https://github.com/cmuartfab/ml-lib
the latest binaries are here:
https://github.com/cmuartfab/ml-lib/releases
i updated the original post in this thread to also include links to the paper just presented at NIME 2015 on this topic.
please feel free to contact me off the forum if you'd like more help.
ali
The github stuff works, but I was trying to look at the project page and example videos (http://artfab.art.cmu.edu/ml-lib/).
sorry, it's a wordpress permalink issue, i'll fix it!
here's the real link in the mean time:
http://artfab.art.cmu.edu/?p=2747
Hi,
Do you know how to build it vor windows 32 and 64 ?
Do you plan a release?
thanks
For whatever it's worth, nickgillian.com is full of broken links. the overview page that you refer to above, http://www.nickgillian.com/software/grt#MachineLearningAlgorithms, returns a concrete5 'unable to connect to database' error, as do all the documentation pages for all the algorithms....
\M
It has already been asked but what about windows externals? Very interessting project, would really like to try some things.
hello all,
we have not yet make windows externals, but we would love your help in doing so; neither of us work with a windows development environment regularly. we are especially keen to get them out because we're moving towards including mol.lib in the max package manager and that requires cross-platform-support. any takers?
cycling74 making the windows port?
Would be great if cycling could kind of 'officialize' some of those great packages.
:)
I don't have the skills to help you out with that, Batchku, but I would like to say that I do greatly desire it!
On second thought, what would it take to create the Windows version? If the only problem is you don't have a computer running Windows and all I would have to do is compile something that's already written, well, if you could walk me through it I would gladly do it.
Hi,
Thank you for your generosity with your time and effort Batchku!
Your work seems very useful for the work im doing right now with the kinect, but im on windows...
Im pretty noob at coding but ill give it a shot in helping ! Or pay a good dinner to the altruist programmer.
please port to windows... pretty please :)
Hey,
Thank you for developing this library - it is very super useful for my school-project!
I have a question about ml.svm - if I would like to use two of these objects (I am using two iPhones - for both hands), how can I train them so, that they would understand different mapping. I mean one object and mapping for one hand and another for other hand. Is there any way to give sort of specific name for ml.svm object? Because right now I can't map them separately in one patch even if I have two of these objects.
I thought that if they get data from different ports and iPhones, then they don't share data with each other?
Best,
Kristin
Hello Kristin,
ml.svm should be able to handle the application you describe. There is no memory sharing between multiple instances of the object so if you have to instances in your patch, you should be abel to train and use them separately.
Perhaps you could share your patch with me so I can take a look?
Regards,
ali
Hello,
I am having a multitude of problems and error codes trying to compile ml-lib for Max on Mac using Xcode. I think the PureData externals were able to compile, but not the ones for Max (which is what I need). I keep getting the error:
/Users/Richard/Desktop/Source Code/ml-lib-master/sources/ClassificationModules/BAG/BAG.h:52:2: Expected member name or ';' after declaration specifiers
screen shot 2017-02-20 at 6 07 56 pm [Attached]
I'm using latest version of Xcode with Max and PureData installed. Am I missing a library or using the wrong settings to compile?
Hello Richard,
My collaborator Jamie will be looking into updating the ml-lib objects to use the latest Max SDK and GRT.
In the mean time, are the binaries posted in the release section of the github not working for you?
Ali
Ali,
I found the release page and the current objects work great! I just wanted to start with going over the code for objects I wanted to use then compile them and make changes later if necessary. Excited to start putting them to work!
"Machine learning is like money laundering for bias."
an acquaintance heard it from Wendy Hui Kyong Chun, but doesn't know if it originates with her. Pretty snappy, huh?
Hi there . Is it 32 bit only?
I cant seem to know how to make it work under windows 10 64 bit
Hey everyone,
since I put some time into the Myo + Max/MSP I compiled a data overview patch and a slimmed down version of myo-osc from samyk:
https://github.com/xhain/Myo2Max
I hope this helps anyone struggling with an initial setup, please let me know if anything isn't working as it should.
M.
@BATCHKU running ml-0.17 on 10.12.6
ml-svm.maxhelp crashes as soon i activate the prob option.
crash is happening every time. Does know about this & know a fix ? thanks!
I cannot find any binaries in any of the releases.
Hello everyone who has followed the ml.lib project since the early days! We are delighted to announce the latest release of ml-lib. More info on the release can be found here.
I just noticed the http://www.nickgillian.com website is down. Does anyone know what happened?
For the sake of posterity for the forum, the information seems to have been moved here:
https://github.com/nickgillian/grt/wiki
Unfortunately, the package is no longer maintained and it no longer runs on a current Mac. What a pity.
Is there no other external for dynamic time warping?
Flucoma is a great package and there is also an experimental external for dtw, but unfortunately it doesn't work properly for me and it doesn't seem to be developed any further:
https://discourse.flucoma.org/t/new-object-early-community-build-4-lstm-and-updates-on-dataseries-and-dtw-max-sc/2224
I think the authors of flucoma have since moved on to other projects, for example FrameLib. FrameLib can be used to create nice dynamic time warping, but will require more time to get yoru head around, i guess.
There are also other numerous solutions, which are more or less maintained. Ircam externals can be a good tool, there is a nice granulator in the spat suite, and lots to be done with supervp and MuBu possibly. (For the record, i can still use Flucoma, but i'm on macos Ventura so idk for more recent OS).
FluCoMa is still actively being developed, just much more slowly since it's no longer institutionally funded (currently at v1.08, whereas the funded release was 1.0 or 1.01 if I remember right).
The DTW and LSTM will be incorporated, but as above, somewhat slowly.
Framelib is very powerful, but completely separate from FluCoMa (developed by Alex Harker) as it existed before FluCoMa and is built around a different paradigm (frame-based processing).