Machine Learning in Max (Hallelujah)


    Oct 31 2014 | 6:06 pm
    hello max list, it's been years. i write with an announcement and an invitation:
    ANNOUNCEMENT:ml.lib: ml-lib is a library of machine learning externals for Max and Pure Data. ml-lib is primarily based on the Gesture Recognition Toolkit by Nick Gillian ml-lib is designed to work on a variety of platforms including OS X, Windows, Linux, on Intel and ARM architectures.
    in other words: we've ported a range of machine learning algorithms (including svm, mlp, knn, hmm, hmm, dtw, ...) to Max and Pd (including ARM/RPi). these libraries allow you to work with complex mappings in a dramatically more powerful ways than scaling and interpolation approaches.
    you'll find some example videos here: http://artfab.art.cmu.edu/?p=2747
    you'll find an uber powerful example here: http://artfab.art.cmu.edu/?p=2732
    INVITATION: documentation for this library is week; the svm, dtw and mlp help patches have some useful things in them; many others don't. i'm calling out to those who are interested in mapping, appreciate machine learning, and are up for helping us help the max community step up how we deal with "meaningless numbers"--as David Z put it years back--in Max.
    Please use the github issue tracker for specific bug reports or requests. Please don't hesitate to contact me if you have any questions or requests (batchku AT gee mail point com)
    Exciting times ahead.
    UPDATE (June 5, 2015): -we presented a paper on ml.lib at the 2015 New Interfaces for Musical Expression Conference; Check it out for more details -please post all bug reports on our GitHub Issue Tracker -we now have a discussion list for ml.lib here
    Warm regards, ali

    • Nov 01 2014 | 5:13 pm
      Hi, This looks like an exciting library and i want to try it. I downloaded the folder, but I can't seem to find any max externals other than the help files. Are they hidden in some folder that I'm missing?
      thanks, Nick
    • Nov 01 2014 | 5:21 pm
      Download the latest "release" which includes binaries:
    • Nov 02 2014 | 4:08 am
      this is super exciting. won't be able to delve in too deep until the semester is over but after that, it's on.
    • Nov 02 2014 | 8:37 am
      fantastic act of generosity to the max community; opens up a whole new world of possibilities for max users;
    • Nov 02 2014 | 8:43 am
      I have to chime in too; while I also have limited time in which to explore and learn from this, the video of Makoto Ono using this with Duplo blocks is astounding. Great work and thanks for sharing.
    • Nov 02 2014 | 3:06 pm
      yeah this is looking really great.
      Alas, I am absolutely too dumb to use this without parental assistance, hehe.
    • Jan 02 2015 | 1:01 pm
      Finally getting around to looking at this and it seems quite powerful, but being a noob to machine learning/neural network stuff, I have no idea which algorithms to use where.
      For example, if I want to be able to recall gestures made with an x/y analog control on a gamepad, what algorithm should I look at? I had a look at the videos in the link above, but all of what I saw was based around the recognition of static positions, and nothing time based. I'd want to be able to train gestures like (doing a full circle or half circle), which happen in the time domain.
    • Jan 03 2015 | 8:31 am
      hello rodrigo, thanks for looking at these. nick gillian (author of the 'gesture recognition toolkit') which we've ported to max/pd has a very extensive documentation website; look at this hyperlink list of algorithms:
      for each algorithm, he actually lists advantages and disadvantages in very readable language. for example:
      in these pages, he addresses the critical and valid questions of "i have this" and "i want this" pretty well. jaime and i are currently working on a paper on ml.lib intended for submission to the NIME conference which will also pull together additional references and documentation; this should also help.
      in the mean time, for your specific case of "an x/y analog control on a gamepad", you should look at the Dynamic Time Warping algorithm (in our world, it's ml.dtw). luckily it's one of the ones that actually has a decent help file (look in the "test" patcher for an example). it should give you a pretty good classification of various gestures performed if you send your x-y as a 2d feature vector. note that dtw will give you a classification when it has one; as you proceed thru the gesture, that classification may change (two gestures may have similar starts but become more distinguishable later). also, dtw will do a very good job of classifying correctly even if the gesture is performed more quickly or slowly in subsequent takes.
      i hope this helps and give us a shout if it doesn't.
      ali
    • Jan 05 2015 | 3:59 am
      Hi Ali,
      good work and thanks for sharing it. However I tried to install the binaries and Max crashes as soon as I try to declare any ml. object in my patch.
      I'm running OSX 10.7.5 and Max 6.1.8. Is my system too outdated to run your library? If yes which are the minimum system requirements?
      I've tried to re build the binaries but it did not work (mostly extern C include files not found). I tried to dig into it (quickly updating all the header path for my system) but it did not work. On my system I've installed MaxSDK-6.1.1, while your project file uses MaxSDK-6.1.4. Do you think that could be the cause?
      Below the crash report (pretty identical for all ml. objects)
      Cheers
      Process: Max [1377] Path: /Applications/Max 6.1/Max.app/Contents/MacOS/Max Identifier: com.cycling74.Max Version: 6.1.8 [9fafea6] (6.1.8) Code Type: X86 (Native) Parent Process: launchd [254]
      Date/Time: 2015-01-05 11:49:53.993 +0800 OS Version: Mac OS X 10.7.5 (11G63) Report Version: 9
      Interval Since Last Report: 908043 sec Crashes Since Last Report: 29 Per-App Interval Since Last Report: 262077 sec Per-App Crashes Since Last Report: 13 Anonymous UUID: C60E9FF5-8D9B-45E7-A7FB-A92D9A45B2A8
      Crashed Thread: 0 Dispatch queue: com.apple.main-thread
      Exception Type: EXC_BAD_ACCESS (SIGSEGV) Exception Codes: KERN_INVALID_ADDRESS at 0x000000005723d2e0
      VM Regions Near 0x5723d2e0: __LINKEDIT 000000000fe98000-000000000fef1000 [ 356K] r--/rwx SM=COW /Applications/Max 6.1/*/*.dtw --> __TEXT 000000008fe26000-000000008fe59000 [ 204K] r-x/rwx SM=COW /usr/lib/dyld
      Application Specific Information: objc[1377]: garbage collection is OFF
      Thread 0 Crashed:: Dispatch queue: com.apple.main-thread 0 com.cycling74.Max 0x00034400 c74_debug_malloc_footer_test + 131 1 com.cycling74.Max 0x00034461 sysmem_freeptr_imp + 54 2 com.cycling74.MaxAPI 0x00b7e6b0 sysmem_freeptr + 32 3 edu.cmu.ml.dtw 0x0fe2a690 flext_root_single_d::operator delete(void*) + 208 4 edu.cmu.ml.dtw 0x0fe05245 operator delete(void*) + 21 5 edu.cmu.ml.dtw 0x0fe58f23 GRT::Log::setProceedingText(std::__1::basic_string, std::__1::allocator > const&) + 99 6 edu.cmu.ml.dtw 0x0fe66f5b GRT::DebugLog::DebugLog(std::__1::basic_string, std::__1::allocator >) + 155 7 edu.cmu.ml.dtw 0x0fe7150f GRT::UnlabelledData::UnlabelledData(unsigned int, std::__1::basic_string, std::__1::allocator >, std::__1::basic_string, std::__1::allocator >) + 207 8 edu.cmu.ml.dtw 0x0fe0ccb0 ml::ml::ml() + 448 9 edu.cmu.ml.dtw 0x0fe01627 ml::ml_classification::ml_classification() + 39 10 edu.cmu.ml.dtw 0x0fe0c5a7 ml::ml_dtw::ml_dtw() + 39 11 edu.cmu.ml.dtw 0x0fe0b867 ml::ml_dtw::ml_dtw() + 23 12 edu.cmu.ml.dtw 0x0fe0b533 ml::ml_dtw::__init__(int, atom*) + 51 13 edu.cmu.ml.dtw 0x0fe1ced5 flext_obj_single_d::obj_new(symbol const*, short, atom*) + 1829 14 com.cycling74.Max 0x0001f9ea typedmess_fun + 70 15 com.cycling74.Max 0x000caf22 object_new_typed_flags + 399 16 com.cycling74.Max 0x000caf5c object_new_typed + 46 17 com.cycling74.Max 0x0000db99 auxtable_search + 128 18 com.cycling74.Max 0x0000f454 newload_internal + 102 19 com.cycling74.Max 0x0000f85c newload + 47 20 com.cycling74.Max 0x0001f9ea typedmess_fun + 70 21 com.cycling74.Max 0x0001fcc1 typedmess + 68 22 com.cycling74.Max 0x00020af5 aeval + 1437 23 com.cycling74.Max 0x00004d31 atombuf_eval + 138 24 com.cycling74.Max 0x0018e1eb jnewobj_new + 2633 25 com.cycling74.Max 0x0001f9ea typedmess_fun + 70 26 com.cycling74.Max 0x000caed1 object_new_typed_flags + 318 27 com.cycling74.Max 0x000caf5c object_new_typed + 46 28 com.cycling74.Max 0x001acde1 jpatcher_newobject + 394 29 com.cycling74.Max 0x001af553 jpatcher_pastefromdictionary + 805 30 com.cycling74.Max 0x001afd01 jpatcher_replace + 1593 31 com.cycling74.Max 0x000c89e8 object_method_typedfun + 132 32 com.cycling74.Max 0x000c8b5b object_method_typed + 177 33 com.cycling74.Max 0x00134f92 jbox_swapme + 125 34 com.cycling74.Max 0x0018d4bb jnewobj_enter + 1530 35 com.cycling74.Max 0x000c8fd7 object_method_imp + 821 36 com.cycling74.Max 0x000c9270 object_method + 143 37 com.cycling74.Max 0x001c1108 jpatchercontroller_entertext_imp + 249 38 com.cycling74.Max 0x001c12ab jpatchercontroller_entertext + 213 39 com.cycling74.Max 0x0011c12b TextfieldComponent::endEdit() + 407 40 com.cycling74.Max 0x0011c2fc BoxComponent::endEdit() + 112 41 com.cycling74.Max 0x0011df57 TextfieldTextEditor::keyPressed(juce::KeyPress const&) + 949 42 com.cycling74.Max 0x003e06db juce::ComponentPeer::handleKeyPress(int, wchar_t, bool, int) + 323 43 com.cycling74.Max 0x0046b971 juce::NSViewComponentPeer::handleKeyEvent(NSEvent*, bool) + 461 44 com.cycling74.Max 0x0046bacf juce::NSViewComponentPeer::redirectKeyDown(NSEvent*) + 57 45 com.cycling74.Max 0x004657cf -[JuceNSView_1_52_105_3 keyDown:] + 157 46 com.apple.AppKit 0x9af40aeb -[NSWindow sendEvent:] + 10891 47 com.apple.AppKit 0x9aed8f77 -[NSApplication sendEvent:] + 4788 48 com.apple.AppKit 0x9ae6ab21 -[NSApplication run] + 1007 49 com.cycling74.Max 0x004689e7 juce::MessageManager::runDispatchLoop() + 731 50 com.cycling74.Max 0x0032a8e4 juce::JUCEApplication::main(juce::StringArray const&) + 68 51 com.cycling74.Max 0x0032aa07 juce::JUCEApplication::main(int, char const**) + 73 52 com.cycling74.Max 0x002525a6 main + 486 53 com.cycling74.Max 0x000025c9 _start + 208 54 com.cycling74.Max 0x000024f8 start + 40
    • Jan 05 2015 | 5:24 pm
      Hello
      On Monday, January 5, 2015, Cycling 74 wrote: Stefano Fasciani wrote:
      Hi Ali,
      good work and thanks for sharing it. However I tried to install the binaries and Max crashes as soon as I try to declare any ml. object in my patch.
      I'm running OSX 10.7.5 and Max 6.1.8. Is my system too outdated to run your library?
      Likely.
      If yes which are the minimum system requirements?
      They have only been tested on 10.8 and above.
      I would recommend 10.10
      I've tried to re build the binaries but it did not work (mostly extern C include files not found). I tried to dig into it (quickly updating all the header path for my system) but it did not work. On my system I've installed MaxSDK-6.1.1, while your project file uses MaxSDK-6.1.4. Do you think that could be the cause?
      Likely. Also, you need flext.
      I have tested the objects with Max 7 as well; they load but I noticed something wonky in the performance; I only tested ml.svm and very briefly; I noticed that while it worked, it slowed down the GUI terribly. Disconcerting, but as max 7 is so fresh, we haven't looked into it further.
      Ali
    • Jan 06 2015 | 10:37 am
      The crashlog indicates that the GRT lib is doing something to trash memory, apparently in the DebugLog or Log constructor while calling setProceedingText() [sic]. How exactly flext's delete operator is being called is a little hard to determine, though, so the callstack might not be accurate.
      The videos are very exciting!
      Jeremy
    • Jan 08 2015 | 6:34 pm
      Someone fwd this library to Autechre. I want to hear what they can do with it.
    • Mar 11 2015 | 5:28 am
      Hey guys, congrats on the release. It looks really awesome! I've not had any problems so far, except a little bit of trouble with dtw (0.16.0-alpha, Max7, OSX 10.10.1).
      It seems the maxhelp file is giving slightly misleading instructions.
      maxhelp suggests: 0. clear 1. Record on 2. Sample ONE 3. Sample TWO 4. Train 5. Record OFF 6. Test ONE 7. Test TWO
      That doesn't work. What does work is: 0. Clear 1. Record Gesture ONE: * Record ON * Sample ONE * Record OFF 2. Record Gesture TWO: * Record ON * Sample TWO * Record OFF 3. Train 4. Test Gesture ONE: * Record ON * Test ONE * Record OFF 5. Test Gesture TWO: * Record ON * Test TWO * Record OFF
      When I do the above, it detects the different signals correctly, which is great. I tried to train it with phone accelerometer (3d) signals, e.g. rotating clockwise vs counterclockwise around various different axis. And that works too - but only if I start the 'record' action immediately before the gesture and 'stop' it immediately after. Which I can kind of understand why (I guess it needs to know exactly which buffer / sequence to look at and map). But that makes it a bit impractical for gesture detection in the long run. Is there an intelligent way to automagically start and stop the record so that I can keep doing gestures and it will keep detecting them without me manually start/stopping the record? (e.g. I could use a timer, but that isn't very smart. Or Maybe if it detects a gesture with a very high confidence, or has extremely low confidence, it could reset?)
    • Jun 04 2015 | 7:19 pm
      Hi there!
      Noticed you just got a paper published on this work at NIME 2015 - congrats, that's awesome. Just thought you might like to know that your externals are already being used at the core of a project my company started which is a VR flying simulator called Airflow. We're using the SVM to take in software joint data from an incomplete IMU-based full body motion capture control rig whose payload is being piped into Max as a blob of OSC floats.
      Check it out: https://www.youtube.com/watch?v=KJyNSaZMWNI // More info available on my website herehttps://www.youtube.com/watch?v=KJyNSaZMWNI // More info available on my website here.
      I was also wondering if you could give me an ETA on a Windows build for the Max externals? The rig running this ride is currently three computers, two Windows machines and a Mac. The only reason we're still using the Mac is because the Windows versions of ml-lib don't exist yet... we could collapse it down to just one VR rendering machine and one machine that does everything else, once that's in place.
      I'm thinking of giving it a stab myself, building against the Max 6 SDK in VS Express, over the next couple of weeks. I'll fork and give you a link if that happens. Since we're all Max users here, I know that things happen in opposite land, so new additions get built in OS X first and then Windows much later... so I get it if it's not a priority until you reach 1.0.
    • Jun 04 2015 | 7:29 pm
      Hello there, thank you for this wonderful reference. the paper was presented and is now on the NIME site: https://nime2015.lsu.edu/proceedings/201/0201-paper.pdf
      We'll be putting a small site together to keep track of projects made with ml.lib for others to see.
      thanks for sharing!
      ali
    • Jun 05 2015 | 5:17 am
      That's cool, thanks for including our project (eventually)!
      I edited my post quite a lot after you read it I think, can you get back to me re: Windows etc?
      Thanks :)
    • Jun 08 2015 | 1:32 pm
      Is it me or are all pages (except the github) down?
      I'm able to find this page via home to the artfab homepage, but there's next to nothing on it. http://artfab.art.cmu.edu/?page_id=2520
    • Jun 08 2015 | 3:19 pm
      Hello Rodrigo, I just tried the github link on the artfab page and it works fine for me: https://github.com/cmuartfab/ml-lib
      the latest binaries are here: https://github.com/cmuartfab/ml-lib/releases
      i updated the original post in this thread to also include links to the paper just presented at NIME 2015 on this topic.
      please feel free to contact me off the forum if you'd like more help.
      ali
    • Jun 08 2015 | 3:20 pm
      The github stuff works, but I was trying to look at the project page and example videos (http://artfab.art.cmu.edu/ml-lib/).
    • Jun 08 2015 | 3:23 pm
      sorry, it's a wordpress permalink issue, i'll fix it!
      here's the real link in the mean time: http://artfab.art.cmu.edu/?p=2747
    • Mar 02 2016 | 11:38 pm
      Hi, Do you know how to build it vor windows 32 and 64 ? Do you plan a release? thanks
    • Mar 18 2016 | 3:21 am
      For whatever it's worth, nickgillian.com is full of broken links. the overview page that you refer to above, http://www.nickgillian.com/software/grt#MachineLearningAlgorithms, returns a concrete5 'unable to connect to database' error, as do all the documentation pages for all the algorithms....
      \M
    • Mar 18 2016 | 11:26 am
      It has already been asked but what about windows externals? Very interessting project, would really like to try some things.
    • Mar 18 2016 | 12:24 pm
      hello all, we have not yet make windows externals, but we would love your help in doing so; neither of us work with a windows development environment regularly. we are especially keen to get them out because we're moving towards including mol.lib in the max package manager and that requires cross-platform-support. any takers?
    • Mar 18 2016 | 4:04 pm
      cycling74 making the windows port? Would be great if cycling could kind of 'officialize' some of those great packages. :)
    • Mar 24 2016 | 11:08 pm
      I don't have the skills to help you out with that, Batchku, but I would like to say that I do greatly desire it!
    • Apr 01 2016 | 1:22 am
      On second thought, what would it take to create the Windows version? If the only problem is you don't have a computer running Windows and all I would have to do is compile something that's already written, well, if you could walk me through it I would gladly do it.
    • Apr 05 2016 | 1:01 am
      Hi, Thank you for your generosity with your time and effort Batchku! Your work seems very useful for the work im doing right now with the kinect, but im on windows... Im pretty noob at coding but ill give it a shot in helping ! Or pay a good dinner to the altruist programmer.
    • Aug 25 2016 | 2:50 am
      please port to windows... pretty please :)
    • Feb 19 2017 | 4:11 pm
      Hey,
      Thank you for developing this library - it is very super useful for my school-project!
      I have a question about ml.svm - if I would like to use two of these objects (I am using two iPhones - for both hands), how can I train them so, that they would understand different mapping. I mean one object and mapping for one hand and another for other hand. Is there any way to give sort of specific name for ml.svm object? Because right now I can't map them separately in one patch even if I have two of these objects. I thought that if they get data from different ports and iPhones, then they don't share data with each other?
      Best, Kristin
    • Feb 22 2017 | 11:08 pm
      Hello Kristin,
      ml.svm should be able to handle the application you describe. There is no memory sharing between multiple instances of the object so if you have to instances in your patch, you should be abel to train and use them separately.
      Perhaps you could share your patch with me so I can take a look?
      Regards,
      ali
    • Feb 24 2017 | 1:38 am
      Hello,
      I am having a multitude of problems and error codes trying to compile ml-lib for Max on Mac using Xcode. I think the PureData externals were able to compile, but not the ones for Max (which is what I need). I keep getting the error:
      /Users/Richard/Desktop/Source Code/ml-lib-master/sources/ClassificationModules/BAG/BAG.h:52:2: Expected member name or ';' after declaration specifiers
      screen shot 2017-02-20 at 6 07 56 pm [Attached]
      I'm using latest version of Xcode with Max and PureData installed. Am I missing a library or using the wrong settings to compile?
    • Feb 24 2017 | 2:46 pm
      Hello Richard, My collaborator Jamie will be looking into updating the ml-lib objects to use the latest Max SDK and GRT. In the mean time, are the binaries posted in the release section of the github not working for you?
      Ali
    • Feb 28 2017 | 4:54 am
      Ali,
      I found the release page and the current objects work great! I just wanted to start with going over the code for objects I wanted to use then compile them and make changes later if necessary. Excited to start putting them to work!
    • Feb 28 2017 | 8:35 pm
      "Machine learning is like money laundering for bias."
      an acquaintance heard it from Wendy Hui Kyong Chun, but doesn't know if it originates with her. Pretty snappy, huh?
    • Mar 20 2017 | 5:58 pm
      Hi there . Is it 32 bit only? I cant seem to know how to make it work under windows 10 64 bit
    • Mar 28 2017 | 10:20 pm
      Hey everyone,
      since I put some time into the Myo + Max/MSP I compiled a data overview patch and a slimmed down version of myo-osc from samyk: https://github.com/xhain/Myo2Max
      I hope this helps anyone struggling with an initial setup, please let me know if anything isn't working as it should.
      M.
    • Oct 11 2017 | 6:39 pm
      @BATCHKU running ml-0.17 on 10.12.6 ml-svm.maxhelp crashes as soon i activate the prob option. crash is happening every time. Does know about this & know a fix ? thanks!
    • Dec 13 2017 | 11:23 pm
      I cannot find any binaries in any of the releases.
    • May 31 2019 | 7:45 pm
      Hello everyone who has followed the ml.lib project since the early days! We are delighted to announce the latest release of ml-lib. More info on the release can be found here.