AudioMatt

I'd really love to figure out how to make multitouch enabled UI widgets. Obviously fingerpinger is great for performance stuff but as for UI, I'm confused. The data needs to be munged in ways I can't comprehend yet. I seem to be facing a ton of challenges.

Is OSX recognizing my fingers as moving the mouse?

Is X finger accidental input from my clicking thumb or palm?

Has anyone navigated all this stuff? I'd love to do things like a pinch enabled rslider or a six degrees of freedom controller but I continue to run into these sorts of issues.

I'm thinking it may not be worth my time.

I'd really love to figure out how to make multitouch enabled UI widgets.  Obviously fingerpinger is great for performance stuff but as for UI, I'm confused. The data needs to be munged in ways I can't comprehend yet. I seem to be facing a ton of challenges.

Is OSX recognizing my fingers as moving the mouse?
Is OSX scrolling my window?
Is X finger accidental input from my clicking thumb or palm?

Has anyone navigated all this stuff?  I'd love to do things like a pinch enabled rslider or a six degrees of freedom controller but I continue to run into these sorts of issues. 

I'm thinking it may not be worth my time.


multitouch-trackpad-anyone-using-it-for-uis-instead-of-performance

While the data are provided to Max by fingerpinger, they are simultaneously sent to the Finder so yes the mouse pointer moves for nothing and you might also click without wanting to do it actually.

You might use the data to control the user interface if you convert them to TUIO and use MMF.

If you have Snow Leopard, you can disable all trackpad finger gestures, such as two-finger scrolling, four-finger exposé etc etc.

The best way to avoid clicking using the trackpad would be to use aka.mouse to constantly re-position your mouse within a limited area where there is nothing to click. Set it so if you press spacebar once it does that, and if you press it again, the mouse is free to move. Easy-peasy.

I think converting the data to TUIO would limit the possibilities that the Trackpad offers, as TUIO wouldn't be able to communicate data such as angle, and probably wouldn't be as accurate.

The problem with the fingerpinger data (or perhaps the trackpad data) is the index numbers: the reason why you can't have 6-degree freedom (by using 6 fingers on the trackpad) is because there are moments where one of the fingers loses contact for a split second, and all the index numbers of the fingers are re-assigned, making it difficult (impossible?) to trac

I've done a big deal on connecting the trackpad to UI objects such as pictslider, rslider, multislider (where you slide your finger across the trackpad and 'draw' the multisliders), matrices (made a custom virtual trackpad-monome), function (controlling ADSR from trackpad, for example), buttons + toggles which respond to how long or how many times you've pressed a certain area on the trackpad, filtergraph~, dials (depending on angle of finger on trackpad) etc etc. Feel free to get in touch if any of these interest you, and I'll be happy to share. Might actually upload them on the toolbox.

Here's a quick solution to your rslider (you need [fingerpinger] in your max-externals folder, or in the same folder as the patcher):

if you do not use the trackpad as a mouse, it means you must have somewhere a mouse connected, to do basic mouse gestures in order to use your computer, right ? so why not check "ignore trackpad clics when a mouse is connected" in Prefs/universal access ?

hm
hope i'm not out of the subject but...
if you do not use the trackpad as a mouse, it means you must have somewhere a mouse connected, to do basic mouse gestures in order to use your computer, right ? so why not check "ignore trackpad clics when a mouse is connected" in Prefs/universal access ?


Unsurprisingly, I don't think I made myself clear. I'm a bit of a scatter brain. :-) I guess I was thinking about it from the perspective of someone just picking up the software.. I fear that even if I go to the trouble of going to MMF, I won't be able to line it up with apples way of thinking about it.

Let me take laonikoss's example to try and demonstrate what I'm talking about. Obviously I could hook fingerpinger up to rslider. The question then becomes this: How do we get it functioning *well*? Intuitively and ergonomically. If we want this to be useful in a typical UI, it can't be always on. You wouldn't want to permanently hard-wire your pitch knob to your third finger. So how do we get it functioning *well* when the mouse is over it and how do you keep the mouse from leaving it, when you're using it! After 10 seconds of using this, you can see how the average user would want to hurl someone off a bridge. If a righty tries to use it when her fingers on the left side of the track pad, her palm screws up the numbers. Also, if you don't put both of your fingers down at the same time, the mouse moves out of the box when you try and use it. If there are scroll bars in the window and the user has gestures checked, putting your fingers down scrolls in the window.

I'm wondering if, from a design perspective, if it's simply the wrong application for such a thing. Perhaps it's suited *only* to more jeff han sort of stuff.

Unsurprisingly, I don't think I made myself clear.  I'm a bit of a scatter brain.  :-) I guess I was thinking about it from the perspective of someone just picking up the software..  I fear that even if I go to the trouble of going to MMF, I won't be able to line it up with apples way of thinking about it. 

Let me take laonikoss's example to try and demonstrate what I'm talking about.  Obviously I could hook fingerpinger up to rslider. The question then becomes this:  How do we get it functioning *well*?  Intuitively and ergonomically.  If we want this to be useful in a typical UI, it can't be always on.  You wouldn't want to permanently hard-wire your pitch knob to your third finger.   So how do we get it functioning *well* when the mouse is over it and how do you keep the mouse from leaving it, when you're using it!  After 10 seconds of using this, you can see how the average user would want to hurl someone off a bridge.  If a righty tries to use it when her fingers on the left side of the track pad, her palm screws up the numbers.  Also, if you don't put both of your fingers down at the same time, the mouse moves out of the box when you try and use it.  If there are scroll bars in the window and the user has gestures checked,  putting your fingers down scrolls in the window.

I'm wondering if, from a design perspective, if it's simply the wrong application for such a thing.  Perhaps it's suited *only* to more jeff han sort of stuff.


ok I'm getting there.... scrolling is still annoying but progress.

I used a little abstraction I made a while ago which outputs the number of fingers touching the trackpad at any point.

I am not worrying about left/right-handed problems as I am using a magic trackpad. Maybe you should get one, if you're planning on investing on using the trackpad in the future.

btw, although your solution using [zl] to output the maximum vs minimum positions of the fingers is neat and elegant, I prefer my solution as it avoids any problems if at any point you lift one of the fingers just a tiny little bit before the other one. (your solution zeroe's one of the fingers' positions if it's not touching the trackpad, while my solution has it stored as the last value entered).

Ok... ok... I think we're getting somewhere! :-) I shouldn't have been so cynical. you're completey right about the min max thing.

Ok... ok... I think we're getting somewhere! :-)  I shouldn't have been so cynical.  you're completey right about the min max thing.


also, for this particular case, there is a very easy way to stop your cursor from moving when using two fingers:

disable ALL trackpad gestures from System Preferences apart from 2-finger scrolling. (must have snow leopard) - as long as you don't have scrollbars on your patcher, when you use two fingers, your cursor should stay in place.

also, for this particular case, there is a very easy way to stop your cursor from moving when using two fingers: 

disable ALL trackpad gestures from System Preferences apart from 2-finger scrolling. (must have snow leopard) - as long as you don't have scrollbars on your patcher, when you use two fingers, your cursor should stay in place.


> "so why not check "ignore trackpad clics when a mouse is connected" in Prefs/universal access ?"

Year, why not... Just discovering that wow, knowing we are using fingerpinger everyday since a year, we must be really dumb.

Over the last couple of years I've been using the trackpad for creating user interfaces for live manipulation, and I just made a little selection of tools which turn the trackpad input into Max/MSP UI controls (e.g. sliders, waveform selection, etc).

https://cycling74.com/tools/matra-multitouch-trackpad-tools/

Hope this gives a few ideas! (I'm sure a lot of these have been figured out by others too)

You can find the toolbox here: https://cycling74.com/tools/matra-multitouch-trackpad-tools/

Hope this gives a few ideas! (I'm sure a lot of these have been figured out by others too)


You guys might be interested in this, which I just figured out:

https://cycling74.com/forums/maxmsp-multi-touch

Also, in this vein, I've written an abstraction which pretty well maps fingers to a set of "parameters" logically.

It parses fingerpinger multitouch data into 12 “parameters” - eg they can be screen controls - in a slightly more intelligent way - it replaces the identifier/finger/state gibberish with what should be a more meaningful number from 1 to 12.

For instance, if you add or remove a finger, all other active fingers will stay controlling the same parameter, rather than sometimes switching one ID higher or lower as they do in the raw data.

Also, if you lift a finger, then put it back down in the same place without changing too much else, it should pickup controlling the same parameter, even if other fingers have been moved and/or removed since it was last down. Basically when there is a new finger, it first checks if it should be using a previous parameter (one that was recently released at that location). If it doesn't find one, it assigns the new finger to the lowest parameter that is not currently assigned to another active finger. If there is a conflict - eg if a finger is put down in a spot where another finger was recently and had a parameter assigned, but another finger has already been put down elsewhere and usurped that parameter, then the lowest available parameter is assigned to the new finger. This is sort of reset every time all fingers are removed. When reset, fingers are mapped with parameter 1 always being the first finger put down, 2 the 2nd, etc. Sounds complicated, but ends up being pretty intuitive I think. So much so that I never really think about any of this anymore when performing. I'm not sure it works as expected 100% of the time, but it usually does :)

It's called atp.trackpadMapping and you can find it here: 

https://bitbucket.org/arvidtpyh/atp.abstractions/src

I don't think it relies on any 3rd party objects, and should work fine in vanilla Max.

You use it by connecting the two outlets of fingerpinger to the two inlets of atp.trackpadMapping, then using the outlet of atp.trackpadMapping just the same as you would the first outlet of fingerpinger, except that the first two elements ("identifier" and "finger #") in fingerpinger's list output are replaced with a single ("parameter") element which is a an int 1-12. More info is available in comments inside the abstraction.

I'm aware that the patch is probably wildly inefficient, since it uses various colls and tables to do lookups all the time, but it seems to work well enough for me so far and is stable. I use it in up to 4 multitouch patches simultaneously every time I perform. It seems to use less CPU than, for instance, the finger node display in the fingerpinger help patch.

With this abstraction and a few other little tricks for finding out how many fingers are down at any time (see atp.fingersHeldDown in the same repository), i've been able to use fingerpinger for things like manipulating gobs of faders for feedback networks, filtergraph~s with 4 filters at once, controlling synths with MIDI notes/CCs and glitchy sample looping with speed, loop start/end and volume control.

You guys might be interested in this, which I just figured out:
https://cycling74.com/forums/maxmsp-multi-touch

For instance, if you add or remove a finger, all other active fingers will stay controlling the same parameter, rather than sometimes switching one ID higher or lower as they do in the raw data. 

It's called atp.trackpadMapping and you can find it here: https://bitbucket.org/arvidtpyh/atp.abstractions/src

With this abstraction and a few other little tricks for finding out how many fingers are down at any time (see atp.fingersHeldDown in the same repository), i've been able to use fingerpinger for things like manipulating gobs of faders for feedback networks, filtergraph~s with 4 filters at once, controlling synths with MIDI notes/CCs and glitchy sample looping with speed, loop start/end and volume control.


Multitouch Trackpad: Anyone using it for UIs instead of performance?