Alexandre

I plan to use my 5 fingers moving on my ipod touch(or iphone) to control sound in max, while moving this ipod upside down in 3 directions, using the accelerometers to control the timber of my instruments, depending on the angles that the ipod will make to the ground. nice thoughts.. So I'm planing to use one of the 3 nice young iphone apps that are able to send the free multitouch coordinates from your 5 fingers simultaneously: MSA Remote, Fantastick, or OSCemote.

Well, until now, I'm only starting by testing the accelerometers data from these 3 apps (+ also from TouchOSC). One could think that they're all the same, but from one to another... it's like day and night, in term of sampling speed of accelerometers data and vertical resolution... The sound is only made from two brutal saw~, but at least you get the funny impression that your iphone is a living animal complaining when you move it. The best rendering, for now, is made with MSA Remote.

- Shows graphically data entry in max with 1 millisec precision in a scrolling lcd. (this was made at the beginning to test wacom data entry), and vertical precision in an other lcd.

- Works with the 4 iphones apps, you only have to set in its settings for each app : -> Output host : your IPv4 address that you will find in your computer settings, -> output port : 3333 , no Input used in this patch. (except for Fantastick, which is a bit more complicated, see below)

++ PRECISE vertical resolution (at least 10 bits)

Ok, true that the accelerometer of the ipod is not so precise, but, if you use my patch while simply putting the ipod on a table, face down, you'll really hear more trembling sound with Fantastick than with MSA Remote. (and see it in the "Data_from_ipod_apps" window)

-> big problems of data transmission : big lags when graphics are moving in max... i could finally resolve this by using a standalone for the data transmission !!? (see below explanation)

-- When moving the ipod suddenly, the latency seems bigger to me than with MSA Remote.(but it's hard to make an objective test for this)

+ With Fantastick you can draw what you want from max. (but my first impression is that the drawing is performing really slow)

-- Samples every... 35 Millisec. (and less when the ipod shows the acceleration view !)

(weird: always sending same data 2 or 3 times)

-- other problem: multitouch is not in full screen on the ipod

++ i really like the useful design of the multitouch mode, with x&y lines, it looks more clear to me than the design of MSA Remote...

About the data transmission of Fantastick :

The data from Fantastick is not of the same format than the 3 other apps. In the example patch from jusu (the conceptor of Fantastick) he is using java objects net.udp.recv and net.udp.send to manage data. But when you use some speed moving graphics in max (for example in my patch), you get samples only every 50 Millisec., or less... and lags... I don't know where does this come from, i suspect strange java priority behaviors. I'd really like to know if there is a way too change these priority behaviors(I'm using java a lot), in order to listen to the object net.udp.recv as often as it can be with udpreceive, and then drawing in max in REAL LOW priority. Of course i tried to use udpreceive but i get some error messages : "OSC packet size (30) not a multiple of 4 bytes: dropping". i also tried with some OSC behaviors ([udpreceive 6661 symbol]), but then i feel lost with the [opensoundcontrol] object...)

So finally, i found a workaround by putting the java in/out stuff in a standalone, and then communicate between the standalone and the patch with udp max objects. I'm sure there must be a better solution, any idea ?

But as a standalone app is too big for this forum (2Mb maximum), you'll have to open "fantastick.maxpat", not in the MaxMSP application, but in the 'other' max application: the MaxMSP Runtime... then find settings of Fantastick down in the ipod settings, put your computer IPv4, out-Port:6661 and in-Port:6662, and then open the Fantastick app.)

- Sometimes you have to restart the ipod app if it's not working.

- There is a precise position of the ipod in space where the patch is quiet but i don't tell you, let's find it..

- Using a standard internet router network, brings, in my case, some lag on the data every 10 seconds or less. Then i put a computer-to-computer network to avoid this.

(Also, except for TouchOSC, the ipod is going to sleep after 1 minute without touching the screen, unless you deactivate sleep-mode in the ipod settings. should be nice to get an option)

Also, before thinking which app is bad or good, note that they're all very young apps which may be subject to updates.

* ( by deduction : knowing extreme values from the accelerometer are -2.32 to +2.30,and seeing that, around zero, values are: -0.018169,0,+0.018169 -> then you get (2.30+2.32)/0.018169 => 256 = 2^8 )

I plan to use my 5 fingers moving on my ipod touch(or iphone) to control sound in max, while moving this ipod upside down in 3 directions, using the accelerometers to control the timber of my instruments, depending on the angles that the ipod will make to the ground. nice thoughts.. So I'm planing to use one of the 3 nice young iphone apps that are able to send the free multitouch coordinates from your 5 fingers simultaneously: MSA Remote, Fantastick, or OSCemote. 

- Works with the 4 iphones apps, you only have to set in its settings for each app : -> Output host : your IPv4 address that you will find in your computer settings, -> output port : 3333 , no Input used in this patch. (except for Fantastick, which is a bit more complicated, see below) 

++ SAMPLES every 17 Millisec. (mean)  
(but with one of 34 ms every 12 sample)
++ PRECISE vertical resolution (at least 10 bits) 

++ Samples every 16 Millisec. (mean)  
-- but resolution of... 8 bits only* !
Ok, true that the accelerometer of the ipod is not so precise, but, if you use my patch while simply putting the ipod on a table, face down, you'll really hear more trembling sound with Fantastick than with MSA Remote. (and see it in the "Data_from_ipod_apps" window)
-> big problems of data transmission : big lags when graphics are moving in max... i could finally resolve this by using a standalone for the data transmission !!? (see below explanation)
-- When moving the ipod suddenly, the latency seems bigger to me than with MSA Remote.(but it's hard to make an objective test for this)
+ With Fantastick you can draw what you want from max. (but my first impression is that the drawing is performing really slow)

-- Samples every... 31 Millisec.   
-- 8 bits only*.

-- Samples every... 35 Millisec. (and less when the ipod shows the acceleration view !)
-- 8 bits only*.
(weird: always sending same data 2 or 3 times)
-- other problem: multitouch is not in full screen on the ipod
++ i really like the useful design of the multitouch mode, with x&y lines, it looks more clear to me than the design of MSA Remote... 

Also, before thinking which app is bad or good, note that they're all very young apps which may be subject to updates. 

* ( by deduction : knowing extreme values from the accelerometer are -2.32 to +2.30,and seeing that, around zero, values are: -0.018169,0,+0.018169 -> then you get (2.30+2.32)/0.018169 => 256 = 2^8 )


big-test-patch-for-iphone-apps-msa-remote-fantastick-touchosc-oscemote

thanks for posting this. i am interested in this topic as well and testing out the different apps as well. have you tried MRMR ?

you can build your own interface with it. unfortunatelly i find it pretty slow in response. 

i found osctouch and msa to be the more responsive, although osctouch has no multitouch and i also haveto restart the iphone from time to time specially when i switch from one of these apps to another.

does anybody know if MRMR is still being developped ? it seemed very promising to me.

Alexandre wrote on Sun, 05 July 2009 01:48

thanks for posting this. i am interested in this topic as well and testing out the different apps as well. have you tried MRMR ?
you can build your own interface with it. unfortunatelly i find it pretty slow in response. 
i havent tried fantastik yet.
i found osctouch and msa to be the more responsive, although osctouch has no multitouch and i also haveto restart the iphone from time to time specially when i switch from one of these apps to another.
does anybody know if MRMR is still being developped ? it seemed very promising to me.

Alexandre wrote on Sun, 05 July 2009 01:48Hi,

TouchOSC does have multitouch, although its "polyphony" is five.

It also now has an editor for creating your own interfaces.

Thanks Nick, i hadnt upgraded TouchOsc for a while and went on their website to discover the interface editor indeed

nick rothwell / cassiel wrote on Mon, 06 July 2009 16:25

Thanks Nick, i hadnt upgraded TouchOsc for a while and went on their website to discover the interface editor indeed 

nick rothwell / cassiel wrote on Mon, 06 July 2009 16:25TouchOSC does have multitouch, although its "polyphony" is five.

they both don't handle "free xy sampling of the positions of the 5 fingers simultaneously", like MSA Remote, Fantastick and OSCemote do. As i'd like to try many things like to control dynamically and organically the harmonic spectrum of my instrument with my 5 fingers, this was my reason for testing these 3 last apps first.

That said, This new interface editor for TouchOSC is really nice and so easy to use !

..And the "5 fingers xy sampling" is missing for it to look exactly like a lemur.. imagine you gather six ipod (120$ on ebay for a 1st generation ipod) together on a piece of wood with TouchOSC running on each, then you have a lemur-like interface for a third of the price.

That said, the data transmission of the xy pad in touchOSC is less efficient than capturing accelerometer data which was already a bit slow for me... (a lot slower than xy data from MSA Remote or Fantastick)

Let's say i understand that the user who is making usual cycles-based-electronic-music may not, a big part of the time, care if the sampling gap of his knobs moving is 50 ms or 5 ms, simply because theses knobs react on mostly shorts rhythmic sounds. But when you like to act in real time on the coloration of some long sounds, then you will get lots of artifacts with gap of 50 ms... (ok, you can still smooth the data with line~ but then you will get an additional 50 ms delay... not a musicians' dream)

Tomorrow i will post the second part of the test: Comparing sampling rates of capture of the fingers positions.

they both don't handle "free xy sampling of the positions of the 5 fingers simultaneously", like MSA Remote, Fantastick and OSCemote do. As i'd like to try many things like to control dynamically and organically the harmonic spectrum of my instrument with my 5 fingers, this was my reason for testing these 3 last apps first. 

Tomorrow i will post the second part of the test: Comparing sampling rates of capture of the fingers positions.


- FS (and I suppose all programs) give out accelerometer

data "raw" as the device reports it, so resolution is what

- Definitely use the OpenGL drawing with Fantastick, it's

- FS lags: "big lags when graphics are moving in max" i suppose

this has to do with scheduling, priorities etc in Max,

I would experiment with "Overdrive" setting and other

performance settings found in Max such as screen refresh etc.

- FS (and I suppose all programs) give out accelerometer
data "raw" as the device reports it, so resolution is what
apple provides

- Definitely use the OpenGL drawing with Fantastick, it's
fast and responsive, not slow at all

- FS lags: "big lags when graphics are moving in max" i suppose
this has to do with scheduling, priorities etc in Max,
I would experiment with "Overdrive" setting and other
performance settings found in Max such as screen refresh etc.

-> big problems of data transmission : big lags when graphics are moving in max... i could finally resolve this by using a standalone for the data transmission !!? (see below explanation)
-- When moving the ipod suddenly, the latency seems bigger to me than with MSA Remote.(but it's hard to make an objective test for this)
+ With Fantastick you can draw what you want from max. (but my first impression is that the drawing is performing really slow)


> FS (and I suppose all programs) give out 

> accelerometer data "raw" as the device reports it, 

Then why MSA Remote gives more accurate resolution than your app ?

See the picture shot in attachment (same ipod / same scale).

I'd really like to have this fine resolution in Fantastick because i'd like to use it instead of MSA Remote. (Of course i prefer to draw my stuff!...)

My following test for fingers moves (i'm going to post them soon here) seem to show that Fantastick and MSA Remote are equivalent for fingers moves sampling speed, and better than OSCemote and TouchOSC. (well until now, my tests were without drawing anything in Fantastick...)

> FS lags: ... i suppose this has to do with scheduling,

Theses big big boring lags - which, i repeat, only come with Fantastick, not the 3 other apps - have nothing to do with "Overdrive" (which by the way must be ON with my patch), or with other max settings. To me, This looks like a low priority problem in the java class UdpReceiver(), which is in the net.udp.recv object, which is in your example patch. I think you shouldn't use that object to receive data from your app. You should use udpreceive, but then i don't get what are these errors that comes from your app in the max window ("OSC packet size not a multiple of 4 bytes")... ?? => Personally, I just found a better workaround since my "standalone stuff" in my first msg here : [aka.datagram] !! (an external from Masayuki Akamatsu, 

I don't get why it only work with [aka.datagram] an not with [OpenSoundControl] but i just know it works, the new patch working with Fantastick is in attachment, no more lag at all with it, please have a look.

> Definitely use the OpenGL drawing with Fantastick, 

> it's fast and responsive, not slow at all

True the OpenGL mode in Fantastick is needed to use it. It's seems ok to use and looks as fast as TouchOSC and the 2 others apps... (But well, i wouldn't call it "fast".. ok the Ipod processor is not a CoreDuo, but so many games are moving so fast on it, then why these four OSC app are so slow to draw xy position lines ?)

> FS (and I suppose all programs) give out 
> accelerometer data "raw" as the device reports it, 
> so resolution is what apple provides

Then why MSA Remote gives more accurate resolution than your app ?
See the picture shot in attachment (same ipod / same scale).

> FS lags: ...  i suppose this has to do with scheduling,
> priorities etc in Max,

Theses big big boring lags - which, i repeat, only come with Fantastick, not the 3 other apps - have nothing to do with "Overdrive" (which by the way must be ON with my patch), or with other max settings. To me, This looks like a low priority problem in the java class UdpReceiver(), which is in the net.udp.recv object, which is in your example patch. I think you shouldn't use that object to receive data from your app. You should use udpreceive, but then i don't get what are these errors that comes from your app in the max window ("OSC packet size  not a multiple of 4 bytes")... ??  => Personally, I just found a better workaround since my "standalone stuff" in my first msg here : [aka.datagram] !! (an external from Masayuki Akamatsu, http://www.iamas.ac.jp/~aka/max/ )
I don't get why it only work with [aka.datagram] an not with [OpenSoundControl] but i just know it works, the new patch working with Fantastick is in attachment, no more lag at all with it, please have a look. 

> Definitely use the OpenGL drawing with Fantastick, 
> it's fast and responsive, not slow at all

The public accelerometer api has a property for sampling interval, that's all. Two options, either MSA Remote interpolates to smoothen out the data or I've missed an undocumented way to somehow get better data.

Thanks for the informative discovery with net.udp.recv and aka.datagram!

Oh, the [OpenSoundControl] error: Fs doesn't use OSC. The data is ascii text transmitted over UDP. Remember the tagline, 0% OSC content! Yeah!

> either MSA Remote interpolates to smoothen out the data or I've missed an undocumented way to somehow get better data.

> Thanks for the informative discovery with net.udp.recv and aka.datagram!

You probably didn't noticed anything in your example patches because your were not afflicting max, but when you start to, you really often can get problems about priorities when using different threads while receiving data (max & java, or max & external) As an example, J.M.Couturier, the programmer of the Wacom max object, had lots of difficulties to find out how to manage a good priority in max for the USB Wacom data... And as I was complaining about irresolvable lags on wacom data in max while using jitter, he wrote to me the only right way to manage this : To put his wacom object, alone, in a standalone, sending data to max5 thru udp...

What is strange then is that i can only get it to work using the "osc mode" of udpreceive object (with a symbol as attribute) before sending it to aka.datagram. All this network stuff is still mysterious to me. If i put the normal udpreceive object without this symbol attribute, i get errors in the max window. Do you know why ?

newbie-in-osc question : In what is it good to not use OSC content ?

What is strange then is that i can only get it to work using the "osc mode" of udpreceive object (with a symbol as attribute) before sending it to aka.datagram. All this network stuff is still mysterious to me. If i put the normal udpreceive object without this symbol attribute, i get errors in the max window. Do you know why ? 

newbie-in-osc question : In what is it good to not use OSC content ?


>> either MSA Remote interpolates to smoothen out the data

>> or I've missed an undocumented way to somehow get better data.

> because your were not afflicting max, but when you start to,

> you really often can get problems about priorities

> when using different threads while receiving data

> (max & java, or max & external) As an example, J.M.Couturier,

Ok. You're right, my workflow has nothing too latency-intensive in it. I've found Max quite manageable by always giving it "breathing room", ie. no single thing has that much traffic going on. But maybe your project really needs the throughput.

> What is strange then is that i can only get it to work

> using the "osc mode" of udpreceive object (with a symbol

> as attribute) before sending it to aka.datagram.

udpreceive was meant to work with [OpenSoundControl] and actually the osc mode is the default. Adding a symbol makes it pass the udp packet as it is, according to this:

https://cycling74.com/docs/max5/refpages/max-ref/udpreceive.html

> newbie-in-osc question : In what is it good

It's rather the 0% that I like. Simplicity.

>> either MSA Remote interpolates to smoothen out the data
>> or I've missed an undocumented way to somehow get better data.
> Maybe we should just ask him ?

> because your were not afflicting max, but when you start to,
> you really often can get problems about priorities
> when using different threads while receiving data
> (max & java, or max & external) As an example, J.M.Couturier,

> What is strange then is that i can only get it to work
> using the "osc mode" of udpreceive object (with a symbol
> as attribute) before sending it to aka.datagram.

udpreceive was meant to work with [OpenSoundControl] and actually  the osc mode is the default. Adding a symbol makes it pass the udp packet as it is, according to this:https://cycling74.com/docs/max5/refpages/max-ref/udpreceive.html

>> Remember the tagline, 0% OSC content!
> newbie-in-osc question : In what is it good
> to not use OSC content ?

> .."breathing room", ie. no single thing has that much traffic 

> going on. But maybe your project really needs the throughput.

Mine also have "breathing room". The main issue about net.udp.recv/fantastick, is that the lags happen, apparently, not when you push the cpu usage (since this topic, i was never at more than 5%), but when you draw "something", in lcd, or maybe also in jitter i suppose. The test patch of the first message in this topic, is not a "heavy" patch, and the lags are here... It's a priority problem which can almost happen from nowhere, bad bad net.udp.recv. (but on the other hand, i love java for my own mxj stuff i'm on)

You must be right, I felt the opposite reading the doc, this doc is not so clear.

>>> Maybe we should just ask him ?
>Go ahead.

> .."breathing room", ie. no single thing has that much traffic 
> going on. But maybe your project really needs the throughput.

Big Test Patch for iphone Apps: MSA Remote, Fantastick, TouchOSC, OSCemote