Kim Cascone

http://gizmodo.com/gadgets/clips/wiimote-hack-is-wireless-multitouch-

I'm interested in implementing something in Max/MSP similar to the 

finger tracking IR LED hack seen in the video above

doing a little research on wikipedia I find that:

- two IR sources (5 LED's on either side) placed around 20cm apart 

are used to triangulate a single position represented as XY coordinates

    - hence only two values X & Y are received which represent the 

    - this assumes a moving wii remote and a stationary IR source -- not 

a stationary wii remote and a moving IR source

is the XY coordinate data a limitation of the [app/object/ 

whatever] decoding the data coming from the wii remote?

in other words, is it possible to get data for multiple IR point 

sources using the IR data coming from the wii remote?

obviously the wii remote *can* do this (its just a camera) since on 

the 'hack video' there are multiple points on the wii screen 

representing fingers reflecting the IR light coming off the LED array

http://gizmodo.com/gadgets/clips/wiimote-hack-is-wireless-multitouch- 
tv-321329.php

I'm interested in implementing something in Max/MSP similar to the  
finger tracking IR LED hack seen in the video above

doing a little research on wikipedia I find that:
- two IR sources (5 LED's on either side) placed around 20cm apart  
are used to triangulate a single position represented as XY coordinates
	- hence only two values X & Y are received which represent the  
position of the wii remote
	- this assumes a moving wii remote and a stationary IR source -- not  
a stationary wii remote and a moving IR source

- my questions are:
    is the XY coordinate data a limitation of the [app/object/ 
whatever] decoding the data coming from the wii remote?
in other words, is it possible to get data for multiple IR point  
sources using the IR data coming from the wii remote?
obviously the wii remote *can* do this (its just a camera) since on  
the 'hack video' there are multiple points on the wii screen  
representing fingers reflecting the IR light coming off the LED array

qs-wii-remote-ir-data

The wii remote isn't being used to do the tracking. It is merely detecting the IR light. In that youtube video the guy was using some other software to track those IR points. You could use jit.findbounds or the cv.jit objects to pretty easily track the IR light picked up by the camera.

the problem is that OSCulator currently only sends XY triangulation data

On Dec 3, 2007, at 9:39 PM, Solomon Bothwell wrote:

> The wii remote isn't being used to do the tracking. It is merely 

> detecting the IR light. In that youtube video the guy was using 

> some other software to track those IR points. You could use 

> jit.findbounds or the cv.jit objects to pretty easily track the IR 

>
> The wii remote isn't being used to do the tracking. It is merely  
> detecting the IR light. In that youtube video the guy was using  
> some other software to track those IR points. You could use  
> jit.findbounds or the cv.jit objects to pretty easily track the IR  
> light picked up by the camera.


why don't you try using aka.wiiremote object instead of osculator? It can draw the IR data onto a jit.lcd

because I'm working on a G4 laptop and there's a problem with the 

I've contacted Masayuki-san and he says he has been unable track down 

besides, I think [aka.wiiremote] only passes X, Y, angle (radians) 

and tracking data for a single point - is this not the case?

but until Masayuki is able to fix this I'm using OSCulator for my 

and Camille (the developer for OSCulator) is willing to make data for 

four IR points available in basic mode on an upcoming release

http://wiibrew.org/index.php?title=Wiimote#IR_Camera

I'm begging him to implement 'extended' mode so I can also get a 

On Dec 4, 2007, at 10:19 AM, Solomon Bothwell wrote:

> why don't you try using aka.wiiremote object instead of osculator? 

because I'm working on a G4 laptop and there's a problem with the  
[aka.wiiremote] object passing IR data
I've contacted Masayuki-san and he says he has been unable track down  
the bug in his code

besides, I think [aka.wiiremote] only passes X, Y, angle (radians)  
and tracking data for a single point - is this not the case?

but until Masayuki is able to fix this I'm using OSCulator for my  
project
and Camille (the developer for OSCulator) is willing to make data for  
four IR points available in basic mode on an upcoming release
http://wiibrew.org/index.php?title=Wiimote#IR_Camera

I'm begging him to implement 'extended' mode so I can also get a  
value for the size of each IR point

>
> why don't you try using aka.wiiremote object instead of osculator?  
> It can draw the IR data onto a jit.lcd


I guess I should have looked before I leapt. You are right about aka.wiiremote only passing xy and angle. I haven't actually used the wiiremote in any projects so I should have been more careful with my response.

Quote: Kim Cascone wrote on Tue, 04 December 2007 10:44

----------------------------------------------------

> because I'm working on a G4 laptop and there's a problem with the 

> I've contacted Masayuki-san and he says he has been unable track down 

> besides, I think [aka.wiiremote] only passes X, Y, angle (radians) 

> and tracking data for a single point - is this not the case?

> but until Masayuki is able to fix this I'm using OSCulator for my 

> and Camille (the developer for OSCulator) is willing to make data for 

> four IR points available in basic mode on an upcoming release

> I'm begging him to implement 'extended' mode so I can also get a 

> On Dec 4, 2007, at 10:19 AM, Solomon Bothwell wrote:

> > why don't you try using aka.wiiremote object instead of osculator? 

> > It can draw the IR data onto a jit.lcd

Quote: Kim Cascone wrote on Tue, 04 December 2007 10:44
----------------------------------------------------
> because I'm working on a G4 laptop and there's a problem with the  
> [aka.wiiremote] object passing IR data
> I've contacted Masayuki-san and he says he has been unable track down  
> the bug in his code
>
> besides, I think [aka.wiiremote] only passes X, Y, angle (radians)  
> and tracking data for a single point - is this not the case?
>
> but until Masayuki is able to fix this I'm using OSCulator for my  
> project
> and Camille (the developer for OSCulator) is willing to make data for  
> four IR points available in basic mode on an upcoming release
> http://wiibrew.org/index.php?title=Wiimote#IR_Camera
>
> I'm begging him to implement 'extended' mode so I can also get a  
> value for the size of each IR point
>
>
>
>
> On Dec 4, 2007, at 10:19 AM, Solomon Bothwell wrote:
>
> >
> > why don't you try using aka.wiiremote object instead of osculator?  
> > It can draw the IR data onto a jit.lcd
>
>
>
>
----------------------------------------------------


no worries...I wish it was as easy it you made it sound! ;)

On Dec 4, 2007, at 1:36 PM, Solomon Bothwell wrote:

> I guess I should have looked before I leapt. You are right about 

> aka.wiiremote only passing xy and angle. I haven't actually used 

> the wiiremote in any projects so I should have been more careful 

> Quote: Kim Cascone wrote on Tue, 04 December 2007 10:44

> ----------------------------------------------------

>> because I'm working on a G4 laptop and there's a problem with the

>> [aka.wiiremote] object passing IR data

>> I've contacted Masayuki-san and he says he has been unable track down

>> besides, I think [aka.wiiremote] only passes X, Y, angle (radians)

>> and tracking data for a single point - is this not the case?

>> but until Masayuki is able to fix this I'm using OSCulator for my

>> and Camille (the developer for OSCulator) is willing to make data for

>> four IR points available in basic mode on an upcoming release

>> I'm begging him to implement 'extended' mode so I can also get a

>> On Dec 4, 2007, at 10:19 AM, Solomon Bothwell wrote:

>>> why don't you try using aka.wiiremote object instead of osculator?

>>> It can draw the IR data onto a jit.lcd

>
> I guess I should have looked before I leapt. You are right about  
> aka.wiiremote only passing xy and angle. I haven't actually used  
> the wiiremote in any projects so I should have been more careful  
> with my response.
>
> Quote: Kim Cascone wrote on Tue, 04 December 2007 10:44
> ----------------------------------------------------
>> because I'm working on a G4 laptop and there's a problem with the
>> [aka.wiiremote] object passing IR data
>> I've contacted Masayuki-san and he says he has been unable track down
>> the bug in his code
>>
>> besides, I think [aka.wiiremote] only passes X, Y, angle (radians)
>> and tracking data for a single point - is this not the case?
>>
>> but until Masayuki is able to fix this I'm using OSCulator for my
>> project
>> and Camille (the developer for OSCulator) is willing to make data for
>> four IR points available in basic mode on an upcoming release
>> http://wiibrew.org/index.php?title=Wiimote#IR_Camera
>>
>> I'm begging him to implement 'extended' mode so I can also get a
>> value for the size of each IR point
>>
>>
>>
>>
>> On Dec 4, 2007, at 10:19 AM, Solomon Bothwell wrote:
>>
>>>
>>> why don't you try using aka.wiiremote object instead of osculator?
>>> It can draw the IR data onto a jit.lcd
>>
>>
>>
>>
> ----------------------------------------------------
>
>


I'm curious why you decided to use the wii controller for your project. It would be really simple if you bought a cheap IR camera and used the cv.jit objects. Are you using features other than the IR camera in the wiiremote?

well the wii remote cost me $35 -- and none of the IR cameras I found 

unless I'm not looking in the right place...maybe you know of one 

plus the wii remote has all the other features which could be used if 

On Dec 4, 2007, at 1:47 PM, Solomon Bothwell wrote:

> I'm curious why you decided to use the wii controller for your 

> project. It would be really simple if you bought a cheap IR camera 

> and used the cv.jit objects. Are you using features other than the 

well the wii remote cost me $35 -- and none of the IR cameras I found  
have bluetooth
unless I'm not looking in the right place...maybe you know of one  
with bluetooth that is < $35?
plus the wii remote has all the other features which could be used if  
so desired
especially for a dance project

>
> I'm curious why you decided to use the wii controller for your  
> project. It would be really simple if you bought a cheap IR camera  
> and used the cv.jit objects. Are you using features other than the  
> IR camera in the wiiremote?


Ahh. You require bluetooth. I know someone who found a IR camera which cost $45 but it definitely does not use bluetooth. I guess your best bet would be to wait for those updates to OSCulator.

Quote: Kim Cascone wrote on Tue, 04 December 2007 15:24

> well the wii remote cost me $35 -- and none of the IR cameras I found 

> unless I'm not looking in the right place...maybe you know of one 

> plus the wii remote has all the other features which could be used if 

> On Dec 4, 2007, at 1:47 PM, Solomon Bothwell wrote:

> > I'm curious why you decided to use the wii controller for your 

> > project. It would be really simple if you bought a cheap IR camera 

> > and used the cv.jit objects. Are you using features other than the 

Quote: Kim Cascone wrote on Tue, 04 December 2007 15:24
----------------------------------------------------
> well the wii remote cost me $35 -- and none of the IR cameras I found  
> have bluetooth
> unless I'm not looking in the right place...maybe you know of one  
> with bluetooth that is < $35?
> plus the wii remote has all the other features which could be used if  
> so desired
> especially for a dance project
>
>
>
> On Dec 4, 2007, at 1:47 PM, Solomon Bothwell wrote:
>
> >
> > I'm curious why you decided to use the wii controller for your  
> > project. It would be really simple if you bought a cheap IR camera  
> > and used the cv.jit objects. Are you using features other than the  
> > IR camera in the wiiremote?
>
>
----------------------------------------------------


1) the Osculator reports X and Y of just one point?

2) you are able to use the datas, from aka patch or Osculator, from the IR camera? In my case they are too much noisy or strange or sometimes....Nonexistent......

> 1) the Osculator reports X and Y of just one point?

no -- OSCulator passes (x,y) coords which represents the 

triangulation data of the camera itself relative to two IR sources

you can use two candles spaced at ~20cm apart to test this

OSCulator passes the data via port 8000 or whatever port you want to use

which you then pick up with a [udpreceive] object in Max

> 2) you are able to use the datas, from aka patch or Osculator, from 

> In my case they are too much noisy or strange or 

>
> can you explain 2 things?.
>
> 1) the Osculator reports X and Y of just one point?
no -- OSCulator passes (x,y) coords which represents the  
triangulation data of the camera itself relative to two IR sources
you can use two candles spaced at ~20cm apart to test this
OSCulator passes the data via port 8000 or whatever port you want to use
which you then pick up with a [udpreceive] object in Max

>
>
> 2) you are able to use the datas, from aka patch or Osculator, from  
> the IR camera?
no not the aka patch

> In my case they are too much noisy or strange or  
> sometimes....Nonexistent......
>
>
> thank,s
>
> L.
>
>
>
>
>
>


FYI: you can also hack a webcam to be infrared:

but it still needs to be tethered via USB

On Dec 4, 2007, at 4:26 PM, Solomon Bothwell wrote:

> Ahh. You require bluetooth. I know someone who found a IR camera 

> which cost $45 but it definitely does not use bluetooth. I guess 

> your best bet would be to wait for those updates to OSCulator.

> Quote: Kim Cascone wrote on Tue, 04 December 2007 15:24

>> well the wii remote cost me $35 -- and none of the IR cameras I found

>> unless I'm not looking in the right place...maybe you know of one

>> plus the wii remote has all the other features which could be used if

>> On Dec 4, 2007, at 1:47 PM, Solomon Bothwell wrote:

>>> I'm curious why you decided to use the wii controller for your

>>> project. It would be really simple if you bought a cheap IR camera

>>> and used the cv.jit objects. Are you using features other than the

FYI: you can also hack a webcam to be infrared:
http://www.hoagieshouse.com/IR/
but it still needs to be tethered via USB
and it doesn't give you accel data
:(

>
> Ahh. You require bluetooth. I know someone who found a IR camera  
> which cost $45 but it definitely does not use bluetooth. I guess  
> your best bet would be to wait for those updates to OSCulator.
>
> Quote: Kim Cascone wrote on Tue, 04 December 2007 15:24
> ----------------------------------------------------
>> well the wii remote cost me $35 -- and none of the IR cameras I found
>> have bluetooth
>> unless I'm not looking in the right place...maybe you know of one
>> with bluetooth that is < $35?
>> plus the wii remote has all the other features which could be used if
>> so desired
>> especially for a dance project
>>
>>
>>
>> On Dec 4, 2007, at 1:47 PM, Solomon Bothwell wrote:
>>
>>>
>>> I'm curious why you decided to use the wii controller for your
>>> project. It would be really simple if you bought a cheap IR camera
>>> and used the cv.jit objects. Are you using features other than the
>>> IR camera in the wiiremote?
>>
>>
> ----------------------------------------------------
>
>


If you have the XY coords of a point you can easily measure rate of change as explained by a bunch of people in this thread:

https://cycling74.com/forums/index.php?t=msg&goto=120998&rid=0&srch=rate+of+change#msg_120992

Or do you mean the tilt data from the wiiremote accelerometers?

Quote: Kim Cascone wrote on Tue, 04 December 2007 17:45

> FYI: you can also hack a webcam to be infrared:

> but it still needs to be tethered via USB

> On Dec 4, 2007, at 4:26 PM, Solomon Bothwell wrote:

> > Ahh. You require bluetooth. I know someone who found a IR camera 

> > which cost $45 but it definitely does not use bluetooth. I guess 

> > your best bet would be to wait for those updates to OSCulator.

> > Quote: Kim Cascone wrote on Tue, 04 December 2007 15:24

> > ----------------------------------------------------

> >> well the wii remote cost me $35 -- and none of the IR cameras I found

> >> unless I'm not looking in the right place...maybe you know of one

> >> plus the wii remote has all the other features which could be used if

> >> On Dec 4, 2007, at 1:47 PM, Solomon Bothwell wrote:

> >>> I'm curious why you decided to use the wii controller for your

> >>> project. It would be really simple if you bought a cheap IR camera

> >>> and used the cv.jit objects. Are you using features other than the

If you have the XY coords of a point you can easily measure rate of change as explained by a bunch of people in this thread:
https://cycling74.com/forums/index.php?t=msg&goto=120998&rid=0&srch=rate+of+change#msg_120992

Quote: Kim Cascone wrote on Tue, 04 December 2007 17:45
----------------------------------------------------
> FYI: you can also hack a webcam to be infrared:
> http://www.hoagieshouse.com/IR/
> but it still needs to be tethered via USB
> and it doesn't give you accel data
> :(
>
>
> On Dec 4, 2007, at 4:26 PM, Solomon Bothwell wrote:
>
> >
> > Ahh. You require bluetooth. I know someone who found a IR camera  
> > which cost $45 but it definitely does not use bluetooth. I guess  
> > your best bet would be to wait for those updates to OSCulator.
> >
> > Quote: Kim Cascone wrote on Tue, 04 December 2007 15:24
> > ----------------------------------------------------
> >> well the wii remote cost me $35 -- and none of the IR cameras I found
> >> have bluetooth
> >> unless I'm not looking in the right place...maybe you know of one
> >> with bluetooth that is < $35?
> >> plus the wii remote has all the other features which could be used if
> >> so desired
> >> especially for a dance project
> >>
> >>
> >>
> >> On Dec 4, 2007, at 1:47 PM, Solomon Bothwell wrote:
> >>
> >>>
> >>> I'm curious why you decided to use the wii controller for your
> >>> project. It would be really simple if you bought a cheap IR camera
> >>> and used the cv.jit objects. Are you using features other than the
> >>> IR camera in the wiiremote?
> >>
> >>
> > ----------------------------------------------------
> >
> >
>
>
>
>
----------------------------------------------------


On Dec 4, 2007, at 6:24 PM, Solomon Bothwell wrote:

> If you have the XY coords of a point you can easily measure rate of 

> change as explained by a bunch of people in this thread:

> t=msg&goto=120998&rid=0&srch=rate+of+change#msg_120992

> Or do you mean the tilt data from the wiiremote accelerometers?

yes that is what I meant -- sorry should have been more clear...

>
> If you have the XY coords of a point you can easily measure rate of  
> change as explained by a bunch of people in this thread:
> https://cycling74.com/forums/index.php? 
> t=msg&goto=120998&rid=0&srch=rate+of+change#msg_120992
yup I know this thanks

>
> Or do you mean the tilt data from the wiiremote accelerometers?
yes that is what I meant -- sorry should have been more clear...


what's extremely annoying about this video is that the multi-blob 

video he's getting is actually coming from the wii running in setup 

he's using a $250 game console to get those two touch blobs.

On Dec 4, 2007, at 4:36 PM, Solomon Bothwell wrote:

> aka.wiiremote only passing xy and angle. I haven't actually used the 

> wiiremote in any projects so I should have been more careful with my 

what's extremely annoying about this video is that the multi-blob  
video he's getting is actually coming from the wii running in setup  
mode!

>
> I guess I should have looked before I leapt. You are right about  
> aka.wiiremote only passing xy and angle. I haven't actually used the  
> wiiremote in any projects so I should have been more careful with my  
> response.
>
> Quote: Kim Cascone wrote on Tue, 04 December 2007 10:44
> ----------------------------------------------------
>> because I'm working on a G4 laptop and there's a problem with the
>> [aka.wiiremote] object passing IR data
>> I've contacted Masayuki-san and he says he has been unable track down
>> the bug in his code
>>
>> besides, I think [aka.wiiremote] only passes X, Y, angle (radians)
>> and tracking data for a single point - is this not the case?
>>
>> but until Masayuki is able to fix this I'm using OSCulator for my
>> project
>> and Camille (the developer for OSCulator) is willing to make data for
>> four IR points available in basic mode on an upcoming release
>> http://wiibrew.org/index.php?title=Wiimote#IR_Camera
>>
>> I'm begging him to implement 'extended' mode so I can also get a
>> value for the size of each IR point
>>
>>
>>
>>
>> On Dec 4, 2007, at 10:19 AM, Solomon Bothwell wrote:
>>
>>>
>>> why don't you try using aka.wiiremote object instead of osculator?
>>> It can draw the IR data onto a jit.lcd
>>
>>
>>
>>
> ----------------------------------------------------
>
>
>


Quote: joshua goldberg wrote on Wed, 05 December 2007 15:21

> what's extremely annoying about this video is that the multi-blob 

> video he's getting is actually coming from the wii running in setup 

i got only the wiimote without console, that's why i don't understand...i hope :)

Quote: joshua goldberg wrote on Wed, 05 December 2007 15:21
----------------------------------------------------
> what's extremely annoying about this video is that the multi-blob  
> video he's getting is actually coming from the wii running in setup  
> mode!
>

I assume its a calibration mode for getting the remote synced with the system. It looks like in setup mode the remote is used as an infrared camera for picking up the IR LED bar data from box ontop of the TV. This is the mode Kim Cascone needs the remote to be in if he wants to use the camera as such.

Quote: lorbi wrote on Wed, 05 December 2007 16:56

> Quote: joshua goldberg wrote on Wed, 05 December 2007 15:21

> > what's extremely annoying about this video is that the multi-blob 

> > video he's getting is actually coming from the wii running in setup 

> i got only the wiimote without console, that's why i don't understand...i hope :)

Quote: lorbi wrote on Wed, 05 December 2007 16:56
----------------------------------------------------
> Quote: joshua goldberg wrote on Wed, 05 December 2007 15:21
> ----------------------------------------------------
> > what's extremely annoying about this video is that the multi-blob  
> > video he's getting is actually coming from the wii running in setup  
> > mode!
> >
>
>
> what does it mean "set up" mode?
>
> i got only the wiimote without console, that's why i don't understand...i hope :)
>
>
> L.
----------------------------------------------------


q's: wii remote IR data