mmiles54

I'm basically trying to build a sampler that a user can control the panning, channel volume and perhaps make the actual sounds using the gestures of one finger. I want to track the x and y points of one of my fingers, using reflective tape.

http://www.youtube.com/watch?v=0awjPUkBXOU

I've heard aka.wiimote is fairly unreliable and hard to get working properly and have been trying out OSCULATOR. I have got infra red signals going into Osculator through the wii controler (using a candle) but don't know what to do to get it into a useable xy format for max/msp.

Also, I have bought a board, battery pack thing and 2 infrared LED's and am hoping to build the kind of setup Johnny Chung Lee has. I'm in the UK though (Bath) and am not sure where and how to go about getting the right stuff. We don't have a ''radio shack''. I think the kind of infra red illuminators and light bulbs you can get would be too powerful and would just light up my whole body as opposed to one finger.

I'm basically trying to build a sampler that a user can control the panning, channel volume and perhaps make the actual sounds using the gestures of one finger. I want to track the x and y points of one of my fingers, using reflective tape. 

I've heard aka.wiimote is fairly unreliable and hard to get working properly and have been trying out OSCULATOR. I have got infra red signals going into Osculator through the wii controler (using a candle) but don't know what to do to get it into a useable xy format for max/msp. 

Also, I have bought a board, battery pack thing and 2 infrared LED's and am hoping to build the kind of setup Johnny Chung Lee has. I'm in the UK though (Bath) and am not sure where and how to go about getting the right stuff. We don't have a ''radio shack''. I think the kind of infra red illuminators and light bulbs you can get would be too powerful and would just light up my whole body as opposed to one finger. 

making-an-infra-red-triggered-sampler-using-the-johnny-chung-lee-method

That's a great video. He is a true genius. Great idea using the reflective tape too.

I'd get all your electronics online through Digikey or Mouser, etc. The LEDs he recommends for the IR pens (in another video) may well be too powerful and possibly too directed, since that's for another application (beaming it to a screen and having the camera track the dots). Probably cheaper, wider-angle, lower-power ones would do fine for the array he built, and again, you could do with a lot fewer than he has, most likely.

You might also look into using an IR camera and the cv.jit objects, that's another way to go about getting the data into Max/Jitter. IR cameras are quite cheap, and there's no Wii hardware or software etc. to go through, it's just straight to Jitter as a camera-grab.

I looked into this in the summer, but never finished my investigations...

1 - aka.wiiremote does have issues - I'm working on a new version that is pre-release at the moment but very reliable, see here for more info:

https://cycling74.com/forums/new-wiiremote-object-for-testing-based-on-aka-wiiremote.

2 - Maplin can supply what you need if you want a store to go to, but are generally very expensive - rapid electronics are pretty fast and cheap for online orders.

3 - There may be an issue with angle with the LEDs for accurate tracking - it's definitely worth considering the angle charateristics of any LEDs you buy - you can look up manufacturers sheets on many electronic component suppliers websites.

4 - as seejayjames says you could take a jitter IR camera approach. Many standard webcams can be hacked to work as IR cams (just google it), but then you have to track the objects yourself - the blob objects in cv.jit work pretty well for this - I was able to get ok results with that kind of system. With the wiiremote you get xy coords plus some size data directly (the analysis is done in the remote).

1 - aka.wiiremote does have issues - I'm working on a new version that is pre-release at the moment but very reliable, see here for more  info:https://cycling74.com/forums/new-wiiremote-object-for-testing-based-on-aka-wiiremote.

Alex, when you were trying the cv.jit objects, were you able to use cv.jit.sort well enough to keep track of the various blobs? I've been running into the "re-indexing" issue a lot... when a blob crosses the X-axis of another, the list changes, as cv.jit.blobs.centroids numbers from top left.

I've had OK luck with cv.jit.sort, but it's still a little flaky if the blobs get too close. Which I imagine is a problem for any system. But was just wondering if you had any caveats about it...getting something reliable and accurate for blob tracking would be a real bonus for the community, since it's all going multi-touch these days :)

I've had OK luck with cv.jit.sort, but it's still a little flaky if the blobs get too close. Which I imagine is a problem for any system. But was just wondering if you had any caveats about it...getting something reliable and accurate for blob tracking would be a real bonus for the community, since it's all going multi-touch these days  :)


You definitely want to use cv.jit.sort to index your blobs, or design your own sorting method - sort seemed to work well for me, although I never developed a fully functional system. You may run into problems if the movement is too extreme between frames, but for slower gestures it should work well. Other approaches might be (if using normal video rather than IR) to colour your blobs in such a way that they can be identified by some jitter native calculation of average colour.

.... I remember that by big problem was with blobs merging (at which point sort will flake out) so that was more of a concern for me than reliable sorting directly - I was thinking a lot about a decent filter to separate blobs as in the case of LEDs the glare on the camera can create haloing around each bulb. To the human eye you can clearly see definite central blobs with surrounding glare, but in jitter as at some point you threshold the greyscale / colour image down to B+W the threshold choice is a compromise between sensitivity of pickup and the point at which merging may occur. I never managed to design an appropriate filter but as far as I caould tell that was what I needed to do - I think what I was thinking about trying to do was determine whether a pixel could be considered glare by trying seeing if it was in a trough (had higher values around it) - so even bright pixels could be identified as glare if necessary.

Ultimately I moved on to other unrelated projects and left this as something to return to - I imagine if I'd carried on I might have ended up approaching the problem by writing an external (I'm not sure how easy it'd be to do with standard jitter objects), but I never really settled on an algorithm in the first place...

Not sure if that's of any use, but those were my thoughts at the time.

Good to hear Maplins has the right stuff, I'm going there now!

REgarding blob overlapping, I really just needed to track the one finger. hopefully the wii won't see any more than the one finger with reflective tape on.

Had a disaster trying to wire up some IR leds last night, lots of burn outs and bad smells. But I have this ordered and it might be here tomorrow....

http://www.amazon.co.uk/gp/product/B00007MCBV/ref=s9_k2a_gw_ir03?pf_rd_m=A3P5ROKL5A1OLE&pf_rd_s=center-6&pf_rd_r=005WF9H63DBY295VVC2M&pf_rd_t=101&pf_rd_p=467128453&pf_rd_i=468294

But I think it might be too powerful and light me up aswell!

Good to hear Maplins has the right stuff, I'm going there now! 

REgarding blob overlapping, I really just needed to track the one finger. hopefully the wii won't see any more than the one finger with reflective tape on. 

But I think it might be too powerful and light me up aswell!


Actually - let me know how you got on - forgot to say I couldn't get any decent / proper reflective tape in Bristol when I tried - the stuff Johnny Lee stuff seems to be easier to find in the states....

Thanks Alex for your thoughts, they are helpful. I'm also looking into tap.tools for its ability to track 4 colored spots at once. Plus the rest of the package looks awesome as well :)

Thanks Alex for your thoughts, they are helpful. I'm also looking into tap.tools for its ability to track 4 colored spots at once. Plus the rest of the package looks awesome as well  :)


I've got a sheet of reflective jacket type material, hopefully it will give off quite a focused beam back to the camera.

I'm at university in Bath in my final year doing a Creative Music Tech course, about 7 miles from Bristol! Do you do Max type work for a living?

Hmmm - I didn't have too much success with reflective cyclewear etc., but as always YMMV - let me know how it works out for you...

No, unfortunately no one pays me to do this stuff - I'm also a student, but a PhD student in composition at York - my hometown is Bristol, so I was just back over the summer visiting when I happened to be working on this...

Making an infra red triggered sampler, using the Johnny Chung Lee method!