Play live webcam feed in Max MSP / Jitter
Hello, I’ve been using Max MSP since before Christmas as part of a course I’ve been doing but haven’t really had much help for it. I succesfully developed a sampler which is triggered by a joypad via the "Hi" object. I am now working on a project (which needs to be done pretty soon!) for which I would like to be able to trigger samples and changes to samples by the feed that comes in from a live webcam.
I have found many articles in which this is mentioned but can’t seem to find the solution to what I need. I have managed to get my webcam working fine on my iBook. I already have a sampler I can try to trigger (I can remove the Hi object input and tailor it to fit the outputs from the webcam).
From what I have read, it would appear that I need to use jit.qt.grab or similar (I am on Macintosh OS), with a type of grid over the top of it in order to split the areas in which movement is ocurring. I.e: if movement in top left, trigger sample 1. If triggered in top right, add granular synthesis, if triggered bottom left, play backwards etc.
I have been reading tutorials and help files left right and centre, have managed to use an object to get a pre-recorded quicktime file to play in Max MSP window, but not to get the video feed that i can see elsewhere on my iBook to play in the Max Patcher.
If you could help me with this it would be greatly appreciated although I understand that you all have lives outside of the forum! I just would like to know how to a) get the webcam feed to play inside the patcher and b) get a grid over it in order to receive data when the webcam’s subject moves.
Many thanks to anyone who can help, it’s driving me crazy!
jit.qt.grab, you have to click "open" to start the camera, and it shows frames once per bang. Does this work in the help file? In my experience, if the OS sees it (like iMovie works fine) then Jitter will see it. Probably it’s not hot-pluggable though. (?)
For tracking motion look into cv.jit (3rd party), this works quite well once you get the settings tweaked right. For sampling individual colors you can use getcell on the matrix, or the suckah object over the video, then depending on where the color is in the video (test with some XY comparators) you can determine what to do with it.
Thank you so much for your response, I’m just trying it now.
I really appreciate your help and will let you know how it goes!
(Don’t think webcam is hot-pluggable – you need to have a little thing called "monitor" running in order to see the picture) I can’t get the webcam to work into iMovie though unless I record the webcam feed and then import it to iMovie after… I’m not that accomplished with iMovie – I’m usually "all about sound"!
Will investigate further for now.
Well, slight progress in that getdevlist brings up a list which includes the webcam I use, however no amount of selecting "open" or fiddling with settings will make that little black screen show the picture. An exported jpeg comes up with black screen also.
I get this in log (don’t know if that gives any clues?):
print: open -1 0
print: exportimage first try.jpg 1
? error: jit.qt.grab: could not create video channel -9405
? error: jit.qt.grab: error opening device for grab.
print: open -1 0
if you get those messages then the OS isn’t seeing the camera, not sure what’s up. AFAIK whenever the cam works in something like iMovie (testing there is good) then Max will see it too. "open -1" means no go :(
That’s weird. The cam definitely works on the commputer because I can see myself on it! It def doesn’t work in iMovie because it says "no camera attached" but I thought it was looking for a videocamera. When i do the getdevlist though, "VideoCamEye" comes up and allows itself to be selected – that’s the name of my webcam, so the OS must accept it. That’s so strange. I’ll keep looking in to it. Basically it’s for a uni project I have an extension for and my lecturers have dissappeared off the face of the planet. So I’m using a programme I’m inexperienced in, with no lecturer help, for my final project! Woohoo! Might try and find another webcam, although few of the cheap ones are Mac-happy.
Will post up how I get on, if I get anywhere I’ll attach the patch.
For anyone who is looking to do the same thing. I have found a work around. Most webcams have a "monitor" function in which you can open a window and see what your webcam is filming. If you use the "jit.desktop" object, and set the coordinates to whereabouts on the screen the "monitor" window is, it display it in Max / Jitter, in a matrix.
Thanks SeeJay for trying to help, I think I just had an awkward webcam and this is a good workaround. Also handy as you don’t even really need to have a webcam – you can just move your mouse around the coordinates for jitter to sense the movement.
Hope this helps anyone who is trawling the forums for the same sort of thing.
Sorry for the (probably too) late comment
I’ve had similar problems with web cams on mac and ended up buying an
external isight. You’ve probably tried this but I had some success by
installing maccam available at http://webcam-osx.sourceforge.net/ and using
it as the driver. Max seems to find cams installed this way.
Hope this helps
> For anyone who is looking to do the same thing. I have found a work around.
> Most webcams have a "monitor" function in which you can open a window and
> see what your webcam is filming. If you use the "jit.desktop" object, and
> set the coordinates to whereabouts on the screen the "monitor" window is, it
> display it in Max / Jitter, in a matrix.
> Thanks SeeJay for trying to help, I think I just had an awkward webcam and
> this is a good workaround. Also handy as you don’t even really need to have
> a webcam – you can just move your mouse around the coordinates for jitter to
> sense the movement.
> Hope this helps anyone who is trawling the forums for the same sort of
yea, dunno when your project gotta be in by but am working on one very similiar. jit.scissors and jit.glue will split the matrix from the grab object up as required and put it back together if need be. There is a tutorial on the Max website called Hold still that will measure movement in any one of these grid sections and this movement can be used to trigger bangs that cause events, (make sure you reference if you use it, cause your tutor probably reads this forum too). Bit of a long way round though, if anyone has any better ideas would love to hear it.
By the way, when using jit.scissors on a jit.qt.grab in my patch, the live feed is only recognised about half of the time in Max 5, worked fine in Max 4 though, anyone else had this problem . B
Project has to be in really really soon – less than a week. I’ve got no further on this. I can’t find anyone who can tell me how to get an output from jit.desktop that will change when what is on the desktop changes and I just don’t know where else I can go with my patch to make it worthwhile. I’ve spent hours poring over forums and looking at patches and tutorials so it’s not a case of wanting a quick fix I’m just really up sh*t creek without a paddle, it would seem.
A few people have downloaded my patch from my other thread but no one has come up with a solution yet I’m finding it all pretty frustrating. (Not that I’m expecting anyone to do the patch for me, I wish I could do it on my own!)
The HoldStill tutorial works with jit.qt and I tried to use some of the elements they used but they didn’t work with desktop or seem to be there for the functions I needed them for. Thanks for the heads up on it though.
Could you post your patch? Dunno if it’ll help the webcam issue (btw did you get that working normally, without having to use jit.desktop?) — but it might be helpful to see where the patch is at now.
A simpler version of something like motion tracking would be to split up the main video feed into (say) 5X5, then each of the 25 submatrices could keep track of their total brightness (use jit.3m to find the mean). Perhaps then something could be "on" if above a certain brightness, "off" if below—like whether a specific effect is applied to a sound elsewhere, etc. The mean levels could also be used to say how much or how little of an effect you want.
You wouldn’t necessarily even need full-color video, either… sometimes it’s neat to use rgb2luma and make things grayscale, which only has one plane of lightness values. This is helpful for things like motion tracking, where you want the best contrast at edges and such. But you could try it both ways (have a 2-outlet gate with one going through rgb2luma, one not).
If you really can’t get the webcam going with jit.qt.grab, maybe create everything you need for it to work, but have a jit.qt.movie element in there too. Then you can at least demonstrate the idea using video clips (like ones you capture with the webcam in a different app, if that’s working). In many respects, using prerecorded video gives you more control, since you can alter things like the playback rate etc. and you’re familiar with the clips. Then the effects which the patch produces through analyzing the pieces of the video become part of your arsenal of instruments, and you can tweak things for performance.
Hope that gives some ideas,
Hi, no still haven’t managed to get it working without using jit.desktop which is quite a pain :-( Have attached patch in its "oh god what do I do now" phase, which is split up using jit.scissors (I tried to use jit.glue to put it back together and get several outputs but all that happened was i had lots of tiny versions of the same thing in every screen! I’ll attach that one later too…
Someone else had suggested jit.3m yesterday although I didn’t really see how the brightness would change it then I realised I was thinking in human terms not mac terms and will be giving it another go today!
Really can’t get the cam to work without desktop so the qt idea sounds like a good one, that way I’d pretty much just mirror the patch only with qt instead of desktop and then you’d just "on" whichever one you wanted I guess… Food for thought! Well, fast-food for thought at the mo!
Cheers for your reply it’s much appreciated it’s all starting to make a little bit more sense now and I can sort of see what I have to do it’s just frustrating when you have a great idea and it isn’t coming to fruition!
I couldn’t open the compressed file you attached. Any way to attach it again as a zip? I think this one is in mac-only format and I’m on XP. If it’s only a patch or two, maybe just paste the code (select all-copy compressed-past into the message text). Using Max 5 right?
No I can’t use Max 5 ‘cos I’m still on 10.3 – every time I upgrade to 10.4 I have to send my hard drive off for repair so I’m waiting until I’ve handed in this patch before I try to upgrade my OS and Max! Eek!
Is this what you wanted? (Hope I’ve copied the right thing)
#P user jit.pwindow 280 209 82 122 0 1 0 0 1 0;
#P user jit.pwindow 196 209 82 122 0 1 0 0 1 0;
#P user jit.pwindow 112 209 82 122 0 1 0 0 1 0;
#P user jit.pwindow 28 209 82 122 0 1 0 0 1 0;
#P user jit.pwindow 281 341 82 122 0 1 0 0 1 0;
#P user jit.pwindow 197 341 82 122 0 1 0 0 1 0;
#P user jit.pwindow 113 341 82 122 0 1 0 0 1 0;
#P user jit.pwindow 29 341 82 122 0 1 0 0 1 0;
#P window setfont "Sans Serif" 9.;
#P number 385 89 35 9 0 0 0 3 0 0 0 221 221 221 222 222 222 0 0 0;
#P number 331 89 35 9 0 0 0 3 0 0 0 221 221 221 222 222 222 0 0 0;
#P window linecount 1;
#P message 385 118 61 196617 columns $1;
#P message 331 118 48 196617 rows $1;
#P button 6 73 15 0;
#P message 164 72 29 196617 stop;
#P message 126 72 33 196617 start;
#P message 91 72 30 196617 read;
#P message 265 118 60 196617 getcolumns;
#P message 212 118 47 196617 getrows;
#P newex 27 149 161 196617 jit.scissors @rows 2 @columns 4;
#P number 68 547 35 9 0 0 0 3 0 0 0 221 221 221 222 222 222 0 0 0;
#P number 24 547 35 9 0 0 0 3 0 0 0 221 221 221 222 222 222 0 0 0;
#P newex 24 483 40 196617 unpack;
#P toggle 27 50 15 0;
#P newex 27 72 58 196617 metro 100;
#P newex 27 97 130 196617 jit.desktop 4 char 320 240;
#B color 5;
#P connect 16 0 14 0;
#P connect 15 0 13 0;
#P fasten 6 7 20 0 158 176 287 176;
#P fasten 6 3 24 0 86 184 286 184;
#P fasten 6 6 19 0 140 176 203 176;
#P fasten 6 2 23 0 68 184 202 184;
#P fasten 6 5 18 0 122 176 119 176;
#P fasten 6 1 22 0 50 184 118 184;
#P connect 3 1 5 0;
#P fasten 6 4 17 0 104 176 35 176;
#P fasten 6 0 21 0 32 184 34 184;
#P fasten 7 0 6 0 217 141 32 141;
#P fasten 14 0 6 0 390 141 32 141;
#P fasten 13 0 6 0 336 141 32 141;
#P fasten 8 0 6 0 270 141 32 141;
#P connect 0 0 6 0;
#P fasten 1 0 0 0 32 93 32 93;
#P fasten 12 0 0 0 11 92 32 92;
#P fasten 9 0 0 0 96 92 32 92;
#P fasten 10 0 0 0 131 92 32 92;
#P fasten 11 0 0 0 169 92 32 92;
#P connect 2 0 1 0;
#P connect 3 0 4 0;
#P connect 17 0 3 0;
#P window clipboard copycount 25;
not really sure what you’re trying to achieve, but if you want to split up an incoming matrix into a grid, and analyze each grid in order to trigger something else, then you’re on the right track. i believe the suggestion to use jit.3m was so you could analyze the brightness of each grid square, in which case you need to plug a jit.3m after each output of the jit.scissors.
right now your patch doesn’t do anything but split a matrix into 8 smaller matrices, so it’s hard to offer much more help.
Thanks Robtherich, I’ve been playing about with jit.3m today and still playing with it now think I may be getting somewhere. Also investigated more in to using a quicktime movie instead and a couple of other things will post back with my progress if anyone is interested. Again, thanks for the help you guys have given me. It’s much appreciated and will be referenced in my work.
Just had a breakthrough by getting the numerical output that I wanted!
Watch this space!!!!!!!!!!
(Insert air-punch and "woohoo!")
(Thanks again for all the suggestions)
Hey man, I’m having pretty much the same problems except on windows. I’ve got my Logitech webcam attached to the computer but when I use the getlist commands with jit.qt.grab, Max doesn’t seem to recognise that there is a camera attached to the computer. Although the "jit.desktop" solution does work it’s incredibly tedious. I don’t suppose you found any other solutions to this problem?
Hey Scriggy, I am trying to accomplish much the same as this project. Would you mind sharing your patch so I can see how it works? I am struggling to get the output from my webcam into max, as the green light next to it shows it switching on, but no signal inputting. Not sure quite what to do, and is also for a project!
Any help is greatly appreciated!