Marsha Vdovin

Interactivity and education are at the core Dr. James Ryan's work.  As a multi-media artist and performer, Ryan uses Max to interactively play with his audience.  As a practicing medical doctor, he uses Max to help patients explore and learn about their own bodies. Illustrating how Max often bridges art and practicability, Dr. Ryan spoke to us about how he is currently using Max to develop changes in the way medical records are created and presented, and how he is applying these same principles to entertain while educating his audience.

Growing up, were you interested in the arts or music?

Oh, yeah. My dad Tom Ryan, he’s a well-established Irish folk musician. So we have always had super-creative types coming through the house since I was a baby. It was all folky stuff, and Irish, but they were still really cool.

Yeah. I started off when I was a kid. I’ve always played music.

Did you go to college here in the States?

Well, I first went for a semester, and I failed. I’m a doctor now. But at that point, I had no intention of doing anything in college at all, really.

 that did a lot of theatrical type stuff. Now, this was before anyone in my world could afford a computer, so it was all camcorder. We’d do a lot of stop motion, cut up type of things with camcorders — the typical things that teenagers explore when they are in groups. But it was fun.

One of the things that I worked on a lot at that earlier phase was conceptual improvisation. We’d pick themes and audience members would shout out a word, and we would improvise our interpretation.

...you can let the signals flow between whatever you can find an object to parse from

Max, I started using around 2000. I had a friend, Deon Foster, who was one of the earliest VJs in the Detroit Electronic music scene, and he was using NATO. He got me excited about the program and I just got really into it with him. We formed a performance group called 

 — Kero, Jimmy Edgar and a bunch of people like that came out of it — and that was the scene that we were part of. A lot of those guys patch, and still do.

Can you tell us more about your performance group?

Shenanigans.tv is a multimedia group that Deon Foster and I started with James Madaski. We used Max and Jitter regularly for live shows and to create audio video tracks for live remixing. We started out playing with the Detroit Underground crew at shows in Detroit.

, where we would write kind of a primitive music notation that we would explain to the audience, and then some of it would be inserted into the patches and could be tweaked real time, and some of it would have been written beforehand.

So we would create these huge sound textures in the audience, of people just making phonetical sounds like, “mmm” and “Ssss”. Depending on where you stood in space, it could sound so weird.

Then we’d have the visual patches that would run. We had designed and built our own audio-video sequencer in Max, which is like our core. Every video element and every audio was synched to our sequencer. So it became like the brains behind all of our patches, for the most part.

That’s what is so great about Max, you can let the signals flow between whatever you can find an object to parse from.

Our principle was we would never sample anything. Everything was completely created independently.

Shenanigans.tv are still going, but we’ve got kids now, and we just don’t have the energy that we used to, to do shows. But we’re actually going to be starting to do a bunch of shows again in the next year.

Honestly, because the online documents were so well written that it wasn’t that difficult. I’m the kind of person who likes to follow a structure, so I really did well having those documents.

Yeah, you go through the tutorials, and then you start cutting and pasting and just changing it up. We would patch until we’d go live. Sometimes we’d be patching in the middle of the show! [Laughs]

Our principle was we would never sample anything. Everything was completely created independently. Obviously computers weren’t powerful enough back then to be able to do what they do now, so you’d always be pushing the limits of the machine.

But I definitely have fond memories of learning Max.

How did you go from there to becoming a doctor?

I was already kind of committed to becoming a doctor by the time I had started using Max. So I was doing my undergraduate type of stuff.

Then I actually took a year off in the middle of Med School and started a master’s in statistics. Well, I failed that pretty quickly, so I spent the rest of the year just working on Max and Jitter [Laughs]. That gave me the chance to get heavily, heavily into it.

Well, I had promised myself that I would maintain a certain relationship with it, because if you don’t use it, you start to forget. Like, “What was the name of that object?” It’s really hard to remember the names of objects after a while. Or particular ways to get around certain problems that you always come to when you’re doing something.

So I thought, “OK, I have to maintain my chops,” so I would just do something. Actually, my oldest son Tommy is 13, and he’s patching now on his own, which is fun.

So anyhow, I’m a family doctor now and I work in small town on the coast of Michigan.

I don’t know if you know much about the state that medical records are in right now in America?

...not only can you use Max to make really cool and beautiful audio-video stuff, but you can use it to make real-life, practical things really quickly.

They’re trying to move them to a computerized network?

Exactly. And they’re succeeding to a certain extent. But they’re atrocious, just horrible.

They’ve really failed in having any sense of aesthetic or beauty.

I just had an experience with my Doctor not having access to my old x-rays and pictures that were ‘still on the old system.’ I was really surprised because this was at a big hospital, in a big city, with a rep for being a bit ahead of the game.

That’s how I feel, and that’s how my patients feel. This is not a weird response.

I’ve become involved with some national scientists and various groups, like 

, that are hell-bent on making it awesome, quite frankly. So we all asked, “Well, what are we going to do?”

So about a year and a half ago, one of the things that I wanted to do was to capture as much objective information as I could with my patients. Like you’re talking about with those older pictures and x-rays is a perfect example. So I thought, “OK, I can definitely whip this together really quickly using Jitter and Max.”

So, I made a patch that basically acts as a matrix switcher between a couple different video inputs, which allows you to capture still images based on where they are on the body. Then you can you store them as part of an anatomical structure. I’ve got my stationary web cam video in, and then I’ve got a digital otoscope for things like looking into ears or doing macro pictures of skin lesions, things like that.

So I put it together and quickly made a really nice-looking user interface. The UI options in Max are great for just throwing together things that look cool.

I mean, it’s really not that awesome of a patch. I’m not going to lie to you, it’s not like the sweetest patch I’ve ever made by a long shot. [Laughs] But Max is a really great way to do rapid prototyping with multimedia.

I think it’s important for people to realize that not only can you use Max to make really cool and beautiful audio-video stuff, but you can use it to make real-life, practical things really quickly.

And with the on-line libraries you don’t necessarily have to design and make a lot of the components.

That’s another thing that I love about Max is that so many people write objects. It’s really easy to take a new piece of technology and start to mix it in really quickly. You don’t have to wait for an SDK often, because someone’s already hacked it, and then they’ve posted the object.

For example, this digital otoscope I use. It has a button on there to take a picture, but you have to work in Windows, and you had to have their SDK in order to capture that hardware’s output. So I just got a guitar foot controller, and hooked it up via MIDI USB through Max, and boom, I’ve got my foot controller to take pictures.

Now it’s not the perfect solution, but it’s really hard to do that in a different language so quickly. So somebody who’s not a professional programmer, like myself, can put together something that does what I need and looks good. And my patients really liked it. It was such a good buzz to run a patch in the office, because I had missed it for so long. It was kind of like this dream come true in a weird way.

I have a really good relationship with my patients, so I would show them the patches. I would take pictures of different parts of their body, and it was really fun just exploring those with my patients. Like the inner ear is a weird space. Everybody found it awesome to see and explore.

Things that we used to spend a lot of time on, just because they looked beautiful, now become immediately simple to use when you want to make something look a particular way.

So the point of all this is ultimately so that when patients come in, all their information goes to them. The goal is that us doctors just become helpers of our patients, coordinating their information, and the patient gets to keep the data and pass it on to their kids.

That can be a powerful thing when you think about it. It’s almost like curating art. It’s like, who am I as a person?

Are there any other devices you’ve been experimenting with?

The other thing that I was doing was using the Kinect gaming sensor. I couldn’t get valuable enough clinical information from it, but it was still really fun to explore moving bodies with kids. Once in a while, if my patients were in a good mood, and I was too, I would take some of the Kinect data and process it through 

 — which are awesome, as always. And then we just goof around with that stuff. Those are just like fun eye candies. But eye candy is good. There’s nothing shameful with that.

Have you applied any of this thinking back towards your artist self?

In a way, yeah. You know when you’re just looking at and listening to abstract audio-video, and then someone else comes in, takes a look and they just don’t like that particular phenomenon. Why? They just don’t like it. That’s the type of thing I really want to start to explore and quantify — perception.

I’m creating a new part of your medical record with Evermed. So anyone can create an account, and that becomes a part of your medical records — it’s an extension of your EHR [electronic health record.] This extension allows us to explore the question “What are our audio-video preferences?”

For example, you get a sine wave and you can turn a knob to sweep it and find your favorite frequency. Then, what’s your favorite strobe rate? What is your favorite rate of moving through a particle system? What’s your favorite ratio between two geometric objects? How many geometric objects are you comfortable with? How many before it’s uncomfortable?” And so on.

This data can be captured for the same user over time, and can be shared and compared to other users. It could be linked to our digital experiences in arts and games.

Through that, we start to find out greater things about ourselves. Ultimately, once we get this populated with enough people, we’ll start to do live shows. So when you come to one of our shows, you can add your preferences to the queue, and that will start to change the patch based on who is there — democratically.

You can’t get more democratic than that! So you’re planning to do shows again? How exciting.

Shenanigans.tv are going to do a show down in Ann Arbor in a few months, and we’re going to be using Max to basically create a lie, an illusion.

We’re going to use body sensors, and explore how individuals respond to our digital multimedia art, but it will actually be our patch, not the sensor data that the audience will be exposed to. We will play the patch to control the feelings of the individual hooked up to the sensor, and with the audience, we’ll explore system manipulation.

“If you don’t help build a system, then it can basically define you however it chooses to.”

Oh, yeah.  My dad Tom Ryan, he’s a well-established Irish folk musician.  So we have always had super-creative types coming through the house since I was a baby.  It was all folky stuff, and Irish, but they were still really cool.

Yeah.  I started off when I was a kid.  I’ve always played music.

Well, I first went for a semester, and I failed.  I’m a doctor now.  But at that point, I had no intention of doing anything in college at all, really.

At the time we had a group called Joey that did a lot of theatrical type stuff.  Now, this was before anyone in my world could afford a computer, so it was all camcorder.  We’d do a lot of stop motion, cut up type of things with camcorders — the typical things that teenagers explore when they are in groups.  But it was fun.

One of the things that I worked on a lot at that earlier phase was conceptual improvisation.  We’d pick themes and audience members would shout out a word, and we would improvise our interpretation.

...you can let the signals flow between whatever you can find an object to parse from


Max, I started using around 2000.  I had a friend, Deon Foster, who was one of the earliest VJs in the Detroit Electronic music scene, and he was using NATO.  He got me excited about the program and I just got really into it with him.  We formed a performance group called Shenanigans.tv and Max was a key ingredient.

There is a record label called Detroit Underground — Kero, Jimmy Edgar and a bunch of people like that came out of it — and that was the scene that we were part of.   A lot of those guys patch, and still do.

Shenanigans.tv is a multimedia group that Deon Foster and I started with James Madaski.  We used Max and Jitter regularly for live shows and to create audio video tracks for live remixing.  We started out playing with the Detroit Underground crew at shows in Detroit.

We used to do something called Animated Scores, where we would write kind of a primitive music notation that we would explain to the audience, and then some of it would be inserted into the patches and could be tweaked real time, and some of it would have been written beforehand.

So we would create these huge sound textures in the audience, of people just making phonetical sounds like, “mmm” and “Ssss”.  Depending on where you stood in space, it could sound so weird.

Then we’d have the visual patches that would run. We had designed and built our own audio-video sequencer in Max, which is like our core.  Every video element and every audio was synched to our sequencer.  So it became like the brains behind all of our patches, for the most part.

Our principle was we would never sample anything.  Everything was completely created independently.


Shenanigans.tv are still going, but we’ve got kids now, and we just don’t have the energy that we used to, to do shows.  But we’re actually going to be starting to do a bunch of shows again in the next year.

Honestly, because the online documents were so well written that it wasn’t that difficult.  I’m the kind of person who likes to follow a structure, so I really did well having those documents.

Yeah, you go through the tutorials, and then you start cutting and pasting and just changing it up.  We would patch until we’d go live.  Sometimes we’d be patching in the middle of the show! [Laughs]

Our principle was we would never sample anything.  Everything was completely created independently.  Obviously computers weren’t powerful enough back then to be able to do what they do now, so you’d always be pushing the limits of the machine.

I was already kind of committed to becoming a doctor by the time I had started using Max.  So I was doing my undergraduate type of stuff.

Then I actually took a year off in the middle of Med School and started a master’s in statistics.  Well, I failed that pretty quickly, so I spent the rest of the year just working on Max and Jitter [Laughs].  That gave me the chance to get heavily, heavily into it.

Well, I had promised myself that I would maintain a certain relationship with it, because if you don’t use it, you start to forget.  Like, “What was the name of that object?”  It’s really hard to remember the names of objects after a while.  Or particular ways to get around certain problems that you always come to when you’re doing something.

So I thought, “OK, I have to maintain my chops,” so I would just do something.  Actually, my oldest son Tommy is 13, and he’s patching now on his own, which is fun.

...not only can you use Max to make really cool and beautiful audio-video stuff, but you can use it to make real-life, practical things really quickly.


Exactly.  And they’re succeeding to a certain extent.  But they’re atrocious, just horrible.

I just had an experience with my Doctor not having access to my old x-rays and pictures that were ‘still on the old system.’  I was really surprised because this was at a big hospital, in a big city, with a rep for being a bit ahead of the game.

That’s how I feel, and that’s how my patients feel.  This is not a weird response.

I’ve become involved with some national scientists and various groups, like Evermed, that are hell-bent on making it awesome, quite frankly.  So we all asked, “Well, what are we going to do?”

So about a year and a half ago, one of the things that I wanted to do was to capture as much objective information as I could with my patients.  Like you’re talking about with those older pictures and x-rays is a perfect example.  So I thought, “OK, I can definitely whip this together really quickly using Jitter and Max.”

So, I made a patch that basically acts as a matrix switcher between a couple different video inputs, which allows you to capture still images based on where they are on the body.   Then you can you store them as part of an anatomical structure.  I’ve got my stationary web cam video in, and then I’ve got a digital otoscope for things like looking into ears or doing macro pictures of skin lesions, things like that.

So I put it together and quickly made a really nice-looking user interface.  The UI options in Max are great for just throwing together things that look cool.

I mean, it’s really not that awesome of a patch.  I’m not going to lie to you, it’s not like the sweetest patch I’ve ever made by a long shot.  [Laughs]  But Max is a really great way to do rapid prototyping with multimedia.

That’s another thing that I love about Max is that so many people write objects.  It’s really easy to take a new piece of technology and start to mix it in really quickly.  You don’t have to wait for an SDK often, because someone’s already hacked it, and then they’ve posted the object.

For example, this digital otoscope I use.  It has a button on there to take a picture, but you have to work in Windows, and you had to have their SDK in order to capture that hardware’s output.  So I just got a guitar foot controller, and hooked it up via MIDI USB through Max, and boom, I’ve got my foot controller to take pictures.

Now it’s not the perfect solution, but it’s really hard to do that in a different language so quickly.  So somebody who’s not a professional programmer, like myself, can put together something that does what I need and looks good.  And my patients really liked it.  It was such a good buzz to run a patch in the office, because I had missed it for so long.  It was kind of like this dream come true in a weird way.

I have a really good relationship with my patients, so I would show them the patches.  I would take pictures of different parts of their body, and it was really fun just exploring those with my patients.  Like the inner ear is a weird space.  Everybody found it awesome to see and explore.

So the point of all this is ultimately so that when patients come in, all their information goes to them.  The goal is that us doctors just become helpers of our patients, coordinating their information, and the patient gets to keep the data and pass it on to their kids.

That can be a powerful thing when you think about it.  It’s almost like curating art.  It’s like, who am I as a person?

The other thing that I was doing was using the Kinect gaming sensor.  I couldn’t get valuable enough clinical information from it, but it was still really fun to explore moving bodies with kids.  Once in a while, if my patients were in a good mood, and I was too, I would take some of the Kinect data and process it through Jitter Cookbook recipes — which are awesome, as always.  And then we just goof around with that stuff.  Those are just like fun eye candies.  But eye candy is good.  There’s nothing shameful with that.

In a way, yeah.  You know when you’re just looking at and listening to abstract audio-video, and then someone else comes in, takes a look and they just don’t like that particular phenomenon.  Why?  They just don’t like it.  That’s the type of thing I really want to start to explore and quantify — perception.

I’m creating a new part of your medical record with Evermed.  So anyone can create an account, and that becomes a part of your medical records — it’s an extension of your EHR [electronic health record.]  This extension allows us to explore the question “What are our audio-video preferences?”

For example, you get a sine wave and you can turn a knob to sweep it and find your favorite frequency.  Then, what’s your favorite strobe rate?  What is your favorite rate of moving through a particle system?  What’s your favorite ratio between two geometric objects?  How many geometric objects are you comfortable with?  How many before it’s uncomfortable?”  And so on.

This data can be captured for the same user over time, and can be shared and compared to other users.  It could be linked to our digital experiences in arts and games.

Through that, we start to find out greater things about ourselves.  Ultimately, once we get this populated with enough people, we’ll start to do live shows.  So when you come to one of our shows, you can add your preferences to the queue, and that will start to change the patch based on who is there — democratically.

You can’t get more democratic than that!  So you’re planning to do shows again?  How exciting.

We’re going to use body sensors, and explore how individuals respond to our digital multimedia art, but it will actually be our patch, not the sensor data that the audience will be exposed to.  We will play the patch to control the feelings of the individual hooked up to the sensor, and with the audience, we’ll explore system manipulation.

The moral of its story is, “If you don’t help build a system, then it can basically define you however it chooses to.”

Text interview by Marsha Vdovin and Ron MacLeod for Cycling '74.


Does your doctor use Jitter? Whether it's doing visuals for Detroit musicians, audio-visual theater, or practicing medicine, James Ryan has found ways to expand his practice with Max. 

an-interview-with-james-ryan

Oh, man, this is fantastic. Pretty much every time I go to the doctor or dentist, I get some idea of how I could have improved the whole experience with a Max patch. You are taking it to the next level. Congratulations!

Absolutely fascinating! I came to Max in a roundabout way, as part of an effort to do data visualization in Jitter. I ended up working on dance projections instead (

), which has been great fun. However, I'm still a psychology professor teaching statistics, so this definitely makes me wonder how I can integrate Max/MSP/Jitter into my teaching. Thanks for the inspiration!

Absolutely fascinating! I came to Max in a roundabout way, as part of an effort to do data visualization in Jitter. I ended up working on dance projections instead (danceandcode.com), which has been great fun. However, I'm still a psychology professor teaching statistics, so this definitely makes me wonder how I can integrate Max/MSP/Jitter into my teaching. Thanks for the inspiration!


An Interview with James Ryan