An Interview with Chris Coleman

Artist and educator Chris Coleman is recognized in the Max community for his work on Maxuino, a Max interface to the popular Arduino microcontroller board. When I spoke with him, Chris also discussed his early sculpture work, teaching, collaborations, and how Max/MSP and Arduino fit into the evolving landscape of electronic media tools and practices.

Why don’t you start by introducing yourself?

I’m from West Virginia, where I got my undergraduate degree in sculpture from West Virginia University – that’s after I studied engineering for five years, then stopping just six credits short when I realized that it was not going to be a creative enough pursuit for me.

I then went on to SUNY-Buffalo, where I studied a whole bunch of things, including virtual reality and electronics, and got my MFA in art. Since then, I’ve been making all sorts of work; everything from bits and gizmos and kinetic sculpture to doing performance work, animations and videos. I’ve been teaching my way across the country, from Minnesota to Oregon and now here in Denver.

You are most known in the Max community for your work with the Maxuino interface to the Arduino controller. How did you get into that?

You know, it’s one of those things that happened: when you teach, you build a lot of tools. If you are teaching a complex technical course in 10 weeks, as we do here, you have to make things easier. And so I ended up coding a lot of different tools to make the transition into things very easy.

I went online and got very excited about this thing called Firmata, which has been run mainly by Hans-Christoph Steiner and now a number of other people who have jumped in and done a quite a bit of work. Firmata is, basically, firmware (in the old school sense) that you put on your Arduino board. The idea is that, once you have Firmata on your board, you can stop programming the Arduino and focus on programming everything else. So if you are going to be programming in Processing, or Flash, or even Max/MSP, you can just concentrate on that.

The old model was that you would adjust something in Processing or Max, then you would go to the Arduino and adjust its programming so that it would accept the input or output of your code; back and forth, coding for two platforms at once. When I found Firmata and began to understand the possibilities, I said “Wow! What else can Firmata do?”

So I’d been exploring using it with ActionScript, and we’ve been trying to bring Max/MSP into the program here (at University of Denver). It seems like we’ve built a critical mass of people now; it was of interest to me when I arrived just three years ago, but something that Trace Reddell and Tim Weaver, other professors here at DU, have been dabbling with it for a while. Now that we’ve got the critical mass of people, it's something that we will be teaching as a tool for a number of uses; having that connection so that Max/MSP can talk to an Arduino means we can work on that part of the program as well. Having I/O other than just the keyboard and mouse really opens up what can be done to talk to the computer.

So I went and hunted down the thing that lets Firmata talk to Max, and it was something called Maxuino. It was almost a piece-by-piece port of pduino, which was something somebody had made for Pd. But they didn’t really know Max; the person who had done it had just done the best he could, and put it out there. But it wasn’t ready for Max 5, and it wasn’t ready for the new stuff they had done with Firmata, so it was no longer functional.

And I said “I think I know enough Max to get it to run…”, and that’s what I’ve been doing ever since. I took the project over from Marius Schebella and have been trying to slowly and iteratively rebuild Maxuino to make it more efficient, to make more sense and easier to use. The last major upgrade is in pretty wide use; it has had over 4500 downloads from 52 countries. I get teachers in Mexico and the Netherlands that are teaching classes with it, it’s very popular in Japan, and that’s cool.

What’s really exciting is that I’ve been working with Ali Momeni, who is up in Minnesota teaching and doing some really amazing work, doing projections on the streets and creating cool performance tools. We’ve been working together; I’ve been doing the backend, while Ali completely remastered the front end – the GUI for it – and we just did a major release of it on a new site, Maxuino.org. It’s a fantastic release and a great upgrade, and at long last it will come to the other boards: the people using the Teensy and the Mega have been waiting for that, and it’s good to go.

Click here to see a full-sized version.

Well, this is a timely occasion to be talking to you. Talking about Firmata, could you explain a little more about what the Firmata is doing on the hardware side?

With the Arduino, you have specific code that tells it which is an input pin, which is an output pin, and whether you want to be listening to analog values, because it turns out that if you are not carefully controlling whether something is on or off, you can get all sorts of random data. And you also want to tell the Arduino what to do with that data – whether to send it to the computer or use it itself. If you are primarily using the Arduino as an input and output port (meaning that you are using it to shuttle data into and out of your USB port), then what Firmata does is give you a simple set of commands that can turn on and off any of the pins, set them to input mode and gather any of the data and put it into an easily transmitted format.

If you have to write all that code yourself, you are stuck with all those states. It ends up being a big pain, to make up and send all your commands. Firmata simplifies that process, so that it is a purely input and output device that can be controlled from the host software.

So your work with the Maxuino is primarily to simplify the interface between the hardware and Max. What kind of things do you work with your student on?

I teach a lot of classes where we are trying to think about how you talk to a computer without a mouse and a keyboard. That’s my big interest. So, we are dabbling with things like multi-touch and interactive devices, using web cams and OpenCV and things like this. But even more, we are getting interested in things like how you start using infrared or bend sensors, or having the computer drive motors and how you get it to feed back out into the real world.

I think because our Max program here is new, I don’t think our students have dived into what that all means. At the moment, we are just replacing a little MIDI controller board with dials and things through the Arduino, but I think this is just the early stage of it. I see immense ways for it to open up performance, and ways you can do it from scratch. They sell great tools for you to use your hand as a controller using an infrared sensor, but we can build one for a dollar, right? There seems to be no reason to buy a specialized device when you can make the exact one that you need.

So I think that is where we are trying to push performative, realtime control; how does action and natural body motion translate is really interesting. Whether that activity controls video via Jitter or the sound work through MSP – I think we are really exploring those options.

To what extent do you use these tools for your own work? Or is your own work a separate thing? Are you in the process of combining them? How does integrating hardware work into your own art?

Well you know, I started out with Max and Jitter as a VJ-ing tool. I was playing around with crazy tools coming out Japan, and they really didn’t have any US-based tools. So I was downloading all of these instruction manuals, and they were all in Japanese. I was introduced to the Max/MSP/Jitter world, and realized that with Jitter I could make exactly the tool I wanted, and I wouldn’t have to keep messing around with these other tools that I didn’t quite understand or do quite what I wanted. Other software had lots of corny effects that I would never use, so how could I concentrate on doing the things that I wanted to accomplish with it?

I started making tools and experimenting with the 3D parts of Jitter, and testing out all of the possibilities with it. And it was great; I worked with Radian and Pan-American and DJ Spooky, working on all these great projects that came through Buffalo NY while I was there. (And B. Fleischman is another great artist I used to work with…) So that was pretty amazing to go through that. Since then, as part of my master’s thesis, I used Max and Jitter to decipher some surveillance footage and translate that information into pulses that controlled a series of fans.

This was a series of 200 computer fans under sheet plastic. Those pulses set up landscapes that moved from one space to another under the plastic. Max/MSP and Jitter was a super-intuitive way to take images of people, pick out pieces of information (like how many people were in the space, how much were they moving, the color of their clothing) and have that translate into data that formed into the landscapes.

At the time, it wasn’t something I thought I was interested in – I didn’t think I would get into coding. But it was so easy to use it – so easy to make the necessary adjustments – and to use Max to do translations in a way that was really powerful to me, then send the data to an output board.

At the time, we didn’t have such things as Arduinos; at the time, I was using the EZIO board, which was made by a professor in Ann Arbor at the University of Michigan. I had been trained on the Basic Stamp, which was the new cheap thing, but it was kind of a pain to program. Watching the Arduino movement take shape is really exciting. Especially with Firmata, I can get a student up and running, and making beautiful things pretty much instantly. They are able to hook up sensors and watch it on the screen; it’s very powerful, and it gets them interested. Even in the two-day overview of tangible interfaces I occasionally teach, people get invested into it really quickly.

"Taking Sides" Collaborative interactive Installation made with Laleh Mehran, which used Max/MSP to drive the interaction and a series of high powered blowers that constantly shifted the cloth landscape according to the generated conversations. It draws from interpersonal and historically politicized discourse. Shifting landscapes between two parties reveal the exchange of ideas and control. The audience is able to take a position on issues thus reinforcing the dominance of one side.

"Taking Sides" is an interactive installation in which every viewer participates. Their presence directly implicates them in a debate, siding with one view or the other. The viewer’s location in the installation space is sensed and interpreted as reinforcement for that viewpoint. The debate/argument is assembled in real-time from a pool of clips whose intensity is translated into physical motion via several fans on each side. The viewer can choose to position themselves with other participants thus increasing the dominance of that position, expressed as growing intensity in physical and sonic exchange; or take the minority position on the other side to attempt to change the balance of power.

In your teaching here at the University, you work in a lot of different environments: in Processing, in Flash, Open Framework and Max. How do you explain when one environment is more useful than another?

You know, right now, we are on the cusp of this really interesting place. If you would have asked me five years ago “Would you be teaching programming languages?”, I would have laughed it off. I would probably have considered Max. But now, with the advent of Processing and Open Frameworks, it’s really opened up my comfort with introducing these things to students without them needing a complex Computer Science background. I still think the tools vary on their usefulness: Processing is still very powerful for doing basic things with graphics quickly, and doing vision stuff, but I’m not ready to make games in it, you know?

Flash is a great space for making games and web interfaces, but I’m not ready to perform with Flash. And so, even with Open Frameworks and Processing, you never want to have to build a GUI. So you will build something and let it run, or you build something and let it run in hardware. But I think this is the brilliant place where Max/MSP and Jitter exists. If you are making something you are going to want to play with – something you will want to finesse and adjust and massage.
[You know, I’m making this thing for the iPhone with Open Frameworks. And it’s so painful! If I want to change a number from 109 to 110 you have to hit run, wait, compile. It’s painful! So it’s just brilliant to have a tool like Max/MSP, where you can finesse things so easily.]

So they all have their place, and I’m learning them all. But there are also many that have gone by the wayside. I can’t tell you how many systems I’ve learned, then I say to myself “Oh, it’s not worth it” and realize that it’s not going to get the students somewhere useful. I guess that’s my big push: to get students to a place where they are doing something useful in 10 weeks.

Ten weeks represents a difficult challenge, since you want to get some depth in the things you are teaching, and take them beyond basic tutorial work. One of the ways you seem to deal with this is by teaching the basics, but expecting the students to take the tools beyond your teaching. Have you been surprised by any of your students’ work?

In one class that I recently did, where Max was introduced over two days, I watched someone that wasn’t even a digital media student – someone that wasn’t even that technically engaged – instantly pick up the mindset and understand the possibilities of what to do with it.

I’m still a young teacher and I believe that students, should they have the initiative (and hopefully I can help inspire that initiative with the introduction), have the tools to really “take off” somewhere. I can’t say it’s everybody, but there are always several people that come back with something interesting – I can’t believe they tracked down that little note from someone else, and they’ve transformed it and reworked it. Watching that is amazing.

But this stuff is necessary. The fact of the matter is that if you walk out the door and don’t know code or technical things, you are going to suffer. I’ve never taken classes in this stuff – I’ve never even taken a Max/MSP class! But it’s important to learn to go out there and push yourself to learn more. I think that is the biggest challenge. The moment you have to take it beyond the basic tutorial, and you are having to do something that is transformative and isn’t laid out by somebody else. That’s when you are making something that is yours.
It takes a lot of thought, and includes combining a lot of other ideas – it’s just like a research paper. We all know how to read, and somebody gives us the key books, but you have to take stuff out of all these books, and other books, and combine them into a new idea. That’s when you have a research paper.
It’s the same process with code. And art.

There seems to be a situation where code itself is becoming art, and the line between the two is starting to blur. You are on the forefront of that, with your work using traditional computer tools in non-traditional ways. For example, one of your installations uses a key computer component – the computer fan – in a completely different way. You have put yourself at the forward edge, and now find yourself teaching students that also hope to be at that cutting edge.

How comfortable to you see them with an Arduino board as opposed to a paint palette? And how do you see that changing from year to year?

Right now I’m still seeing fairly significant difference between those who understand code and know how to tease it into interesting ideas, but don’t understand how to tease the ideas in an artistically interesting way. Right now, I think there is still a big problem where people will take time – maybe it’s a problem with our time constraints – people will take time to work out problems in the code, or you can work out the aesthetics with the code, but not many people are able to bring the two together.

I think this is a criticism that is often brought against Max work. With any electronic medium, the question is how you move beyond the point where the most sexy thing you’ve done is the code; where you show of the patch or show off the code, maybe showing off your electronics. Because you are more excited about the technology you used to do the work than you are about your idea.

You do bring up a point: We are on an edge where, when you say “Show me your art…”, if a person has come from this direction, what they show you is their code.

Right. Again, though, I think these tools increasingly allow people that aren’t trained as coders to make conceptually interesting things happen. Max was an early adoption for me; it allowed me to decipher a video image into these datasets that were being sent to a board. Now, I can start to conceive of creating the low-level code necessary to do this, but back then it wouldn’t have happened. So, a lot of these options are coming to the point where you don’t have to have two years of engineering education to get there – things like OpenCV. In 10 minutes, for a demonstration at one of our new student days, I whipped up a program that tracked faces and shot shiny particles out of their heads when they approached the table. And it was simple!

The interest point is that – people will say that Flash has always done that sort of stuff. There have been sexy GUI’s that hide a lot from you if you want. But the argument has always been that what you end up with comes out looking like it came from that program. That’s always been the power of mixing things. How do you make Processing not look like a Processing program? I think we are moving toward the beautiful place where you can code, and you have access to enough power, to make it not look like a specific program.

For now, though, it’s amazing to have tools like Max, Processing and other things to give to my students as a jumping-off point for them to do their work.