Imagining New Environments with Max/MSP/Jitter
When you think of multimedia technology you think mostly about the technology. When you experience Dana Karwas' work you think of the rich organic layers of experience. Dana is working in the nebulous grey area between art and design. As a trained architect she is commissioned to do design works for giants such as Knoll, yet as an artist she creates amazingly tactile and organic performances like her work Party Dress and the installation Fursicle. Although based in architecture, Dana’s work uses high-end technology such as Max/MSP to explore social interaction and levels of identity within public space.
I'm really interested in what you do. You have an undergraduate degree in architecture and then a master's degree from the Interactive Telecommunications Program (ITP) at NYU.
What is that program?
It's a multi-disciplinary program that incorporates new media, video, programming… any type of digital media. It's all about collaboration with art and technology. People are learning all the new programming tools, as well as all the new physical computing tools. The projects range anywhere from small robots to experimental cinema to moving sculptures. It's basically interactive media and collaboration, so you work a lot in groups, and then you have these art and technology projects near the end.
That sounds really exciting…
Yeah, it really is. The students come from all over, all over the world, as well as from multiple backgrounds. You'll have physicists paired with a writer, or an architect paired with a programmer. There's this incredible opportunity for collaboration to happen.
I went to an interactive art program, as an undergrad in the '80s. It was wonderful, because you create your own thing, and do pretty much whatever you wanted. Like, "I think I'll make a film today."
Exactly. There's a lot of freedom… and imagination is huge at that program.
It's about your ideas, as opposed to the craft.
Yeah. So it's about making prototypes, and just exploring new things. Most people that I know that go there end up having so much fun, just in the classes. "Oh, we'll take a class about architecture. We don't know anything about architecture, but it's here." So they get the opportunity to take different routes, and explore being different types of artists, you could say.
Let's go backwards a little. What led you to architecture school? Were you an arty, crafty kid?
I was more of a math and science kid. I was really fascinated with architecture in regard to looking inside people's houses at night. I was always curious, like what kind of space they had. As a kid, I would always be looking, going on walks in the neighborhood, and looking inside these spaces that were different than the space that I was familiar with. I really liked going over to my friends' houses and seeing their rooms, where they lived… it was really people based. These houses themselves weren't really that interesting, it was actually more about what people were doing inside of them.
When you started college, did you think you wanted to be an architect?
Yeah, I knew that I wanted to go to architecture school while I was still in high school. So I had the opportunity to take some classes in high school. I focused more on math, I guess because I was told that you needed to be really good at math to be an architect. though I also was very artistic. So I would do the art classes, but I saw the math as more of a pull. I would do drawings of floor plans for my parents, or redesign the basement. It was very residential based at that time. I had no idea who many of the even basic architects were at that moment… I did know who Frank Lloyd Wright was.
Tell me about your experience in architecture school. Were you leaning towards being a traditional architect?
No, not at all actually. I went to school in Kansas, which I'd say is more of formal education in architecture proper, where you're learning about structures, the history and theory behind many of the famous architects you see. I would never get good grades in studio. I got a D on my first model. I had no sense of craft or scale at the time. But I still really liked it; I knew that that's what I wanted to be. I didn't really fit into the conventions of the school, until they introduced me to the computer. It's a five-year program and the computer is introduced in the fourth year. So I started designing with the computer, and making animations, which allowed me to create architecture that didn't exist. That was really exciting, to make these spaceship-like places, and fantasy dream projects, as opposed to building ‘a house’. The computer allowed me to express my interest, which was architecture and film at the time.
When you graduated, did you go straight to grad school? Did you work in the field?
The interesting thing about Kansas architecture program is they have an incredible study-abroad program. So one summer I got to spend in Venice, and then another summer I got to spend in France, just working for architects. It was in Venice that I saw the Art Biennale, where they had some interactive exhibits, and I thought to myself, "I want to do those." So that's why I went to grad school for interactive media, because I liked the idea of informing my curiosity about architecture through cinema, and also manipulating space with light and video. That was really cool to me at the time.
You're work is so interesting. I was going through your web site. I love the dress piece, Party Dress.
Oh, yeah, ‘The dress’, that's one of my more recent projects--we took a very practical approach on that. We were designing it in 3-D, trying to be really systematic about the process, but then, when it came time to actually fabricate it, it was like a craft party. We were sewing with one machine, and pinning stuff. The fabrication of it was very hands on, the design was very tech-y. I think we lost some technology along the way, but the piece itself was a blast… five women wearing one dress.
Did you enjoy the design process more than the fabrication?
Actually, I liked the performance of it the most. The fabrication was just kind of like, well, maybe we should have hired someone to do this. But it was also really funny because we didn't have any money; it was just my sister and I, sewing this giant 250-yard long dress. [Laughs.] It took us a week… but in the end, my favorite part was the actual performance of it, the chaos of implementing this scene inside of New York, and having it transform. Seeing how people react to it is pretty cool.
Let's talk about Max/MSP/Jitter. When did you first start working with Max/MSP? How did you know about it?
I saw Luke DuBois' show, at ITP. He has an end-of-the-year show where he has all these crazy projects on display. I was curious because it involved people manipulating video, and then projecting it onto an interesting space. There was some relationship with video that was unexpected, where it was going. Someone was projecting on a curved wall, and I thought, "Wow, I want to learn how to do that." So it's really at ITP and through Luke's class that I was introduced to that world.
Tell me more about ITP.
That's the grad school, Interactive Telecommunications Program at NYU. When I saw this show, I was really blown away by it. The accessibility of programming, and also what it looked like to program. It's very visual. I could build little machines in Max/MSP/Jitter that would do what I wanted them to do, without having to write code, and processing, or using Java. So I was drawn to the interface right away. The first thing I ever did with it was I did like a VJ event at The Crowbar, a club in New York. I was just manipulating video, doing VJ stuff. I was very practical with it at first, just manipulating video, but I liked building in it, I felt like it was very architectural, the actual process of building a patch. Visually, it made perfect sense to me. It was very intuitive, coming from an architecture background. We're very systematic with the way that we go about either building a model, or putting together a rendering that made some link to the way that I think architects design. So that's why I loved it… I still love it.
Did it take you very long for you to get the hang of using the programs? What was your learning curve like… did you lock yourself in a closet for a week?
I kind of fumbled around with it for a few weeks, but then, because I had this event, this deadline, and a project that needed to be done. I just locked myself up for two weeks and figured out how to use it.
What's the first piece that you did with it? Is that on your web site?
I did a piece called The See-Through Wall, I'm pretty sure it's on my web site. It was a wall that, well, that you could see though [laughs]. So that's really the first piece that I used Max on, which was shown at the Chelsea Art Museum… it was a lot of fun. I collaborated with Gabriel Winer, a filmmaker. We made this see-through wall, which had a projector on one side and a camera on the other. We got to calibrate the space, so it was kind of like live programming while we were setting it up. That was interesting to me, because we were able to tweak the art piece at the museum, while we were setting it up. So that was quite exciting for us.
The See-Through Wall
So there's this real-time immediacy about Max/MSP and Jitter that was working for you.
When you have an architectural project, there's this incredibly long process that is almost completely removed from the end product, whereas in new media, you can get instant results and manipulate spaces and forms on the fly, which, for my fast-paced life at the time, was very exciting…. I liked this immediate feedback.
It's so much easier than the days of shooting film and having to wait for it to be developed.
Oh yeah. So the first piece was The See-Through Wall. Then I became interested in using it as a tool to connect other applications or devices. I was using it as a way to connect cell-phone text messages to visuals. I would translate a text message into a hip-hop rap, and it would be like a dancing music video.
That one’s called Freestyle. I kind of wanted to give a space or a place for a text message that was outside of the phone. Basically you could send Freestyle a text message, it would remix it into a rap or a hip-hop tune, and your message would be rapped back to you with a beat. Then the picture or the video that you send with the text would be manipulated, so you'd get this packaged, mini hip-hop video that would move and dance to the beat of the music and to a text message. They would be in computer voice, and I used the built-in voices, the voices that are on the Mac, that come with the program--I think the Mac OS has like 21 different voices. So Eric Singer wrote an object, that was text to speech, or something of sorts, and I just played with a bunch of different objects, and managed to make this monster of sound and image and cell phone data. It was cool. I became very familiar with the Coll file.
What does the Coll file do for you?
It is able to list the text for me in a certain way, and then I was able to pull from it what I wanted. I had it in a typical museum setting, so I had to censor the text messages, because people would send inappropriate words. Using a Coll file, I was able to put all the bad words in that people could possible send, and then it would skip, and just give a beep, instead of the word. I had to use it as a form of censorship, in a way because there would be like high-school kids coming in. Freestyle was at the Museum of Television and Radio, so they'd get a massive amount of foot traffic from schools. The kids would see it, and of course they're completely familiar with text messages, so they would quickly catch on, but there would be a bunch of beeps and then they realized that they were being censored.
What are some of your favorite objects?
I used an object that allows you to connect to a physical device, or multiple other devices. I think it’s called Serial… I'm not very good at remembering the names. And then jit.qt.movie is my life; I kind of live within that world. I'm very simple with the objects. I try to keep it extremely simple, so it doesn't get too out of control. But all the cv.Jit plug-ins, for tracking, I've used. I know that there's Blob, cv.jit.blob, or cv.Jit … so any of the cv.Jit objects, that whole package. I use those often when I teach Max, to allow students to quickly grab something from my web cam and translate it into something else. [cv.jit.]Faces is probably one of my favorites. It's the one that lets you show the many faces that are looking at the screen. Everyone always gets a kick out of that one. There's one that's another external that you have to download, I think it's called Bonk~. I don't know if it's an external, but I like the ones that have the fun names, because I think they're really funny. We were going to make one for Luke called jit.dubois--we're still trying to figure out what that would actually will do--all of his students want to make it. And then there are any of the openGL objects that I really love as well.
Have you ever written your own objects?
Not really… I'm trying to figure out what they would be if I did. I think it would have to be some type of building generator. You'd plug in personality traits about yourself and then it would spit out some type of form that would potentially be your house, or some object that would represent you in 3-D. I like messing with it.
That would be really cool. I was looking at your piece, Fursicle.
Did you use Max/MSP in that?
Yeah, there's a lot of Max/MSP in Fursicle it's kind of like an explosion of Max/MSP. [laughs] everything that I know how to do. My collaborator, Liubo Borissov, who currently teaches at Pratt Digital Media, and I realize that there's just so many possible connections to other programs. We connected Max to Bluetooth, had a camera picking up movement, another camera that was taking photos in the room that would catch people… like petting the carpet. You had to call the exhibit to get in, so we'd capture you're phone number, and then the exhibit would call you at 5 in the morning the next day and make animal noises for you.
I love that! What I like about that piece is that it's so primal, and yet it's high tech.
We wanted to go primitive with the actual objects, but make it this insane--this explosion of technology. So we thought of every possible thing we could pull off in like two weeks, putting this together. It involves PHP, and some other programs as well, to actually get the phone system to work. We had a number you could call--you would actually call the Fursicle--and it would give you instructions on how to go through the piece. We were projecting images, also. Zebras, and tigers, or whatever you would be stepping on, you would see it projected, and you'd also hear it. It's one of the more silly ones. I like that one.
Have you tried any alternate controllers, such as a Lemur surface, or anything?
Yeah. I had a Lemur, from JazzMutant, I played around with, but for the Fursicle piece, we actually ripped apart a Bluetooth keyboard and just scaled it up, and made switches that were speaking through the Bluetooth keyboard, or what was left of it, to the Max patch. I liked using Bluetooth in the past. I usually use a cell phone or Bluetooth that I kind of get around making my own controller. I used Arduino before, to send information from Max. It's a micro-controller that you can send information to from Max, using Serial. It's one of my favorite devices to use, to take Max/MSP/Jitter, and have a physical presence, either through lighting or sound or moving a motor. The Arduino would be what I would use to get out of, or send to Max/MSP, from the physical world. So it's just connected with a USB printer cable to this micro controller. And again, it goes through the Serial object. One of the things I find really interesting about it is you can so easily send information to other programs, or other devices.
So once that happened, it really opened things up and was really exciting.
Yeah, it really is. I've taught Max to architects and designers, and I find that the architects catch on so quickly to the programming and to the interface. I've taught it up at Columbia for a year, and those students learned it right away. I mean, it took them maybe like two weeks, but they really adapted to the programming environment. And their patches were gorgeous. Very organized. [Laughs.]
Eric Singer, he owns this place called The Lemur Complex, and I frequently taught classes out there, and one of them was called Max for Chicks, and it was kind of a cosmopolitan approach to Max/MSP Jitter. I think I had eight students; most of them were men… which were ironic in itself. It was a complete blast to teach it, because I approached in a holistic way, based around example. I would show a QT Movie, and then I would dissect it. As opposed to showing, "This is an object, this is a patch." It was more holistic… and was really fun.
What would you like to see Cycling '74 add in future updates to Max/MSP/Jitter?
I would have to say--this is going to sound a little insane [laughs] but I would like to see some type of further connection ability for the architecture community. I think if it was able to connect to something that is used as a tool by architects, such as AutoCAD, to generate plans and sections, a very specific type of document. If Max/MSP could be used in such a way to make it easier for architects to make changes on that document, that would be very cool. It's like a dynamic design tool, for a house, or a building… whatever.
So that you could use voice control through a headset, kind of thing?
Yeah, to easily make changes, so you could control the design, the details of the design, and the material through the interface of Max/MSP, which would allow an incredible amount of time to be saved in an office. So if you're making some crazy, curved building, and you have to change one of the curves in the drawing, it will have a domino effect in all of the drawings, which takes a very, very ridiculous amount of time to redraw now. If it could just be some simple way to use it as a tool for producing drawings… I would love to see something happen in that regard.
That would be exciting. It kind of reminds me of that movie Iron Man.
Right, right, something like that. I think Max is a really incredible tool. I'm a big advocate of it. Actually I'm kind of a big saleswoman for it, as well [laughs]. I see it as a tool that, if you think of something you want to do, you can probably figure out how to do it with this, because you're basically building a little machine… I tell everyone they're building their own machine, that's been how I view it. I think there is a way to make it more accessible for architects, to be used more in practice, as opposed to art projects. I know composers use it, as a composition tool for music, it's really incredible, it's very integrated into production and whatnot. But there are very specific tools in the architecture community that are standard, so some type of way to update the standardization of the architecture tools, because they're not as advanced as I would like them to be.
There are some programs that are doing that currently. I have a friend, Gil Akos, who was a student of mine at Columbia that is using a program called Grasshopper. He's doing something similar, trying to refine the tools that are used in the architecture community through programming. He discovered Grasshopper through Max/MSP, and his design philosophy is based very much on Max/MSP/Jitter, in the architecture community. He had a very interesting project for using Max: he used it to control a moving wall. It would respond as sound, so the pattern on the wall would change, and be articulated based on the sounds the microphones that were embedded in the wall would pick up, and Max was controlling the whole thing. The architecture community really likes it, one, because of the interface, but especially because of the simple feedback. It's a very intuitive program for architecture students to use. It always has a great response in that community.
So you use Max professionally for design projects and proposals as well?
Professionally… yeah I do use it. I guess the most substantial project was a project, Knoll Wa, for Knoll, in France. It was cool, because we got to bring the program and introduce it to the furniture-design community. We had a two-story projection on the side of a building. We wrote a program that would edit the scenes randomly. So if you were walking by this exhibit every day, you wouldn't see the same thing. We used it as like a live editing tool for the exhibit. That was basically what got us the project, this idea that it's not only a movie, it's dispersed over two floors, and we have this knowledge to write our own program for it.
And it was an original film?
Yeah. We shot the film in New York. They shipped us two tons of Italian Knoll furniture, and it arrived, and we shot in Boylan Studios, which is up the street from where I currently work. It was a two-week project, and then they flew us over to Paris, and we automated the system. We thought we were going to have to go turn it on every day, but we were able to automate that as well, again through the Serial objects, to turn on these 10k projectors every morning. So we ended up just having fun in Paris, and not having to worry about the project, because the whole system was automated. We stayed for about a month, in anticipation of turning the switch on every morning, but then we realized that using the Serial object, we didn't have to. With the help of Luke and Liubo, and these other serious, Serial programming guys, we figured it out the night before it opened. So we were very thrilled. The Serial object gave us a couple of extra weeks.
So that was a professional project, more of a design commission than an art installation? Even though some people probably can't tell the difference…
Right. They wanted us to have it as a spectacle, like this crazy, new thing for the launch of the new furniture line. So they brought us in to not only do the film, but also create the environment for that one particular evening. It was cool. It was really cool to be able to use Max at that level, and get paid to use it, and see it integrated into the design field, and have people ask us questions about it. It was very cool.
It’s fascinating to hear about the different applications, especially outside the typical ‘art scene’ that Max/MSP/Jitter is being used as a tool.
As a tool, I see it having a ton of potential. Not only in the architecture community, but all across the board. The reason why I appreciate it so much is because it was very simple to grasp and catch onto. The instant feedback is one of my favorite things about Max/MSP.