An Interview with George Lewis and Damon Holzborn, Part 1
There's a by-now legendary exchange of letters what first appeared in the Computer Music Journal in the 1990s, and was later copied in the publication Array called "The Mins of Max." It was an article written by Henjan Honing and Peter Desain that detailed their views the strengths and weaknesses of "the popular Max programming tool."
It's a fascinating document to read now that we've got Max 8 on our desks doing code export and running Jitter and downloading Package Manager add-ons for whatever our specific artistic practice might be. The article occasioned a flurry of responses, some of which come from people you may know.
It's fun to go back and re-read, but the part of the exchange that really stuck with me was a postscript written by George Lewis, called "Max in a Musical Context." (you'll find it at the very end of "The Mins of Max" letters exchange referenced above). Reading it today is very interesting in that his ideas about what constitutes interactivity in the context of the work of that time. In some ways, very little appears to have changed in large areas of compositional practice.
But that's not the whole story, and George has been involved in the changes in perspective as a thinker and practitioner from the early days. His Voyager program is a case in point: I'm hard-pressed to name another piece of software that's been around, in more or less the same form, for anywhere near as long, and something that's duetted with such a large audience of improvisers along the way.
Of course, the program itself has migrated from hardware platform to hardware platform, and from FORTH to its current existence as a Max program. Over the last several months, the newsletter and some of our blogging has focused on issues related to algorithmic composition, and I really wanted to have a chance to discuss some of those questions and contexts with George. When I approached him, he suggested that Damon Holzborn would be a valuable person to include in our conversation, since Damon was both instrumental in creating the Max version of the Voyager, but also for the way that his own artistic practice intersects with their collaborative work. Here's a record of a long relaxed afternoon's conversation.....
Gregory: First, thanks a lot for doing this. I really wanted to have this conversation as part of our recent newsletter focus on algorithmic composition, even though the Voyager, strictly speaking, is not a composition program.
George: Well, that's an interesting point right at the outset. Let’s hold on to that….
One of the things I thought I'd start with is that there's a real sense in which when people talk about the Voyager, it's described as a one of any number of things. It's a system, it’s a program. It's an environment, and so on. Can you talk a little bit about the way that you've come to see the Voyager?
George: I'll go first about this whole system/composition/improvisation question. Let's go there with it. From the standpoint of the computer, it doesn't know the difference. And so that's a point where you start to think, well, what kind of differences really are there and how do we mark those differences between composition and improvisation? This seems to be something that technology allows us to do. It really problematizes that whole distinction.
Now, back in the days of - let's say, Lejaren Hiller and the ILLIAC and all that - that was clearly geared toward saying, “Well, this thing is composing.” And the reason why it's composing is, number one, it's all out of real time. And then there was a procedure by which it would reject things. So actually, the algorithm would follow rules and input numbers that didn’t fit those rules would be rejected. So right away, there are two sort of markers of what people thought composition was. You don't accept just anything - you accept the right thing.
And the second thing is that it happens out of real time. And whether or not you're using punch cards - there's been a lot of changes in the computing environments and no one uses that kind of punch card stuff anymore - In real time, whose real time are we talking about? Are we talking about the computer’s real time? the people's real time? Voyager never throws anything away. It's one of those Booker T. Washington “Cast down your bucket where you are”-type programs. Everything has a use in some way. In the end, I wasn't able to find any culturally unbiased way… I’ll put it that way… of distinguishing between composition and improvisation that would work.
So my calling Voyager a program that improvises is a political decision. It's an ideological decision. You know, I could say, some people say that improvisation is real-time composition. Well, that could be true, but then there's this idea of composition as a term of art and also one of social location. In that sense, real-time composition really doesn't exist in that way, in that social location where composition is. These are the kinds of things you're grappling with. If it's a composition system, the only problem is that you don't have any control over what it does. You can't tell it to do this or that, You can't compose with it. It's not an instrument. You can't tell it what to do. It's sort of a player. It's got its own instrument, whatever it is - the piano, or the old days it had the synthesizers. So in many ways, it was bringing up a lot of social issues right at the outset around what music was and around what composition was. And these are culturally imbued issues surrounding what I guess is still called sort of Eurocentric or Eurological modes of thinking about composing or thinking about improvisation. So I want to leave it there. There's a lot more to be said.
And Damon, when the time came for you to work on porting the Voyager - to open it up and look at the code for the first time--how did it appear to you? You’re younger than George and I, and one of the things I wonder about is whether my even asking that question about composition and improvisation is bound to the world that I've lived through. It's entirely possible that your intellectual and emotional furniture is completely different.
It could be that you came to your artistic practice without that idea that there’s an impermeable boundary between composition and improvisation and that - as a coder – you saw those questions differently. The first time you opened the Voyager, were you surprised at what you saw?
Damon: Well, it’s been a while, so it’s hard to remember. But first of all, the way we worked was from pseudocode. But it wasn't like George plopped five 500 pages of pseudocode on my lap. He was busy creating the pseudocode, then I'd go off and code some stuff. And then by time I was done with that, he'd have a new pile of pseudocode. So in truth, all I ever saw was one small subset when I started this project. And then as we went, we started adding pieces. So it was as if I was building a house as he handed me blueprints… “Well, right now you're gonna deal with this room, so here's the blueprints for this room. Go do that… OK, now you're ready to do this other room. Go do that.” There was never a moment where I had to get my head wrapped around the whole thing at once – because that's just not how it happened.
That might have been merciful….
Damon: Probably. This wasn't like George handed me some abstract design doc - this was actual pseudocode from actual working code. So this wasn't a thing that I was going to have to find the problems with. It’s not like having a design doc and then you start coding and you start finding all the problems in the design or other coding challenges. This is a different situation where it's, “This is working code, make it work over here instead of over there.”
What was the circumstance under which that started? Was it a specific performance or what?
Damon: Yeah. It was that the Virtual Concerto performance with the American Composers Orchestra at Carnegie Hall. George, maybe you could say more about that piece-
George: It was interesting what you said, because I barely remember myself how I would send you these blocks of code. I tried to send you the blocks according to the way they would be built up as a FORTH program - you start with the very primitive stuff and work your way up and you get to the upper levels of it. I guess the upper level of it in the Voyager would have been what's called “The Phraser” now. But to answer your specific question, Gregory, we had a concert.
I had a commission from the American Composers Orchestra. They were involved with trying to get orchestras to be more involved with improvisation. They said, “Can you make a piece that involves improvisation somewhere?” I said I'd imagine I could do that. Michael Geller, who was the director at that time, is a very progressive guy. And the orchestra can do amazing things. But even in a lot of works, you still don't improvise that much. I was trying to figure out how to do that, and then they said, “Well, maybe it would be great to have a soloist.” So that sounds pretty good. And then they said, “Would you do it?” And I said, no, I don't want to do that because when you play on your own piece, you try to learn your part, you’re constantly worried about how the piece is going and whether the third horn’s a little flat or something like that. And I was distancing myself from a lot of live performance on the trombone anyway. And so I said, “Well, look - how would you feel about this? I do know someone - or rather something - that could do the solo part….” And I told them we we can make a computer program that functions as a piano soloist. And they said, “Hey, that's a great idea!” They went for it, and that's when we started working on it.
First, we had to cut the existing system I was using down a little bit. What you see here is the result of that process
George: There were 64 channels of sound makers in the virtual orchestra version of Voyager, which was being played anywhere from 1992 to this time. And I was constantly trying to get more. I was working on 128. You know, I want to really have as much of a large orchestra as I could. And so we boiled it down for the Virtual Concerto. We realized we didn't need more than six fingers on the piano to get a very full sound. So that was just a really simple matter of just not using the other fingers. Damon was so orthogonal and thorough about how he was translating that I would imagine that the other fingers are still there or could be there because of the way he made the program.
Damon: I think there might be. But because it was a piano version I don't think we started with 64. I think we started with 16 before realizing that it needed to be 6 instead of 16. But I think you might be right. I think if you look in some of the patches, there might be things that count to 16 - a few vestigial 16s in there (laughs).
It looks like there's a there's a hunk of the patching called something like setphrase. As nearly as I can tell, it chooses what instruments are going to play. It groups them together and then gives them instructions about what they do. So in a way, it's kind of thinking about the structure from the top down.
The deadline was already set when you started the port, right?
Damon: Yeah. We had a small number of months, it was two or three months, something like that. George? It wasn't very long….
George: We were creating a piano player… we were creating a piano soloist. The Yamaha Artists Services people arranged things very nicely. We were living in San Diego at the time, so they sent a Disklavier to my house. We spent a lot of time there trying to figure out how the thing would play as we created this piano player – to figure out if we could really have six fingers and then figure out little things that we wanted to improve about the way the piano player played--its touch, in particular, which needed to be more nuanced. It wasn’t just about sending MIDI velocities. I was hearing people doing Nancarrow-type stuff, and when the piano got soft you just heard clicks from the solenoids. To make it so the piano could play super-soft and super-fast at the same time, we made a two-dimensional lookup table that related duration to velocity, based on Bob Willey’s paper on “Undocumented Features of the Disklavier.” The new Disklaviers are much more nuanced than the ones they had in 2004, so I think that feature isn’t really needed now.
There's a kind of verisimilitude that you're looking for. It's kind of like those old Heider-Simmel psychology films where you see these triangles running around. They're just triangles, but they appear to have intentionality, or they seem to have goals or plans. Part of that is attributed by the person doing the viewing. Voyager is a lot like that. It does these things… It does try to manifest certain kinds of intentions. But a lot of it is also attribution on the part of the listener.
What breaks up those kinds of attributions is the thing that Voyager used to do, where it would be playing in a certain way and things were going very well, and all of a sudden, for no reason, it would start doing something else. And when it started doing that on a consistent basis, you’d say, “Oh. That’s just random.” And then that whole verisimilitude thing is lost. There’s this paper by Chris and Uta Frith, neuroscientists who I met in the 90s, where they talk about involuntary signaling as an aspect of biological provenance--every motion you make tells people that you are biological. Even babies notice it. They are talking about visible motion, but the same thing happens with sound. Voyager has to signal through sound that it is a human-like pianist, and the sudden appearance of odd decisions undermined that impression.
I remember walking around every day going to my favorite coffee shop around the corner from… where were you living back then, Damon? Were you on Adams Avenue or somewhere?
Damon: Yeah, probably on Adams by then.
George: You were living around there - didn't you make a record called Adams Avenue?
Damon: Yeah. Adams & Bancroft.
George: So I was walking around there and finally this sort of weird sort of servo/cybernetic thing came to me where the idea was that instead of having things change in the way the setphrase thing that you saw, Gregory - the way it operated was that it would do a certain kind of behavior for a certain period of time--these so-called behavior sets, as I was calling them…. They’re what you can see on the screen. It's like the number of fingers on the X axis and then going down the Y axis are those parameters. That would be the behavior set. They could all be independently specified - every aspect of it. So they could all do the same thing, or all do completely different things. So that's where the higher level groupings come in.
But the idea that suddenly the number would come up, it would do something for three to seven seconds, and then it would do the same thing or do something very different - that started to pall. I didn’t feel like I could take that into Carnegie Hall.
So we developed a different system, which was a series of transitions where it would look at where it was - which is the closest Voyager gets to listening to itself.
It looks at where it is in terms of things like volume or tempo or things of that sort, and then it sort of moves towards some new state – It sort of set a goal for itself and moves toward that state. In the meantime, a new goal might come up because two things can set the goal - if it's playing by itself, there's no input, so it decides by itself. But if someone is playing, the goals get set, in part, by the outside. Not always, but they know they can be set by either one or the other thing. One finger could be set by the outside, and another “group” could be set on its own. Everything is made so that you can group at higher and higher levels, but - at the beginning - they're all totally atomic players. One-celled organisms that play….
Damon: And that automatism, too, goes down to the next level. It’s decided on a per-parameter basis, which makes it able to be clearly connected to the player while also being independent the same time - by dividing up the task with things it’s listening to and things it’s not, so it can be following the rhythm of the player and following that parameter (and the pitches are almost always following if I remember correctly), but some other parameter is going to be contrasting or just ignoring input and coming up with its own thing. You get a mix, so you get a clear connection to the input while at the same time get something that's not just imitating it and you get a rich and complex set of interactions with the player.
A very tempting thing to do on your first encounter with Voyager is to test it – “I'll do this and then see what Voyager does and then I'll do this….” You probably wouldn’t sit down with a human player and think, “OK, I'll play a couple of loud notes and see what happens. Now I’ll play some quiet notes and see what happens….” Voyager doesn't reward that at all - it doesn't reward that any more than you as a human improviser would. If somebody was doing that with you, how would you react to that? There's no way to react. And that's why Voyager doesn't reward that, because neither would you. It's an organic sort of system. And it's such a complex set of interactions that it's not - like George said earlier - you don't control Voyager. This isn't a thing where, like, I do this and Voyager does that. You have to just play and trust that Voyager will listen.
I would say that on the Voyager recordings,<a href="https://www.youtube.com/watch?v=7I9UwLkljDs"> Roscoe Mitchell does a pretty good job.
George: Oh, yeah.
He listens. I mean, maybe that's the deal. You have to listen. And people who are not improvisers in many cases are probably not accustomed to working that way and actively listening. Voyager listens not only to you, but it listens to itself. I know I'm talking to a man with a degree in philosophy, so please forgive me if I get my phenomenology wrong…
George: It was an undergraduate degree (laughs).
…but there's a sense of agency and activity around that idea that I think of as different. And – to me - one of the things that makes the Voyager unique in its ability to respond to you and to itself.
George: Well, you could say that. I would say, in the earlier days, it basically responded to the player. And then, if it decided not to respond to the player, it would just make up everything on its own. So, I mean, it wasn't like - let's say - Robert Rowe’s Cypher. He did this very wonderful thing where he did get the system to really listen to itself in a way that Voyager really doesn't do.
You know, it's interesting to go back to that business about whether people are improvisers or not. I think it's more of a question that being an improviser sometimes means that you get involved in presenting yourself. In music, especially, that may be something that some people do. But the people I came up with… they're very aware of the environment and their place in it and the way those constantly shift, and so Voyager does that kind of thing, too. As Damon said, part of it may be listening to you, and part of it may be just doing its own thing and accompanying you in some way.
It has this kind of very basic modes - for a certain channel, there are three basic states. One is that it tries to follow you as closely as possible - doing what you're doing, following whatever curve it finds in your work. Another thing is that it does the opposite - you go high, it goes low. And the third is that Voyager just ignores you. The ignoring part is very powerful because ignoring is where you get differences – the psychological notion of difference. In other words, you need to have the sense that the system can do something on its own and that it's not locked into what you're doing. That starts to feel - from the improviser or improvisers playing with it - like you're dragging around this great weight and nothing can happen unless you do all the initiating. Whereas if it peels off and does its own thing, then it's like, “Oh wow, was that an amplification of what I did?"
I worked with for a long time with Don Ritter, a Canadian video artist who did a lot of interactive video. We made these interactive video with a pre-Voyager-type system and his interactive system. What we discovered was that his interactive sessions were very responsive. And I said, “Well, maybe they're a little too responsive…. they don't do anything unless I do something.” So Don built this whole superstructure where it would do things on its own. We didn't think that would work in the visual domain, but it turned out to be very powerful. There are some online pieces on Vimeo - early pieces we did.
This would have been in the late 80s or something, and some stuff made after Voyager was made in ’87. We worked at the M.I.T. Media Lab for a while on these things. He’s been doing lots of wonderful things since then.
Do you think that's partially a consequence of the idea that the benefit you get that derives from the attention of your audience and the sense in which they complete what you give them, because – essentially - we're machines for making sense. We really want to make sense.
Damon: People tend to underestimate how easily we make connections. I think that realization is a key strength of Voyager. I remember working on music with Hans Fjellestad for our friend Mark Latiner’s play the year after I finished my undergraduate degree, and we just did it pre-recorded. It was roughly timed. When we watched the play so much just sounded like a score to a movie - right on the hit. We were up in the back just laughing because it seemed like we totally planned this all out. That was an important illustration early on in my creative life where I realized that you don't have to work that hard for people to make connections. With Voyager, you have this combination. There's always some parameters that Voyager uses to make a connection at any given moment, so Voyager has plenty of room to do its own things with other parameters.
My recall may be off, but my first memory of ever seeing the Voyager was at an ICMC. There was someone on the program who was supposed to perform, but their rig wasn't working correctly. Someone had prevailed upon you to step in and do something in place of them. I remember watching you get up on stage – you had an IVL Pitchrider…
George: That’s right.
…and your horn and a computer, and you just plugged the thing in, turned it on, put the trombone to your lips, and off you went. And the thing that I most strongly remember was you taking a pause. I could hear the thing pausing for a moment, switching gears – I suppose I might want to say that maybe that was me wanting to make connections, now – but some part of my brain lit up and I thought, “Good God, this thing listens to itself. If George isn’t playing, the Voyager takes a solo! Why the hell doesn’t everybody do that?”
George: Well, not everyone wants their computers to improvise.
Do you know the work of the League of Automatic Music Composers? For me, this is where that whole attitude comes from. David Behrman took me to a rehearsal with Jim Horton and Rich Gold and John Bischoff and …maybe Tim Perkis. This was 1977 or 1978. I was watching these people and they had these KIM-1 microcomputers, 6502 microprocessors and all assembly code. And they had all their programs on a little local area network they were using – they wrote it up in the Computer Music Journal – and it sounded like a quartet of people improvising. It was a quartet of computers making stuff up. And they had this thing that I read about later where they would just let the network play and they would just stand back and the computers listen to the other computers doing what they did. And I heard that, and I said, “This is what I want to do. If I get to play, that's cool, too. But the idea is that it should have something of its own to contribute. And that's been going on from then until now. I don’t know why most people don’t do it. I just decided to go to extremes with it. It's not like it's an innovation - it's just like a preference.
Part of that lack of preference might be either a relative underrepresentation of improvisors out there or the idea that we still privilege a very narrow view of compositional activity, and that there's not necessarily an incentive for someone to explore that or the extent of their exploration is going far enough to find something that provides them with useful material for their composition. You write a system, turn it on, have a cup of coffee and listen to what the computer comes up and repurpose it… the computer plays the generative part of the genetic algorithm, and it’s still “my work” because I’m the one choosing what to exclude, to go back to what we were talking about at the very beginning.
George: But I did it in a different way. There's no point recording it. You can record everything if you want to set that up - I mean, it's easy now. Back then, it would take too much memory, you know? Those KIM-1s were a thousand bytes (laughs)….
I was looking in my parents’ attic and I found all my old notebooks when I was building “The KIM and I.” I could send you some of that stuff - I took pictures of it. It's just basic assembly code to do different things - to play notes or do accelerandos or whatever. But I would listen to the thing play for hours and I'd say “Well, that didn't sound too good. Let's stop it. What can I do to get that to sound different?” So I’d get it to sound different and then listen again. And some reason it sounded worse, and then you realized that you had to take the good with the bad - you try to make it so that most of the time you like what it does, and some of the time you have to grin and bear it. But you just have to figure out, “Well, maybe it's not as bad as I'm hearing it, or maybe there's something about this, or maybe someone else would like it even if I don’t like it….” It's a sculpting process, a real-time kind of sculpting process.
Damon: I was thinking about this recently in the context of my latest release - Alternator I - which was a modular generative patch. And it occurred to me a few days ago that there's actually a lot of similarities. Part of it is very different - the modular patch is only good for one thing, basically. It does that thing and then I tear it apart. And it’s meant to run by itself, not interacting with another player. But there's also that sense that it's a lot of very simple interactions that form a complex whole. So there's an LFO doing something, but it’s in a complex web of cross-modulation where there’s something else interacting on it and then those are acting and this and the other. What that brings me to is that actually this type of interaction model that we're talking about is happening often, but not always with computers.
I think that how I worked in the Alternator release is typical of a lot of people doing generative modular work. They have this monster synth – well, mine isn’t a monster – but they have this monster synth and they set up a complex web of interactions that runs to make the performance. And that actually, to me, sounds similar to how Voyager works. There’s a lot of similarities with that and what we’ve been talking about, whether it's Behrman and the KIM-1 crew or the modern version of Voyager, where - rather than trying to get some overall plan using machine learning, for example - you set up a complex web of interactions and then trust that that's going to produce something meaningful. I think the modular people have definitely learned or relearned that - you hear that stuff all the time, and I think that's what that's pretty much what's happening.
George: I remember seeing Alvin Curran. He would set these things up with his Serge modular system, set something in motion and it would just run by itself forever. He would say, “I make organisms.” Very similar to what you’re describing.
So the configuration of the organism is an instantiation - a modeling of his thought process. Same with Damon’s Alternator. But I’m thinking about this in terms of one of other things that I first heard about the Voyager back in the day at Sonology with Joel Ryan was that there were ideas built into your earlier work that model the behavior of specific people that you knew. I recall hearing that there were four of them. I can probably do three of them – I know that Douglas Ewart was one of them, Joelle Léandre was one, too. And the whole reason that I went to hear Derek Bailey perform for the first time was that Joel Ryan told me that Derek was one of the four, but I don’t know who the fourth person was….
George: Well, if you're talking about the group that played Rainbow Family at IRCAM in 1984, Steve Lacy would be the fourth.
But what really became a crucial part of Voyager was Joel’s idea about multiple “clocks” that can operate asynchronously, which was a part of his own performance system. It’s a very supple way of handling duration, and in the original 64-channel MIDI version of Voyager, spread out across eight audio channels, it allowed the system to establish several “ensembles,” each with its own rhythmic identity and general behavior. In the pianist version, you could do this, but there is no notion of “handedness” in the Voyager pianist.
When you think about the idea of a program that kind of is related to their behavior as you understand them as improvisers, what does that mean?
George: There’s a funny thing on the Web - people were very excited about it as some sort of note that got out. It was supposed to be Thelonious Monk's instructions to the band. I saw and I said, “That’s Steve Lacy's handwriting.” That's probably Steve taking notes of what Monk said when he played with Monk. One of the things that Steve wrote down was that you have to make the band sound good.
So how do you do that? Well, you try to listen and compliment and provide an environment that people can move around in - something that's compatible. You figure out degrees of compatibility. You take standpoints, different kinds of standpoints on what people do. It's not like you model what they do - I don't do that. But what you have to do is you have to think ahead. You think, “What if somebody does this? What should the system do? What are some of the possibilities?” Maybe it's like plays in basketball – what if you go to the basket and you have two people over here…. What are you going to do about it?
So that's what I try to do. I try to make it so that Voyager has a lot of different ways it can respond that seem plausible to what someone is going to do. And so in order to do that, there have to be a lot of different possibilities you have to build in for what's possible, it can't just keep responding in the same way because, one, the musicians will get bored - they'll hear that right away because they're very experienced. The second thing is there's always going to be a kind of rigidity where the musicians say, “Well, it just doesn't understand what I'm doing.” That can happen with anybody. It could also happen today. I played it once with John Oswald, and it didn’t work out at all. Not because of John – he’s a great player and great improviser. He just has a very different way of thinking about the world than what I was doing with Voyager. So I built it for a certain group of people - like Derek, like Steve, like Joelle and Douglas and people who worked out of those traditions.
I think of a lot of people over the decades, and there are people that don't operate like that at all, you know. So that will be a challenge to figure out how. If, say, John Zorn started doing his theater of musical optics, how would that work with Voyager? I think Voyager could handle it today - I don't think so, years ago - probably not. There's an account of a Voyager performance with Jon Rose in John Corbett’s book, Extended Play. And actually, I did change the program around to try to make it work better with Jon, but it really wasn't going very well then.
So when I do change something around, that's a challenge because Voyager should really change itself. It should just say, “Oh, this what's going on, and I'll do this….” But if have to sit there and go, “Oh, I’ve got to tweak this,” that means I didn't account for some possibility, which is clearly present.
___
This is the first half of an interview with George Lewis and Damon Holzborn about the Voyager project.
You can find the second half of that interview here.
For Further Reading
There are some interesting and informative pieces on the Voyager and George's work in general that you may find useful (They're the places I've gone for edification and clarification myself, along the way). Here are three great places to start:
The best way to start your background reading is by hearing from George himself: Too Many Notes: Complexity and Culture in Voyager (originally published in the journal Leonardo) will get you started and give you a lot to think about.
Paul Steinbeck's George Lewis' Voyager is a great introduction to how the AACM and George's work on the Voyager relate to one another, with some interesting insights into how the Voyager 'worked.'
George E. Lewis: The Story's Being Told is a genial and wide-ranging interview with Trevor Hunter that ran in Newmusicbox that you'll find enlightening.
For Further Listening/Viewing
by Gregory Taylor on July 7, 2020
