An Interview with Keith McMillen

K-Bow

Keith McMillen Instruments recently impressed all of us at NAMM with demonstrations of a new pair of string performance devices, the K-Bow and StringPort, both of which include some very rich software applications written in MaxMSP. The K-Bow, a bluetooth-based wireless gestural controller integrated into a violin bow, has just started shipping so we thought it would be a good time to catch up with Keith and find out more about the project. I met Keith at his studio in Berkeley, CA, home base for Keith McMillen Instruments and the BEAM Foundation. When I arrived, Keith was looking over prototype boards for a new violin pickup with Joel Davel, who also designed the boards for the K-Bow. Once they wrapped up their discussion, Keith sat down and gave me a thorough tour through the technology and ideas behind K-Bow, with Max programmers Barry Threw and Nick Bonardi contributing details about their involvement in the project.

How long of a process has this been between working on the hardware and the software? How did it all begin?

KM - It's been about three years. I woke up one day and, how all great projects start, was like, 'how hard could that be?' Of course, if you knew how hard it was, you'd never start in the first place. And then you reach a point where you can't stop. Right now to be a modern musician, you have to be a performer, programmer, and composer all at once. There are really very few people who are good at all three. I wanted to make a system that someone could walk up to, that's all been tested, and is really extensible for whatever you want to do with it.

BT - I have been involved with this project since conception. It's been three years I've been working with Keith now, including the work I did for TrioMetrik.

So did you have a detailed plan for the K-Bow, or did it evolve over time?

KM - Well I knew the gestures that I wanted to capture, and then there was an experimental stage that involved those sticks [points to some dowels with wire running down the length] and a white block, figuring out what is possible. Then I started making a few boards - the first one just had the bluetooth transceiver, the accelerometer, and the grip and hair sensor. The K-Bow now consists of two very tightly packed boards: the "emitter" which sits under the bridge and, and the bow itself which handles all the bluetooth communication. The bow circuit gathers all the sensor information, communicates via RF and IR with the emitter, makes a tidy data packet and sends it to the computer.

Was bluetooth part of the project from the beginning?

KM - I experimented with ZigBee, and at the time (3 years ago) I liked some things, but it wasn't that standardized and I was concerned about the long-term longevity of the technology. Also, bluetooth is included in every laptop nowadays so it seemed like a good place to start.

Right, with ZigBee you need a special serial interface board, and it seems to still be pretty DIY, whereas you can pick up a bluetooth dongle at any computer store or just use the laptop's built-in bluetooth.

KM - Although, a footnote, the bluetooth transceiver in your laptop is optimized for a mouse or keyboard, and the little $20 USB dongle will get you a much greater range. Also in that spirit, all of the charging hardware uses a standard USB plug found on any laptop, although it does ship with some high-power chargers as well.

That brings up an interesting technical issue. It seems like power management is pretty important in a wireless device, since you can't depend on plugging in to bus power. How much of that is built-in to K-Bow?

KM - There's 4 chips on each board that do nothing but power management. Everything runs off of a 5.6 gram Lithium Polymer battery, so there is a charging regulator, and also chips that make sure that no matter how much charge the battery has, the output voltage stays the same. The K-Bow also constantly transmits your battery charge levels to the K-Apps application, so you always know how much run time you have, and you can configure the bow for different battery performance settings. After a given amount of time of inactivity it will go into a low-power sleep mode, and then eventually power down. After it gets down to below 5% battery power, it automatically shuts down to keep the battery life up around 1500 charge cycles.

So, how did you decide to make a violin bow? Did it come out of your work with TrioMetrik?

KM - Exactly. And there is just so much information that is generated by a bow when you are performing. I had done the Zeta violins and I figured it was time to do a bow. I had also been working with Max since 1989, developing this sustainable performance system, and starting over probably four times. For TrioMetrik, our performance system ended up having about 250 user interface patches all nested, so I had a lot of material to pick from. The problem was, the only system existed in this room, so none of the players could do much to rehearse at home. Also, we were doing so much with foot controllers that it was just distracting. So, we had a need for a better performance solution, and we also needed a source of funding for the work the BEAM Foundation is doing to promote computer-based composition.

BT - There is a problem in electronic music performance which has two sides. One side is that there is no real interpretive performance of electronic musical works. There has to be a system for performers to load other composers works and replay them. The second opposing side is that western classical musicians don't have access to computer technology. We have many performers that have spent years building virtuosity on their instruments but they don't have easy access to the open sound world, and musical communication, that a computer provides. The thing about technology is that it is always the metaphor for how we view ourselves. We need a music that uses the tools of this century building upon the foundations of the past.

The KBow is a step toward solving this problem by bridging the gap between what is often thought of as the base of western composition, the string quartet, with the computer music technology that has been created in the last 60 years.

Besides you, how many other people have been involved in bringing the K-Bow to its current state?

KM - Chuck Carlson, whom I've worked with for over 20 years, wrote the firmware. He's put a lot of work into this. A lot of the patches were functioning already because they were pulled from existing projects, but Barry Threw came in and started to put it all together and making it into a consistent system. He's very persistent and patient. It couldn't have happened without him. Nick Bonardi then came on board and did all of the graphics and user interface work. Joel Davel, who you met earlier, did the circuit-board layout. All in all about ten people have contributed something.

BT - I have coded most of the software for KBow (excluding firmware), including Max patches, the custom externals, and various other parts. There are sections that were modeled after the TrioMetrik work, but many things in the KBow project also had to be rebuilt from the ground up based on earlier ideas.

What does the manufacturing process look like for the K-Bow?

KM - It's diffuse, in that the boards are fabricated in India, and then loaded in Fremont, CA.

Then, in El Sobrante, the bows themselves are built by Dan Maloney, a luthier I worked with at Zeta for many years, so the bows come back looking really nice. We also have a machinist, Jon Anderson, who builds all of these custom parts, like this titanium screw on the end of the bow that saved us 2 grams of weight. One good example of the collision of technologies needed, Dan and I had to do a month-long research project on how to glue horse hair to titanium. When the bows come back from Dan, Joel will run the quality control and I will test each one before they go out.

So, let's have a look at the K-Bow software.

KM - Well, it's a standalone application written in Max. Here is the main Application window that we call K-Apps. At the top there is a main preset recall menu. This allows you to load a whole set of presets for the entire application. Each of the windows in K-Apps also allows you to store and load presets for those settings as well.

From here you can call up the K-Data window. This gives a graphic display of all of your bow controls. All of the controls can also be calibrated by opening the Calibration window and setting minimums and maximums. You can also store different presets to accommodate the techniques required for different pieces. Some pieces are very delicate and you don't want to have to engage the full range of the sensor.

BT - The preset storage system, and the communication in this patch are all based on pattr. This allows some nice perks like being able to access any parameter in the patch at any time. The modulation screens are the best example of this - the destination menu contains every parameter that is addressable in the patch, and it is all dynamically generated. If I were to add any random new control, it would show up in the menu and able to be targeted by the bow without any extra work.

Great, so give me a tour of the different controls the K-Bow offers to a player.

KM - First we have bow hair tension - as you press the bow against the string we get a reading from that. Then, of course we have our 3-axis accelerometer that tells us how fast the bow is moving in each direction. The next thing is an infrared proximity sensor that tells you where on the bow (length) you are playing as well as distance from the bridge. There are cones of infrared light coming from the emitter that are picked up by the sensor on the bow handle. That's accurate to about a quarter of an inch. And then there is bow tilt and grip.

That's really not bad. I would expect the infrared and accelerometer data to be a lot noisier looking from my experience. It seems pretty well-conditioned.

KM - I killed myself over this. It's a 12-bit analog-to-digital converter inside of here, and there is tons of signal conditioning before it even gets there. I think there is something like 20 Op-Amps inside of the bow. When you power it on, the sensors all auto-calibrate, and the signal quality is actually really good. It's accurate to about 0.1%. Now if we open up the Time Graphs window you can see a timeline display of the K-Bow data.

That seems useful. How do you see this working in practice?

KM - Well, it's really good for impressing people [laughter], but maybe you want to isolate which parameters are going to be useful for a particular piece. You can turn on just the X-axis for example and start playing, and then maybe you realize for that piece that the Z-axis will give you more useful information. The timeline allows you to really see what's going on with the data. It's also really useful for training or practice for someone just getting started. It can be very overwhelming to have all these controls, so we usually tell people to just start with one, make a wah-wah effect with the grip or something.

The user interface looks very cohesive, and I can't spot very many standard Max UI elements in there. Nick, what was your process like for creating the graphics?

NB - As a long time wannabe violin player, this is just another reason to tolerate my initial steps into the string world where every noise I make is reminiscent of small mammals dying. I started working with Barry and Keith in November of last year. I was brought on to design and implement the GUI initially. At that point concepts were laid down, but there was still much to be done. As the project progressed I ended up co-programming along side Barry. The most challenging part was crafting the user experience to make it both straightforward and flexible. For the graphics, Flash and the Photoshop "contact sheet" process are essential for creating custom knobs in Max 4.6. I'd usually animate everything in Flash, export a PNG sequence, then line them up using the contact sheet process in Photoshop.

So I notice all of these are continuous analog controls. How do you handle triggers?

KM - Well, let's open up the Triggers window. Inside there you can create triggers based on the accelerometer or grip data, and you have controls for the amount of sensitivity and there is an articulation control to desensitize the opposite direction after something gets triggered. This helps keep you from accidentally triggering things. There is also a certain amount of hysteresis, like with a comparator circuit, based on the sensitivity setting. All of the data from K-Apps can also be sent over MIDI or OSC to other applications.

Okay, now that we have sensor input, what can we do with it?

KM - For example, let's open up the K-Tone window. This has a full set of essential rack effects - dynamics (using the OMX objects created by Keith), pitchshifting, filters and delays, etc. Now let's get into Modulation. This is really where all the fun happens. We have lots of these "mod-lines" where you can take any source, scale and offset, run it through a table (or create your own table), and pick a destination. One other thing that makes this more of a performance system is that one of the destinations is the 'mod-line' itself, so you can create conditional layering of your modulation. For sources, in addition to the direct sensor data we also have things like overall playing speed or movement, or gesture recognition sources. There is also a looping window and a phase vocoder that allows you to scrub through a sample using bow position as well.

Click here for a full-size image.

One of the interesting things to me about physical controllers is how they impose or encourage a certain type of ergonomics or style of performing. Do you find in practice that people play differently with the K-Bow?

KM - It's additive. There's a way to teach, and to play stringed instruments, and there is a lot of tradition there. Since there's no way of perverting that concept, I realized I had to be additive. So players can still do everything that they do normally, but they also get a lot of extra control over their sound by mapping their bow movements to the processing controls in software.

Violinists usually pick it up and say, "Oh this is pretty light," and then they'll play for a couple of minutes and say, "it's not bad," and then after about ten minutes half of them say, "this is better than my bow!" The classical string world is very traditional, and there are certain things you just can't violate. The violin is probably the strongest icon for classical music and the bow is really part of that. My goal is to take traditional Western Classical Music and smash it into the computer age. There are some things you can get away with and some things you can't.

BT - While the first release of the KBow software is for a single performer, much of the amazing power of this technology comes from tying the data of several musicians together. When you have the up bow of the violinist affecting a filter sweep on the viola, or the bass transposing the whole ensemble...well, you have a completely new genre of music performance in which there are many more complex structures possible to compose. These are the directions we are interested in exploring with both the KBow and our upcoming products.

One thing I'm always paying attention to in people's custom performance systems is how much they have to look at the screen to perform effectively. How is this managed in K-Apps? How much of the UI is necessary for performance?

KM - Well, depending on the complexity of your setup, you might need to peek at it now and then just to get a sanity check. For this purpose, we created the K-Live window. This window gives you a summary of everything. It shows me the condition of the sensors, how much the bow is moving, which gestures are getting recognized. So you can glance at that to see what's happening and then go back to what you were doing. In this window you also get basic mixer controls so you can adjust the gain and 3-band EQ. Everything in this window is adjustable by hand, or you can assign an external controller like a foot pedal as well. Granted there are also things like the Loop window or the Phase Vocoder, where it's complex enough that you might want to keep an eye on what's happening.

Click here for a full-size image.

Barry, what would you say were the biggest challenges in bringing all of this together?

BT - Conceptually the most challenging part when designing software like this is to strike the right balance of ease of use and power. I think with the KBow we have done a good job. Ideally, we would really like performers without programming knowledge to be able to create complex musical structures, and be able to trust the software during performance.

Technically there are four major challenges that always present themselves in projects of this scope:

One is processor usage. With any music system with many parts managing DSP is a huge concern. Making sure things that are not in use are not taking any processor cycles is the key to making the whole thing run smoothly. Second is data storage. This may be the biggest and most time consuming issue, but Max offers many tools such as the pattr system to make it way easier. We are also still in 4.6 because our product timeline just didn't match up with 5 development, however, there are more new pattr features in Max 5 that I wish I had every day.

The third issue is organization, which can be difficult with a visual programming environment. There is a certain critical mass of a Max project where if you haven't thought through your abstraction and organizational process early on you get stuck and have to rewrite some things. I don't know any good advice around this problem but experience, however our whole system is very modular. Parts can be reused very easily across applications.

It is incredibly important to have a consistent naming structure for all sends and receives, parameter names, patch names, etc. to keep the patch development manageable and maintainable. Many max programmers don't run into this issue right away because the patches are made for themselves, but another programmer should be able to approach your patch and start coding on it as easily as possible. Consistent naming is the most powerful tool to make this happen, along with documentation. We have a document in flux that lists of all the hundreds sends and receives in this patch, where they go, and what their purpose is.

Well, I'm impressed by the complexity of the system underlying this seemingly friendly interface. It seems like you could just keep digging deeper and deeper to get what you want.

KM - I look at it like this. First off, if we sold a bow that worked great, with no software, that would be terrible. Then if we made a bow that wasn't easy to start with, that would be terrible. And also, if you made a system that after 4 years you've exhausted its performance capabilities that would be equally terrible. It's a challenge to balance all of these things, but it is really a wonderful challenge.

K-Bow Product Page on Keith McMillen Instruments Website