Pat is doing this crazy project with a whole orchestra of robots both being triggered by the computer or in realtime via his guitar. Amazing.
Besides many other aspects one specific issue makes me question how it works: How can there be such a good note detection without midi? He is not using a midipickup (his guitars only have a normal pickup plus one microphone output for a more direct string sound). As far as I am cocerned a really good note detection only works for monophonic instruments or voice in realtime. The IRCAM institue in Paris offers this score following device but it seems they are working hard to make it happen for polyphonic instruments like guitar. And in another video you can see Pat doing some crazy chord stuf and all these vibes and pianos and bottles playing with him.
If anybody knows some sort of software or even max patch to achieve this level detection I would be surprised and thankful!