I’m using a system developed by my friends at Sensacell (www.sensacell.com). Sensacells are 6" tiles with 36 color LEDs on them that can be built into fairly large arrays. They also have capacitance sensors built into them that can detect, say, a user’s hand, even through a glass window, as in the case here.
We developed special hardware to simplify the communication with the cell arrays so that my Jitter program simply sends (over TCP/IP) a video frame to our box then receives a frame back. The outgoing frame is displayed on the tile array and the incoming array is a grayscale "map" of the sensor data across the array. The jitter program uses the input to manipulate the output in real time.
Please send me any comments or questions!
Bob Stratton firstname.lastname@example.org