Video(Live camera)-to-Sound(Turning video to sound)

JUNK AI's icon

How will you do that?

Gregory Taylor's icon

The task, like virtually any Max patching task, has three parts.


1. How does Jitter represent a video (hint: it's a matrix of values for red, blue, green, and alpha)?

2. What do you have in mind when you way "sound?" Do you want to play samples? Generate audio on the fly? apply effects to mic input? You'll need to know something about how MSP works with "sound."

3. You'll need to figure out how to take the stream of number data from the video and modify those numbers so that they are in the form and range to do whatever it is you have in mind when you say "turn video to sound).


My suggestion to you is to break the problem down into those three pieces, and to research them individually. The good news is that there's pretty good documentation for all that. Here's a starting point in terms of looking at how video is represented:

https://docs.cycling74.com/max5/tutorials/jit-tut/jitterwhatisamatrix.html