Stereo Sonification
Hi, I was wondering of some people might have some insight into how I can achieve this. I'm using a patch from the Baz youtube tutorials to drive an Oscbank from the data from a video stream. I'm wondering if anyone has any idea how I might be able to use the x position of the pixel data (starting from the center outwards) to place the individual tones in the stereo field.
I've been having the thought that the Oscbank may not be the proper choice in this situation, and maybe I have to make my own stereo poly~ oscillator that uses the x position to control the placement. Or now that I think of it, using 2 oscbanks maybe and inverting the amplitude values on one side seems like it might be a simple solution.
Any insights would be greatly appreciated, thanks!
ioscbank~ is much more computationally efficient than an equivalent number of cycle~ objects, but of course the oscillators of ioscbank~ are all mixed together into a single channel of audio. If you split the audio synthesis among multiple ioscbank~ objects, you can send them L-R (two) or L-C-R (three) or L-LC-C-RC-R (five, as in this example). Five virtual source locations are probably sufficient for distributing sinusoidal tones (which are inherently hard to localize perceptually). This example with 1200 oscillators should run okay on most modern computers.
Hi Chris, thanks a lot, that's a great help. I do have a few questions so if you have a chance here goes:
1. I don't need to separate color, so can I replace the jit.rgb2hsl with jit.rgb2luma and skip the unpack with any detrimental effect?
2. How are the jit.expr working with the patchers (counter), and as well, is there a way I can send a number box to the expr to be able to change to lowest pitch to my liking?
3. What is the jit.spill doing before the jit.iter that is different/better then my example?
4. My video is 1280x720, is it just the jit.movie, jit.matrix, and the jit.submatrix that I have to tweak to set that up properly? Just wondering if the jit.expr will need to be tweaked too.
5. Am I correct in assuming then that the pan position will still only be accurate to 1/5th the video segment (L LC C RCR). I'm looking to get the utmost position accuracy, and have a pretty beefy computer, so if I want the utmost accuracy, am I correct in assuming that I'll have to use a method other the ioscbank (ie cycle~)? And if so, what other methods are there to consider?
Thanks a bunch for the patch and info, that's a great help!
I confess I didn’t look at your patch before sending my previous example, but...
1. In my patch I’m mapping hue to pitch, and level to intensity. Hue values from 0-255 are mapped onto equidistant pitches (not equidistant frequencies) from A-110 to A-1760. Level values from 0-255 are mapped onto linearly-spaced intensities (not linear amplitudes) from -80 dB to 0 dB. Those just seemed like fairly intuitive mappings to me, but they're essentially arbitrary mappings. You can use brightness from jit.rgb2luma however you want.
2. Yes. See the help file and/or manual for jit.expr. My base frequency was 440 Hz. You could replace that with in[1] and supply a value in the right inlet.
3. The combination of jit.spill and iter is pretty much the same as using jit.iter by itself. Either way unfolds a matrix and sends the contents out as individual values.
4. I reduced the video down to 40 x 30 because that’s 1200 pixels, which seemed largely sufficient given that (in my example) there were only 256 possible pitches. 1280x720 is nearly a million pixels per frame, 768 times as many as in my example. If you think that many oscillators is needed, by all means, go for it. :) The jit.expr object operates on the entire matrix, calculating the correct output value for each cell, so it will behave the same on any size matrix.
5. Five locations in a sound mass of 1200 constantly-changing oscillators seemed like about as many as I thought I’d be able to distinguish. There’s nothing (other than CPU power) to stop you from achieving “utmost position accuracy” by assigning a unique panning to each of your 921,600 oscillators, though. If you want each tone to have a unique panning, you’ll need to use cycle~. If you’re willing to give some tones the same panning, multiple ioscbank~ objects will do.
Thanks again Chris! Ya I might maybe make 10 zones or so. The reason is, is I'm planning to using the captured audio to do a bunch of PVOC processes to other audio. So the audio rendered from this patch will be used to essentially drive a bunch of spectral processes on other audio sources.
Thanks again for your help, and all your other contributions to the forum. Since I've been into Max and using the forum, your posts time and time again have been very helpful!
Greg