Motion track ---> Panning
Im currently using cv.jit.hsflow to try and use motion tracking to pan sounds in an octaphonic system. Im not sure how to go about implementing this however. I need to crossfade between each speaker linearly using the x axis value and I also need the speed of hand motion to dictate the crossfading rate for faster/slower panning.
The difficulty is having smooth transitions between speakers so that the panning is natural.
Sounds like you may need to look into VBAP - here is a paper for a good overview:
http://lib.tkk.fi/Diss/2001/isbn9512255324/article4.pdf
There is likely no need to redo work already done by others, for example there are some VBAP tools in the UBC toolkit:
http://debussy.music.ubc.ca/muset/toolbox.html
There is also Sem's patch: https://www.dropbox.com/sh/4b2i4bk35qkm16i/AAD9QzPP7u4PyMNlaGLqOCNta?dl=0
It will be more work if you need to adjust the speaker layout somehow, but between these two options there are both perfect octagonal orientation and also a rectangular 4L4R orientation.
There is a panner module in the UBC toolkit which will take in normalized XY coordinates and pan an audio source within the octagon. It could be exactly what you are looking for? I used it for all of the common gestural controllers like Kinect and Playstation Move.. :)
So to use VBAP I need to first define the azimuth angles of each speaker in the circular array. This will be a value out of 360 I presume with front right and left being 90 and -90. Next I need to create the vbap object with controls for spread and azimuth followed by the matrix and delay objects with the dac. Is this correct?
That sounds essentially correct. It depends which solution you use.
For several solutions, these are already defined for specific layouts, and you just need to change everything to accommodate your layout.
If their solutions use 8 regular speakers and so do you, it should be much easier.
I am working on adapting a few solutions to traditional 5.1/7.1 layouts, which is a bit trickier since 1 of the speakers becomes a sub instead, and two speakers would go away in 5.1...
Keep in mind that when VBAP is well implemented, it doesn't matter what the layout of the speakers is, it will accommodate. Kind of like ambisonics, you will get better detailed resolution in general if there are more speakers covering a more even spread of the total area.
VBAP can do height as well, like for when speaker installations are hiding behind fulldome projection surfaces, etc.
Thanks very much kcoul, really appreciate it. Can I also ask how to extract usefuldata from cv.jit.blobs.centroids? I have a jit.cellblock object which is spitting out 7-8 cells worth of data. Am I right in assuming that that is x,y,size of each centroid blob?