Panning and 3D trajectories (with RVBAP?)

I don't have much hands-on experience with 3D spatialization, but it would seem apparent to me that you cannot (easily) simulate a sound whose SOURCE is inside of the space contained by the speakers. Maybe if you were right at the sweet spot where all 6 speakers were aiming, you could do some crazy phase stuff that would trick your ears into thinking the source is within the hexagon of speakers, but apart from that I believe that any sounds would appear to originate from outside of the hexagon.

Also, I would assume you'd need a minimum of 8 speakers to get full 3D spatialization. If you're in a rectangular room, there would be one speaker in each of the 8 corners of the room. I could be wrong.

I'm not sure what RVBAP is, but you should be able to derive the equations you'd need using trigonometry. Pick where the source of your sound is in 3D space (outside of the speaker zone) and then find the distance from the virtual sound source to your listener (inside the speaker zone) in the x, y, and z planes. Treat the x, y, and z distances as a "vector" (a quantity with both magnitude and direction). Use that vector to determine the volume of the sound coming out of all speakers. The way I would approach it would be to first look at things in the x plane, and pan "up" or "down" depending on the value of the x vector. Then, pan "left" or "right" depending on the value of the y vector, and finally pan "in" or "out" for the z vector. You will also need to delay the sound coming out of the speakers so that the time of arrival matches up at your listener's location.

This will really only work well for one listener who is in the sweet spot.