Head-Coupled Perspective or Fish Tank Virtual Reality

    May 17 2013 | 1:25 am
    Hi, I'm attempting to do a responsive visual environment on my monitor using the viewers perspective to create the perception of 3D. Like Johnny Lee did in http://youtu.be/Jd3-eiid-Uw but ill be using a Kinect for head tracking.
    As you can see in the patch below, I'm pretty close. But my camera position is not correct. In practical terms, I need to set the origin of the 3D scene at the center of the screen and consider the screen frame’s dimensions as constant values. In this coordinate system, a virtual camera is set at the position of the user’s head, pointing towards the origin. As the user’s head moves, the parallax of the 3D scene is altered by varying the virtual camera’s field of view and its offset position. I'd love some help on this one!

    • May 17 2013 | 1:27 am
      To play with my patch change the X and Y of the position box (RED) connected to the (jit.gl.camera alchemy)
    • May 17 2013 | 12:45 pm
      I am unsure what you question is. I changed the X/Y values and can imagine it working when you have the kinect attached.
    • May 17 2013 | 5:08 pm
      Hi Diablodale, I was checking you work earlier for some insight!
      Basically the problem that remains is the Fish Tank Virtual Reality. When you changed the X/Y position, the camera glided a bit to the left/right or up/down in relation to the space. This should not happen. The space and camera has to be fixed to the monitor, the camera has to be where the user is and pivot on a fixed point on the center of monitor. If this is not clear check out the second video from this website. Manfreda is doing the exact same thing I'm attempting to do:
      This new patch includes more 3D objects
    • May 17 2013 | 9:26 pm
      Hey yagodequay,
      this is a very cool idea. I guess the solution lies in correctly filling up your frustum and rotation/perspective matrices. Sadly, my knowledge of these terms is not enough to give you more detailed info, although I'm eager to find out how to tackle this!
    • May 17 2013 | 11:08 pm
      Here is a super-crud example. I took yours, simplief, and hacked frustum values. You can use the float boxes to change your position. I fixed them at +/- 1 each. You could feed in kinect head values into them if you prescale to (-1,1).
      There is more to do including some math. I suspect you have to relate the typical distance of the viewer from the monitor into the calculations. Probably some trig in there to for triangulating the frustum left,right,top,bottom values depending on there your eyes are. I just hacked it to be a linear scale which it definitely incorrect.
    • May 17 2013 | 11:26 pm
      Thanks Diablodale, I followed yours and Vjacobs advice on the frustum and got great results. The math has to be polished but it's effective.