screentoworld and worldtoscreen messages
Jun 03 2016 | 1:48 am
I had assumed that the messages 'screentoworld' and 'worldtoscreen' to jit.world (and jit.gl.render) would translate between xy pixel coordinates and the xyz position in world space that corresponds to those coordinates. To do this correctly, the conversion would obviously have to take into account the current camera position, scale and rotation, since these change the relationship between the world and the screen. Unfortunately, this isn't what seems to be happening.
The 'screentoworld' and 'worldtoscreen' messages work as long as settings for jit.world (or jit.gl.render) (and jit.gl.camera) are left at the defaults for position, rotation, scale (as well as camera position, rotation and lens angle). As soon as any of these parameters are changed, the relationship between screen and world coordinates breaks down. What you actually seem to get is simply a conversion between 2D screen coordinates and the (also 2D) xy plane (which is parallel to the screen).
Example attached. First drag the square around on the screen; it matches the mouse position. Then change any of the world or camera parameters and see what happens. The square stays glued to the xz (blue) plane.
Am I missing something? Is there some other way to determine the correlation between screen and world coordinates?