Sensory Fiend

Does anyone know of any objects that will allow me to project onto a corner of a room and in jitter 'draw' the perspectives of the corner and then perhaps apply those perspective over an image. Essentially combining virtual and physical perspectives...

mapping-perspectives

anyone have any detailed info on using maya shader code within jitter with jit.gl.shader?

where should i look for hints on editing the shader code to work with Jitter cg syntax?

also, how should the shader code be exported from maya?

From: jitter-bounces@cycling74.com on behalf of Kit Webster

-----Original Message-----
From: jitter-bounces@cycling74.com on behalf of Kit Webster
Sent: Fri 11/10/2006 7:31 AM
Subject: [jitter] mapping perspectives

On Nov 10, 2006, at 5:57 AM, Knoth, Brian wrote:

> anyone have any detailed info on using maya shader code within 

> where should i look for hints on editing the shader code to work 

> also, how should the shader code be exported from maya?

You may want to explore some general research online about Maya's 

shading language and porting to Cg or GLSL. Or investigate what is 

supported by the Collada export plugins available for Maya. Inside 

the Collada XML output, you should be able to find Cg or GLSL shaders 

Keep in mind however, not all kinds of shaders available in high end 

modeling/animation/rendering systems are suited for use in a typical 

GPU framework, and also Jitter doesn't have any easy support for 

multi-pass shaders currently. We're looking at a few possibilities w/ 

r/t multipass shaders for Jitter 2.0, but in the meantime you'll 

probably have to accomplish this with jit.gl.sketch's glbindprogram/ 

unbindprogram methods (explicitly binding a different shader program 

for each pass). Doing this from JS with jit.gl.sketch's new immediate 

If you do come up with a good solution for your work, please take a 

minute to share your findings with the list.

> anyone have any detailed info on using maya shader code within  
> jitter with jit.gl.shader?
> where should i look for hints on editing the shader code to work  
> with Jitter cg syntax?
> also, how should the shader code be exported from maya?

You may want to explore some general research online about Maya's  
shading language and porting to Cg or GLSL. Or investigate what is  
supported by the Collada export plugins available for Maya. Inside  
the Collada XML output, you should be able to find Cg or GLSL shaders  
that might be possible starting points.

Keep in mind however, not all kinds of shaders available in high end  
modeling/animation/rendering systems are suited for use in a typical  
GPU framework, and also Jitter doesn't have any easy support for  
multi-pass shaders currently. We're looking at a few possibilities w/ 
r/t multipass shaders for Jitter 2.0, but in the meantime you'll  
probably have to accomplish this with jit.gl.sketch's glbindprogram/ 
unbindprogram methods (explicitly binding a different shader program  
for each pass). Doing this from JS with jit.gl.sketch's new immediate  
mode might be the easiest way to manage.

If you do come up with a good solution for your work, please take a  
minute to share your findings with the list.

You're going to have to do the maths modelling your projector's

projection transform with your opengl projection transforms. Here's

http://www.merl.com/people/raskar/raskar.html

He published a great book with Bimber called "Spatial Augmented

Reality" available from A K Peters. If you have access to the ACM

portal, there's a SIGGRAPH 2004 course called Projectors: Advanced

Graphics and Vision Techniques that will give you pretty much

If this is too much, then just try it out by hand in the space and see

> Does anyone know of any objects that will allow me to project onto a corner of a room and in jitter 'draw' the perspectives of the corner and then perhaps apply those perspective over an image. Essentially combining virtual and physical perspectives...

You're going to have to do the maths modelling your projector's
projection transform with your opengl projection transforms.  Here's
the man who knows everything about this:
http://www.merl.com/people/raskar/raskar.html

He published a great book with Bimber called "Spatial Augmented
Reality" available from A K Peters.  If you have access to the ACM
portal, there's a SIGGRAPH 2004 course called Projectors: Advanced
Graphics and Vision Techniques that will give you pretty much
everything you need to know.

If this is too much, then just try it out by hand in the space and see
how close you can get.

On 11/10/06, Kit Webster  wrote:
>
> Does anyone know of any objects that will allow me to project onto a corner of a room and in jitter 'draw' the perspectives of the corner and then perhaps apply those perspective over an image. Essentially combining virtual and physical perspectives...
>


thanks wes ill take a look. Basically all I want to do is create a 3d overlay of a real image (or in this case draw in any curvatures or angles that the projector is projecting onto). Surely there is a simple way to do this?

Ok, I guess I completely misunderstood what you were talking about.

You want to draw lines on an image in 3d? I was thinking 3d lines

over a real scene for some reason. The simplest way to do this is to

convert the opengl viewport into the same coordinate system as the

image pixels. So, if I have a videplane, pixel 100, 50 will be at

opengl coordinate 100, 50. You can do this by calculating the width

and height in GL coordinates of your scene and Z=0 and then converting

pixel coordinate to GL coordinates with these values.

Here's a patch that demonstrates this. Hopefully in can get you

started. There are 3 steps as noted in the patch,

Ok, I guess I completely misunderstood what you were talking about.
You want to draw lines on an image in 3d?  I was thinking 3d lines
over a real scene for some reason.  The simplest way to do this is to
convert the opengl viewport into the same coordinate system as the
image pixels.  So, if I have a videplane, pixel 100, 50 will be at
opengl coordinate 100, 50.  You can do this by calculating the width
and height in GL coordinates of your scene and Z=0 and then converting
pixel coordinate to GL coordinates with these values.

Here's a patch that demonstrates this.  Hopefully in can get you
started.  There are 3 steps as noted in the patch,
best,
wes

I only seem to be able to create one line. How can I draw an entire 3D picture and use that information to 3D manipulate an image or a piece of video?

For instance, I have a ball and a cube sitting in the corner of a room. I physically project my jitter window over the ball and cube and then (draw) over their perspectives. The idea is to be able to project natural textures over the blank physical objects.

Ok, so you are actually projecting over a physical space. To do this,

go back to my original email. You're going to have to calibrate your

projector and measure the space. You need to figure out what your

projector's projection matrix is and then the modelview matrix that

will get you into your physical scene's object coordinates. Once

there, you can draw in normal opengl coordinates and have things be

where they're supposed to be. I can't really give you any more help

than this. You're just going to have to figure out the coordinate

transforms yourself and how that math works.

> I only seem to be able to create one line. How can I draw an entire 3D picture and use that information to 3D manipulate an image or a piece of video?

> For instance, I have a ball and a cube sitting in the corner of a room. I physically project my jitter window over the ball and cube and then (draw) over their perspectives. The idea is to be able to project natural textures over the blank physical objects.

Ok, so you are actually projecting over a physical space.  To do this,
go back to my original email.  You're going to have to calibrate your
projector and measure the space.  You need to figure out what your
projector's projection matrix is and then the modelview matrix that
will get you into your physical scene's object coordinates.  Once
there, you can draw in normal opengl coordinates and have things be
where they're supposed to be.  I can't really give you any more help
than this.  You're just going to have to figure out the coordinate
transforms yourself and how that math works.

On 11/10/06, Kit Webster  wrote:
>
> I only seem to be able to create one line. How can I draw an entire 3D picture and use that information to 3D manipulate an image or a piece of video?
>
> For instance, I have a ball and a cube sitting in the corner of a room. I physically project my jitter window over the ball and cube and then (draw) over their perspectives. The idea is to be able to project natural textures over the blank physical objects.
>
