Help with jit.gl.multiple and camera position/rotation

Pedro Santos's icon

Hi. I've built an audiovisual synth a while ago and I'm trying to re-implement the sound in surround.
Each sphere produces a sine wave, with its frequency depending on its Y position, its panning on the X and Z position and the amplitude is attenuated with distance.
I have a matrix with the position of the spheres controlling jit.gl.multiple.
What I would like to do is recalculate this matrix based on the gl camera position and orientation in order to control the sound's position.
Any advice on doing that?
If you look at the following synth's video, I think it becomes easier to understand: http://vimeo.com/29404191

Thanks in advance!

Pedro Santos's icon

OK, I will try to simplify the question:
Is there an easy way to do a 3d transformation (combined position and rotation) to a matrix?
This matrix contains object position coordinates for jit.gl.multiple.

I have a matrix with object positions.
I move and rotate the gl camera and get position and rotation information (getposition,getrotatexyz).
I want to transform (translate/rotate) the original matrix and obtain a new one that contains the new object positions in regard to the camera's position and rotation.

Any tips out there? Thanks.

Pedro Santos's icon

I guess I'm talking with myself here, but anyway here it goes, in case anyone searches the forum in the future...
I've had some problems coming with the answer because to simulate jit.gl.camera's rotation, the calculation order needed to be YZX instead of XYZ. Maybe there's an easier answer, but this one works and I guess is efficient.

Max Patch
Copy patch and select New From Clipboard in Max.

Jadie Rage's icon

I don't have an answer to your question but that video is gorgeous. I would love to know how you did the trails in that. It looks like you are storing a fixed length history of geometry data, passing it to a [jit.gl.mesh] and drawing it with a texture that has partial transparency? Something like that?

Pedro Santos's icon

Hi, Jadie.Thanks for the kind comments regarding the video!

You are absolutely correct in your assumption about the process employed:
I send a one-column matrix to jit.gl.multiple to define the object positions and draw the spheres.
This matrix is also sent to the leftmost column of a multi-column matrix. On each frame, that column is offset 1 pixel to the right, making a memory system with the previous positions.
The object used to transform the matrix to geometry is indeed jit.gl.mesh with several "draw_mode" configurations: quad_strip, tri_strip. line_strip, quads or points. jit.gl.mesh's colour is semi-transparent.
Then I capture the whole scene to texture and process it with a blur-based feedback system.
The camera can be fixed or dynamic. When in "dynamic mode", it follows one of the objects with a certain offset (using position and lookat messages).

Pedro Santos's icon
Max Patch
Copy patch and select New From Clipboard in Max.

Here's an update to the solution I've posted above. The gen code was simplified and I've deleted a few unnecessary objects.

vichug's icon

Hi,
this was mesmerizing ! thanks ! btw i might reuse your patch to learn how to navigate through a virtual space using jit.gl.camera and a joystick. So thanks again :)

vichug's icon

Hey, it's me again.

So thanks to the "matrix translation and rotation" thingy, i could do something to navigate inside your 3d opengl scene, with controls alike those of a FPS game (using arrows and mouse). It will be useful to me at least. Here it is :

Max Patch
Copy patch and select New From Clipboard in Max.

this makes me think that perhaps i missed something very obvious with jit.gl.camera, cause i would have thought you could "recenter" the view at each point where you are, so that you can go from that point and navigate relatively to each position you're at, not the original origin point... or do you need to recalculate the matrix each frame to do this ? i guess that was your point since the beginning of this thread then...

Pedro Santos's icon

Hi, vichug. Thanks for your comments!
The patch I posted needs Max 6, as the main purpose of it resides inside jit.gen and it is also using jit.gl.camera and attrui.
The patch you posted is a Max 5 patch that has nothing on it. I don't know what might have happened... post the patch again if you need further help.

Just to clear some doubts:
You don't need my solution to navigate in an OpenGL scene. I guess you could do it better with jit.gl.camera and jit.anim.drive, although I've never tried it...

What I wanted to exemplify in my patch was how to calculate the relative position and orientation of objects to the camera.
So, in this mode, instead of moving/rotating the camera, the camera is fixed (0, 0, 0), I calculate the equivalent transforms and apply it to the scene objects instead. I move the objects and not the camera.
The visual end result is the same, but I do this not to render the images but to calculate the sound of each object in regard to camera proximity and orientation.

vichug's icon
Max Patch
Copy patch and select New From Clipboard in Max.

hm, this is indeed odd. I might have done something wrong, here i try again. And now that i'm trying, i realize that there might be a problem with my Max, cause it always pastes saying "max5_patcher"

Anyway, calculating the sound of each object in regard to camera proximity and orientation is undoubtedly something i'm very interested in ! though i don't really know how to do it, cause i'm a beginner with Jitter, and i'm doing the Jitter tutorials nowadays, so i'll probably understand it later.

vichug's icon

with a quick glance at jit.anim.drive, it looks like it could help navigating through a field. But since i do need to have the relative coordinates of each object recalculated each time i move, your solution might be the best one still... i'll know better about this in some weeks. And again, this way of navigating in 3D, is really not easy to do with the jit.gl.camera only, because each time you rotate, i wanted that if you move forward, you move forward relatively to that new angle, not to the original angle.
Btw i'm not sure i'm asking for help here, because i'm conscious that i miss some basics in Jitter to understand most of what is happening here... though this patch did definitely help me.
i reupload the patch here, just in case the past compressed didn't work

4641.navigatingafield3.maxpat
Max Patch
Pedro Santos's icon
Max Patch
Copy patch and select New From Clipboard in Max.

If you use animmode = local in jit.gl.camera, jit.anim.drive should react as you want. Here:

vichug's icon

Hey, well, so, yes, indeed it does. Again, thanks !

mauro's icon

Hello Pedro,
I saw your video "Sound Atom"
I want to congratulate you because it is a wonderful job! is very poetic.

I wrote you a private message on vimeo

bye
Mauro

Pedro Santos's icon

Hi Mauro. Thanks for your kind words regarding the video. I would like to have more time to implement some features I have in mind... but... ;-)

I've responded to you on vimeo, but here's the short version: if I was using Max 6 when I made this patch, I would probably have used jit.gl.path to draw the trails instead of drawing them with jit.gl.mesh... so that's my recommendation for you.

Wetterberg's icon

Hey - not to go too far off-topic, but how'd you handle the glow and such?

Pedro Santos's icon

Hi, Wetterberg. It's not really glow, as I don't affect only the "highlights" through a lumakey process, but the end result looks somewhat similar...

I render to text the scene and the rest is obtained through post-processing effects.
The processing chain looks roughly like this:
full-scene motion blur (multiple renders accumulated on a texture per frame displayed)
blur effect with temporal feedback
combination of the original texture with the processed one

Pedro Santos's icon
Max Patch
Copy patch and select New From Clipboard in Max.

Here:

Wetterberg's icon

great stuff, can't wait to take this apart.

lyve forms's icon

thank you so much for sharing, really beautiful