jit.gl.camera to audio mix
Hello all,
I am wondering what methods other max users out there would recommend looking at, when trying to map jit.gl.camera position movements to audio level changes.
I have a 3D environment with several jit.gl.gridshape objects, that become visible as the camera moves position along the y plane. The audio that is linked to the gridshape is scaled to get louder the closer to the camera the shape gets, and subsequently quieter once it has been passed. To do this I have just used some clip and scale objects linked to the camera y position. It works ok, but if I want to randomise the movement of the camera position a bit more, and perhaps use the x and z planes, does anyone have any recommended objects/maths that I should pay attention to?
I think I could probably use the same clip and scale objects to do this, but i'm sure there must be a more efficient and effective way in max to do this. Or perhaps there is a better method of moving through a space visually and sonically?
Any thoughts would be greatly appreciated.
Cheers,
Rick
Given that your camera is at position Cpos = (Cx, Cy, Cz) and an object at Opos = (Ox, Oy, Oz), you can get the distance between both with the following formulas:
D = sqrt((Ox - Cx)2 + (Oy - Cy)2 + (Oz - Cz)2)
or in vector math: D = length(Opos - Cpos)
(the length operator is available in jitter operators, in jit.gen or jit.gl.pix)
Then you might want to apply log or exp function to convert that distance to an audio gain.
For spatialisation, you need to know if your object is more toward the right or the left of the camera, more in front or behind, more above or below.
For that, and also to get the distance, polar coordinates may be more practical. You can use [cartopol] to convert 2D coordinates (X,Y) to polar coordinates (distance, angle). You can do that on two planes, like (X, Z) and (Y,Z) to have both the panoramic angle and the elevation for example.
If you provide an example I can show you some ways to integrate this in your patch.
EDIT: for spatialisation, you will also need the direction of the camera, not only its position. It is available as an attribute of [jit.gl.camera], you can get it by sending it a getdirection message at each frame, or by sending a bang at each frame to a [getattr direction] with its left outlet connected to [jit.gl.camera].
Hi TFL,
Amazing, thanks for that.
It would be great if you could help, I just have to figure out how to provide an example that you will be able to read/understand from the sprawling/messy patch i've been making!
I will get back to you soon.
Cheers,
Rick
Maybe it's best to start from scratch just for this specific part of your project: a jit.world, a camera, two gridshapes, two audio files to play in a loop, bits of what you already have in-between these objects, and it should be good!
Hi,
I managed to use the D = sqrt((Ox - Cx)2 + (Oy - Cy)2 + (Oz - Cz)2) formula you sent and it works well. Thanks.
The spatialisation polar coordinate setup might take me a bit longer to figure out, but you're right this would be preferable.
When I can, I will attach a reduced patch with my workings to see if you had any further suggestions.
Cheers,
Rick
Hi TFL,
Here's what I came up with for distance calculations so far (in a simple patch that hopefully you can understand). If you get any time then please let me know any pointers or further suggestions on it?
It seems to work so that's the main thing!
Cheers,
Rick
Hey! For some reason your patch runs at 2fps on my computer because of these two [p shape_transform_feedback]. If I remove them I get back to 120fps. Taht's weird because I can run the Galaxy Chickens glcore example (from which your subpatcher is based) above 120fps without issue, so that's definitely something wrong in yout patch.
I tried to read a video and play a bit with the various available controls but couldn't get a single image out of it.
Given that I'm not sure what the result should be, I'm not going to debug the patch, but I can make a few observations:
When you share a patch over here, make sure to make it work as simply as possible: use builtin assets, make them to load automatically (basically [loadmess read chickens.mp4] on every jit.movie and same for audio buffers with anton.aif, drumLoop.aif, prim.loop.aif, etc. as you want), so that the user just has to do one or two clicks to make the patch to run without having to dive into all subpatchers. Some comments can help too.
Instead of sharing the patcher file, use copy compressed (makes it easier for the rest of the community to help you: no need to download a file)
On jit.world, the first argument defines the worlds name. If you set it that way, you shouldn't use the @name attribute. It's either one or the other: [jit.world thomswords] or [jit.world @name thomswords]
Same for other jit.gl objects, for which the first argument defines the drawto context. So when you write [jit.gl.mesh thomswords @drawto shape1] you define two different drawtos, which is wrong. [jit.gl.mesh shape1] is enough.
[p object distance] can be greatly simplified using a couple [vexpr] (instead of unpacking each argument) and expr and/or gen (clip, scale, sqrt are all available in gen). The more you put in your gen/gen~/jit.gen/jit.gl.pix, the better usually performance wise.
I think you can remove
@capture 1from your [jit.gl.node], and connect the underlying [jit.gl.pix] to [jit.gl.pass] only. Maybe the performance issue I had actually came from this.If you plan to use the same parameter values for the tf.vectfield shader used in both transform feedbacks, you can actually just use one [jit.gl.shader] and reference the same shader name in both [jit.gl.tf]
Here's how your [p object distance] can be simplified/optimized/easier to read. About the math itself, I feel like there might be some optimization to do, but I didn't dug in too much.
Hi TFL,
Thanks, I shall have a look at your simplified patch and get an understanding of it.
Apologies if you couldn't use my patch, and thanks for the tips on how best to share, as well as the copy compressed link. Noted for next time.
I'll look at the other tips and see if I can make use of any.
Many thanks again,
Rick
Hi, Riccardo. Maybe you'll find this topic useful for achieveing your purpose.
I would then take care of the audio processing like volume attenuation and panning using an ambisonics package from the package manager, like ICST Ambisonics.
Hi Pedro,
Thanks for the link, I will look into it, and the ambisonics suggestion.
Much appreciated.
Cheers.