Trouble updating multiple instanced jit.gl.buffers in real time
Hi all! I'm trying to figure out the best way to pass data to jit.gl.mesh instances through multiple jit.gl.buffer objects in real time, and I'm running into some difficulties.
I'm working on a project that involves drawing lots of planes. For the sake of performance, I'm trying to do everything with instancing, using jit.gl.mesh and jit.gl.buffer. For each instance, I need to control 16 values in the shader, which potentially need to get updated smoothly every frame. I've been trying to pass the data over using four jit.gl.buffer objects with their @type attribute set to vertex_attr0-3.
The system I built uses JavaScript to update the buffers. I tried doing this in a straightforward way by setting the data in matrices and then banging the buffers, but I found that only the first buffer to receive a bang in a given frame gets updated. I then tried using a round-robin approach to bang each buffer in turn on subsequent frames. This does cause all the buffers to get updated, but unfortunately the round robin approach leads to stuttery animation because the updates to each buffer are happening at 1/4 of the frame rate.
I was wondering if it would be possible to run the round robin approach faster than the frame rate, thinking that banging all the buffers within the time taken to draw one frame might resolve the problem. To this end, I decoupled the math from the bangs and tried using a high-speed qmetro to fire the buffers sequentially inside a single frame. This didn't fix the issue - it still only updates one buffer at a time gets updated. However, it did reveal some surprising behaviour: the buffer that gets updated changes when alternately clicking on the Max patch and the Jitter window.
It seems like there's a something like a race condition going on with jit.gl.buffer and jit.gl.world, and the first buffer to get updated in a frame is the only one that gets updated. I'm wondering if this is intended behaviour or a bug, and if there's a workaround.
I'm attaching a test setup comprised of the following assets:
the test patch
four JS files
the shader
The buffers are carrying data as follows:
vertex_attr0: colour
vertex_attr1: position
vertex: attr2: rotation
vertex: attr3: scale
These buffers are set up this way for the sake of this demo to make the issue obvious, but in my actual project, I need to use all 16 channels (actually 17, but I'm hoping I can pack the extra channel into the normal buffer, which I don't need to use for its standard purpose).
There's a live.tab that allows you to switch between the JS files and test out the different approaches:
JS round robin: cycles through the buffers every 4 frames
V8 round robin: same thing but in V8, just in case JS execution speed was a contributing factor
JS all at once: bangs all the buffers every frame
JS subframe updates: cycles through all the buffers at a faster-than-frame-rate speed
Any input would be greatly appreciated!
Hi there!
The issue is not a synchronization problem between jit.world and jit.gl.buffer. The real problem is that the same inlet of jit.gl.mesh is being used to attach multiple buffers. This is legit (it doesn't throw an error), but is fragile when many buffers are used.
The fix is simply to connect each buffer to a different inlet. Once that is done, you can update them using whichever approach you prefer. In the example patch below, I replaced the JavaScript with jit.gen for the per-instance varying values, and used standard expr syntax for the uniform values.
Note: I embedded the .jxs shader for easier file management, but the code itself is unchanged from yours.As you also noted, there is some opportunity to pack the data more efficiently, since several buffer planes are currently unused. As a small optimization, you could also pass cos(frame) and sin(frame) directly to the shader, either through jit.gl.buffer or as uniforms if the values are shared across all instances. That would let you avoid computing sin and cos inside the shader, which is generally a relatively expensive operation.
Hope this helps!
Hi Matteo, thanks so much for the quick response and the fix! Thanks also for the tip on computing the rotation outside the shader.
Is the one-buffer-per-inlet requirement documented anywhere? I didn't see it anywhere that I looked. If it isn't present, I think that it would be worth adding to the docs, and it would also be helpful if jit.gl.mesh threw a warning when multiple jit.gl.buffers are connected to the same inlet. I spent quite a long time trying to debug this and couldn't find any mention of this requirement, though it's totally possible I missed it. Anyway, I'm just really glad that it works - thanks again!
For context, I'm developing a set of Max for Live devices that are intended to support the creation of animated graphical music scores (in the style of e.g. Treatise by Cornelius Cardew). The current design uses one device as the parent (containing jit.world and assorted helper logic), as well as a set of child devices (one for graphical symbols, one for text, one for background images). To add a new graphical element to the score, the user adds the appropriate type of child device and configures it. This works fine from a UX standpoint, but the problem is that each child device contains its own set of Jitter objects (jit.gl.mesh, jit.gl.multiple, jit.phys.body, etc.). This means that the memory usage of each device is quite large - last I checked it was up to about 100MB of memory per child device. In the instance of reducing this cost, I'm trying to see if I can handle all of the rendering using instancing instead.
Hi Syrinx,
With jit.gl.buffer connected to jit.gl.mesh, the connection is used to bind a resource, not to pass data.
When jit.gl.mesh receives a matrix, the matrix contents are uploaded to the corresponding buffer. For example, if you send a matrix into the color inlet, its contents are uploaded to the COLOR buffer attachment.
By contrast, connecting a jit.gl.buffer does not send data into jit.gl.mesh. Instead, it declares a dependency, similar to how jit.gl.model/pbr works, or how the middle outlet of jit.gl.node is used.
Using a patchcord to connect a jit.gl.buffer is simply a patching-style way of expressing that dependency.
Also, if multiple jit.gl.buffers are connected to the same inlet, jit.gl.mesh does not bind all of them at once. The most recently received connection replaces the previous one.
About your project, first is really cool! Second, instancing can definitely be the way to go. If the "child devices" contain the same vertex data, instancing can drastically reduce memory usage.