Keep computation load constant across projectors
Hi everyone,
I'm building a real-time visual performance patch to run together with Ableton. I need to keep computation load consistent to avoid dropouts in the audio. Basically I would like to have the same load regardless of the projector resolution (Full HD, 4K, etc).
This is the signal chain:
shaders layer ← [jit.gl.slab @file Patcher:/my/shader.jxs @adapt 0 @dim 1280 720] -> [jit.gl.layer mix]
│
▼
jit.gl.node "mix" ← [jit.gl.node ctx @name mix @capture 1 @blend_enable 0 @erase_color 0 0 0 1 @adapt 0 @dim 1280 720]
│
▼
post-processing chain (example [jit.gl.pass mix @fxname bloom @quality low])
(bloom → downsample → blur → ...)
│
▼
mask layer (multiply-blended with a mask video layer)
[jit.gl.polymovie] -> [jit.gl.layer ctx @blend_enable 1 @blend multiply @preserve_aspect 1]
│
▼
jit.world ctx ← output window / fullscreen (here I am assuming a full hd projector as output)
[jit.world ctx @fsaa 0 @floating 0 @enable 1 @output_texture 1 @dim 1920 1080 @size 1920 1080 @erase_color 0 0 0 1 @fps 60 @sync 0 @displaylink 0]
Do you think this signal path is correct to guarantee that resource usage stays roughly constant regardless of the projector I plug in (Full HD, 4K, etc.)?
From my observation, a GPU load is usually pretty consistent as long as the scene remains the same (same textures, processes, geometry in use). Fluctuation happen when you add/remove processes or objects from the scene.
In your case, you want to avoid audio dropouts which might be caused by CPU spikes. The GL chain usually have an impact on the CPU when transferring data to the GPU (loading textures or geometry), when you transfer textures back into the matrix realm, or when you dynamically change the GL chain (like by repatching jit.gl objects with scripting or using a [matrix]). Looking at your signal chain, I would especially be careful in the way you deal with [jit.gl.polymovie] since this is where the CPU can be affected, potentially quite badly depending on the video format and what you do with those videos.
The resolution of the output projector shouldn't alter your CPU load assuming you always set it to 1920x1080 (the resolution of your [jit.world]).
Hi, the polymovie loads some 720p videos that I use as masks in blend multiply with the output of the shaders. I assume the multiplication depends on the resolution of the output window, but it should not be super expensive to do. Also as you said, I am using a fixed texture size in output at 1920x1080 on the jit.world.
What I would like as a confirmation is about the [jit.gl.node ctx @name mix @capture 1 @blend_enable 0 @erase_color 0 0 0 1 @adapt 0 @dim 1280 720] where I've the attached the post processing effects chain. It is correct to say that all the effects will work internally at 1280x720 regardless of the output window?
It is correct to say that all the effects will work internally at 1280x720 regardless of the output window?
Yes, the (jit.gl.)passes applied on the mix context should respect the node dim.
You can double-check that by attaching to your last [jit.gl.pass] a [jit.fpsui] and change "fps" to "dim".
Might be worth increasing the node dim to your output dim (1920x1080): the additional GPU cost shouldn't be too big, and you'll get a crispier output. Even if your source videos are 1280x720, there is a chance that your processing chain add up details that might benefit from the increased resolution.
Thank you for the confirmation! Great suggestion for the [jit.fpsgui], very useful 🙂
I'll try 1920x1080 fixed resolution on the post processing chain, see how the system handle it 👍