Mark Kerr

I'm working on a patch based on "The Video Processing System" tutorial by Andrew Benson.

https://cycling74.com/tutorials/the-video-processing-system-part-4

I have a doubt here.. what if after all this processing done with the shaders on the GPU I wanna insert something like.. the Debris example (Recipe 27) in the chain ? Or any other kind of processing the requires operations on a matrix ?

Moving back to CPU is gonna kill the fps?

I hope someone can enlighten me on the use of matrix operations and shaders in the same processing chain.

Hi,
I'm working on a patch based on "The Video Processing System" tutorial by Andrew Benson. 

I have a doubt here.. what if after all this processing done with the shaders on the GPU I wanna insert something like.. the Debris example (Recipe 27) in the chain ? Or any other kind of processing the requires operations on a matrix ?
Moving back to CPU is gonna kill the fps?
I hope someone can enlighten me on the use of matrix operations and shaders in the same processing chain.

gpu-then-back-to-cpu

cmon'.. I thought this was kind of a question of general interest..

Yes, it will impact your FPS. I'm unsure what your question is exactly. You can simply try it and see what the performance change is.

yeah, sorry if my question wasn't as clear.

What I'm asking is basically if there's any recommendation when mixing processing on the GPU and CPU and if it's ok to do the kind of conversion I'm showing in the attachment.

Sure I can see it affecting the FPS but I don't know if there's any way to avoid it.. and yeah I was wondering on a general level if that's something that's generally avoided since Andrew Benson in his Video Processing System tutorial does all the processing using shaders.

yeah, sorry if my question wasn't as clear.
What I'm asking is basically if there's any recommendation when mixing processing on the GPU and CPU and if it's ok to do the kind of conversion I'm showing in the attachment.
Sure I can see it affecting the FPS but I don't know if there's any way to avoid it.. and yeah I was wondering on a general level if that's something that's generally avoided since Andrew Benson in his Video Processing System tutorial does all the processing using shaders. 

For maximum speed avoid any and all readback to CPU, and where possible, program shaders to do this. There's no way to avoid some significant impact on the FPS by reading back to the CPU.

Where there is something which doesn't translate to the GPU easily, such as image tracking with cv.jit, or FFTs with jit.fft, or other image analysis for parametric control of the Max patch, then you will want to read back. jit.gl.asyncread will probably be your fastest option for readback to the CPU, but it can introduce a frame or so of latency which might be undesirable. I'm unsure of your application. If you are looking to get mean brightness and perform gain control on the image, or something else which needs to have the CPU frame processing in perfect sync with the GPU frames then the potential latency from this asynchronous method isn't appropriate. IN such a case use the method in the image you've posted.

Basically, if you could tell us what you want to do on the CPU vs GPU and why, we could offer you more useful information, but the short of it is don't readback to the CPU unless you have to. If you do have to readback to the CPU, for maximum performance, decrease the resolution where possible, and use jit.gl.asyncread if acceptable.

For maximum speed avoid any and all readback to CPU, and where possible, program shaders to do this. There's no way to avoid some significant impact on the FPS by reading back to the CPU. 

Basically, if you could tell us what you want to do on the CPU vs GPU and why, we could offer you more useful information, but the short of it is don't readback to the CPU unless you have to. If you do have to readback to the CPU, for maximum performance, decrease the resolution where possible, and use jit.gl.asyncread if acceptable.


thanks a lot for the infos ! Very explicative.

I was messing around with Andrew Benson's tutorials and basically I wanted to incorporate the Debris patch

https://cycling74.com/tutorials/jitter-recipes-book-3/

in the processing chain of The Video Processing System, which uses textures and shaders.

I actually did it using the method in the img above but I just wanted to make sure I was doing a really stupid / fps-killer thing...

Hey Joshua,
thanks a lot for the infos ! Very explicative.
I was messing around with Andrew Benson's tutorials and basically I wanted to incorporate the Debris patch
https://cycling74.com/tutorials/jitter-recipes-book-3/

in the processing chain of The Video Processing System, which uses textures and shaders.
I actually did it using the method in the img above but I just wanted to make sure I was doing a really stupid / fps-killer thing...


GPU, then back to CPU... ?