Duffield

Just wanted some thoughts. I'm doing some stuff that was written on the CPU and am trying to convert it to GPU based processing. 

The original patch is super complex, but I'm just looking into different approaches and looking at my Activity Monitor performance. These are just tests with example code.

I used the same video in both. (640 X 480, H.264 **yeah I'm working with a client and their source material is H.264, they're ultimately going to be using HD**).

jitter-examples/gen/jit.gl.pix.srcdim.dstdim :

With jit.gl.pix.srcdim.dstdim my CPU is at ~50+%

Macbook Pro 2015 (Retina, 15 inch Mid 2015)

Hey All,

Just wanted some thoughts. I'm doing some stuff that was written on the CPU and am trying to convert it to GPU based processing. 

The original patch is super complex, but I'm just looking into different approaches and looking at my Activity Monitor performance. These are just tests with example code.

I used the same video in both. (640 X 480, H.264 **yeah I'm working with a client and their source material is H.264, they're ultimately going to be using HD**).

Patches used for test are:

jit.submatrix helpfile :
With jit.submatrix my CPU is at ~30%

jitter-examples/gen/jit.gl.pix.srcdim.dstdim :
With  jit.gl.pix.srcdim.dstdim my CPU is at ~50+%

Any thoughts on why this is?

My Setup:
Max Version: 7.3.4
Macbook Pro 2015 (Retina, 15 inch Mid 2015)
OSX 10.12.3 
2.5 GHz Intel Core
16 GB Ram
AMD Radeon R9 M370X

Thanks in advance!




Hey All,

Just wanted some thoughts. I'm doing some stuff that was written on the CPU and am trying to convert it to GPU based processing. 


performance-disparity-jit·submatrix-vs-jit·gl·pix·srcdim·dstdim-cpu-is-faster!

jit.submatrix doesn't have to perform a matrix copy, it's simply a reference to an existing matrix. sending a matrix to any texture object (e.g. jit.gl.pix) requires a texture upload.

So, if your source material was already residing on the GPU (Open GL texture) instead of a regular matrix, it would be extremely light in CPU utilization. On the other hand, going from texture to jitter matrix would be extremely slow...

Sorry, just to clarify and specify what I'm trying to do in pseudocode (again as patch is complex, has a few bugs unrelated to this question, and I'd have to check with client to ensure they're cool with me sharing the patch as a whole :/ ).

2. Iterate through individual frames. I think we're anywhere around ~15-30ms per frame to keep the CPU down, but this is a limit we imposed.

3. Simple processing (cropping, rotation, etc.)

4. Placing multiple cropped sections algorithmically into a matrix that is exported into a .jpg (16000 X 720). 

This is not-unlike the movie2Image jitter-example, but I coded it differently from scratch.

This is all non-realtime processing but we're trying to speed up the "rendering" times if possible by moving to the GPU. I'm investigating this and gave no promises but am also curious :)

1. If we're doing non-realtime rendering, is there any advantage to moving to the GPU or are the benefits more with real-time performance?

2. Related to the first quetsion, originally it was all on CPU but I'm trying to optimize by moving to GPU. My plan was to do all on the GPU (jit.movie @output_texture 1), then go to the final matrix to export the .jpg. Given what @Pedros Santos said are there no benefit because I'm going from the GPU texture to the matrix in the end? I was thinking of using jit.gl.asyncread before going to the matrix that is exporting the .jpg

Would be good to know before I go ahead and spend the time coding it all :)

Thanks for this!

Sorry, just to clarify and specify what I'm trying to do in pseudocode (again as patch is complex, has a few bugs unrelated to this question, and I'd have to check with client to ensure they're cool with me sharing the patch as a whole  :/  ).

1. Video loaded in via jit.movie
2. Iterate through individual frames. I think we're anywhere around ~15-30ms per frame to keep the CPU down, but this is a limit we imposed.
3. Simple processing (cropping, rotation, etc.)
4. Placing multiple cropped sections algorithmically into a matrix that is exported into a .jpg (16000 X 720). 

This is not-unlike the movie2Image jitter-example, but I coded it differently from scratch.

This is all non-realtime processing but we're trying to speed up the "rendering" times if possible by moving to the GPU. I'm investigating this and gave no promises but am also curious :)

QUESTIONS:
1. If we're doing non-realtime rendering, is there any advantage to moving to the GPU or are the benefits more with real-time performance?

2. Related to the first quetsion, originally it was all on CPU but I'm trying to optimize by moving to GPU. My plan was to do all on the GPU (jit.movie @output_texture 1), then go to the final matrix to export the .jpg. Given what @Pedros Santos said are there no benefit because I'm going from the GPU texture to the matrix in the end?  I was thinking of using jit.gl.asyncread before going to the matrix that is exporting the .jpg

Would be good to know before I go ahead and spend the time coding it all  :)

Rob would certainly answer this with much more knowledge, but I'll give it a try, since he's been visibly busy, I mean, just look at the jitter changes in 7.3.4... ;-)

I would say that the big advantage of the GPU is indeed the speed at which it can process certain kinds of data, namely the kind usually associated with visual content. But every now and then you might need a certain algorithm that, by its complexity, can only run on the CPU, as it has a more general-purpose instruction set.

Even though you're not doing realtime rendering, I would still try to implement the system using the GPU, as it can be useful for the offline render to finish processing at less than real-time. It is true that texture readback (GPU to CPU memory) should be avoided whenever possible, but in this case I think the benefits of offloading the heavy processing to the GPU should surpass the eventual penalty of reading back to the CPU at the end, especially if you use somethink like jit.gl.asyncread or even sending it through Syphon and using Syphon Recording to save the file. You would have the added advantage of using the GPU for image processing while freeing the CPU for video encoding.

1. Test the maximum size of an OpenGL accererated scene. Since the jit.window size will certainly not match your desired rendering resolution (16000 X 720, is that correct?), you might have to render to a jit.gl.node with a specific size.

2. Regarding your point 4: I wouldn't try to place multiple cropped sections algorithmically into a matrix (a jit.gl.texture in this case) but instead try to compose (scale, position, blend, etc) each "object" in the OpenGL scene.

Thanks so much for your help! Also, yes Rob's been doing awesome stuff :)

1) Thanks for the heads up on the jit.gl.node. No it does not match the resolution of the window.

2) You mentioned that it might be better / easier to have each frame as an object in the OpenGL scene. So in other words, you think it would be better to do something like instance a jit.gl.videoplane for every consequetice frame? This gets a bit tricky because the size of the crops, and source images will vary. I just figured putting it to a texture is easier then instancing a varying number of objects based determined by how many times a cropped region can fill up the final output composition (e.g., if the final composition was 100 X 1000, and the cropped image was 100 X 100, ten would fill the composition, if cropped region was 200 X 100, 5 would fill it, etc.) Note these numbers are just used to make calculating easier for explanation vs the actual dimensions that are to be used.

Attached is a test image of the outputted effect (but downsized from the original 16000 wide composition). 

Testy-Image.jpg

Hey Pedro,

Thanks so much for your help! Also, yes Rob's been doing awesome stuff :)

 Sorry, a couple quick questions: 

1) Thanks for the heads up on the jit.gl.node. No it does not match the resolution of the window.

2) You mentioned that it might be better / easier to have each frame as an object in the OpenGL scene. So in other words, you think it would be better to do something like instance a jit.gl.videoplane for every consequetice frame? This gets a bit tricky because the size of the crops, and source images will vary. I just figured putting it to a texture is easier then instancing a varying number of objects based determined by how many times a cropped region can fill up the final output composition (e.g., if the final composition was 100 X 1000, and the cropped image was 100 X 100, ten would fill the composition, if cropped region was 200 X 100, 5 would fill it, etc.) Note these numbers are just used to make calculating easier for explanation vs the actual dimensions that are to be used.

Attached is a test image of the outputted effect (but downsized from the original 16000 wide composition). 

Again thanks a lot!


, using jit.gl.cornerpin inside a poly~ serves your intents.

Basically, you define the array size (x by y images) and the patch calculates the coordinates of the GL planes according to the window size. Then you could route the texture to each plane using the message "target $1" to poly~.

See if the patch I posted in this topic, using jit.gl.cornerpin inside a poly~ serves your intents.

Thank you sooooo much for your help and the patch! :D

Hey Pedro,

Thank you sooooo much for your help and the patch!    :D

Performance Disparity? jit.submatrix vs jit.gl.pix.srcdim.dstdim CPU is faster!