Is it possible to process 32-bit floating point data with openGL in Jitter?
I would like to transfer 32-bit floating point data matrix (FFT data) to openGL and back to "matrix land". But everything that comes from openGL is in char format (ARGB). Is there any way to process 32-bit floating point data with openGL in Jitter?
Thanks, Tadej
I regularly process 32-bit floating data (e.g. x, y, z coordinates, colors, etc.) in OpenGL. By default OpenGL in Jitter renders to a window. If you were to peek at the data in that window, it typically renders in ARGB with 0->255 values for each plane.
Would you please be more specific in your input->output (lots more on output) so that we can better understand your question?
Thank you diablodale for your answer!
The thing is that I am sending a 2 plane 32-bit floating point matrix into openGL. What I get out from openGL is a 4 plane char matrix (last 3 planes are the same). So its a CPU to GPU and back to CPU transition. So my question is basically how to get a 32-bit floating point matrix out of openGL?
please see the patch if you have time (i am sending it via sendspace because a pfft subpatcher is included and also a 3,3Mb jxf file/200x2048 matrix containing FFT data):
In the patch there are 4 main things. One is matrix "spectrum" holding the original FFT data. Then is an openGL part with Gaussian blur that processes the matrix "spectrum". Processed matrix is written into matrix "spectrum2". jit.peek objects in pfft subpather reads "spectrum2". as you will hear, the recording is heavily distorted due to 32-bit to 8-bit(char) conversion.
PS: you will also notice a 200x2048 window displaying the openGL rendering context. I would like to get rid of it if possible. But as far as I know, the openGL rendering context needs to have the same size as incoming matrix, and jit.window seems to be the only reliable object to do that (i can't define the size of rendering context with matrix for instance).
PS: I am using the jit.asyncread to get the matrix out of openGL (in combination with jit.videoplane, as it seems that it doesn't work without?)
Havn't tried your patch, but have you tried adding "@matrixoutput 1" to the GL objects you want as outputs?
to get rid of your openGL window, you can add "@visible 0" to your jit.gl.render
jit.gl.slab does not have the option for "@matrixoutput 1", at least is not listed in the reference file. i tried anyway but it doesn't work. also i have bad experience with that attribute because it seems that it just don't work. probably there would be no need for jit.asyncread if that would work. also the "complications" with "@cathch something" and "@name something" when using jit.gl.texture to get the matrix out of openGL would not be necessary.
if i put "@visible 0" to jit.window the window disappears but also my openGL output stops working at the same time. on the other hand nothing happens if i put "@visible 0" to jit.render.
thanks anyway for your answer!
so is there a way to get a 32-bit floating point matrix out of openGL or not? i'm sure someone has to know the answer...please show some mercy:)
I've been noodling on this. Here are some ideas that I would explore if this was my project.
Perhaps use CUDA or OpenCL to do the work. If you have a graphic card which supports one/other, then tap into the GPU that way. https://cycling74.com/forums/opencl-in-jitter-part-one-jit-cl-noise
Instead of rendering to a window or pwindow, try rendering instead to a texture using the @capture functionality. This will remove the need for your visible windows. And as a guess, I wonder if you can declare the texture to be float32 with the # of planes you need. And then write a GL shader to send the data to those planes. This is a wild idea and I have no experience taking this approach.
Also, if you only need a single floating point number as output, you might be able to get it to work by casting the 4-char data into a 1-float matrix. You would need to to your OpenGL manipulation to set it up to write the correct floats into the correct bytes. And then use jit.coerce
Just connect the output of jit.gl.slab to jit.matrix.
jit.matrix supports a jit_gl_texture message for reading from a texture form slab's output. jit.gl.asyncread doesn't support floating point output.
See the jitter example: render/slab/jit.gl.slab-float-readback.maxpat
Thanks million times Joshua!!! I've been trying to find a way to do that for about two months! I've just tried applying separately horizontal and vertical Gaussian blur to amplitude plane of FFT matrix and it sounds soooo sexy:D
Also thanks to you diablodale!
quote: "Instead of rendering to a window or pwindow, try rendering instead to a texture using the @capture functionality. This will remove the need for your visible windows."
I've tried that but I don't know how to set the size of openGL rendering context that way. And I need it to be of size 200x2048. I've also tried to define the rendering context with jit.matrix instead of jit.window but again, can't set the size.
Set the window to @visible 0 (you still have to have it in your patch as the actual context) and set the context size by using @dim 200 2048 on the texture you're capturing to :)
Picking up this thread after a long time. Is there any possibility that jit.gl.asyncread will get a float32 mode in the future? I'm working on a motion analysis patcher, and doing readback (even of a 1-pixel texture) by connecting jit.gl.slab directly to jit.matrix gives noticeably worse framerates than using jit.gl.asyncread, but it would be really great to have floating point precision with the latter method to be able to register small movements.