Using jit.gl.pix for particle system entirely on the GPU
Hi,
I'm currently working on some experiments where I'm using jit.gl.pix to run a particle system which stays entirely on the GPU. I'm pleased with what I'm seeing so far, but have hit a bit of a dead end and have been banging my head against a brick wall trying to get past this, so wondered if somebody with some more jit.gl.pix or glsl experience could help me out...
The principle appears to work fine, and I've been able to make nice looking particle systems with well over 500,000 points that run at 30fps on my iMac, with a single texture holding particles x, y, age and opacity. However, I'm now wanting to use the particle x and y positions to look up data from another texture, and this is where I'm hitting trouble. The attached patch illustrates this: I am taking a camera input, passing this into a texture, and then I want the particles to take their opacity value from the nearest corresponding pixel in the camera image. I can achieve the effect that I want when I have the particles arranged in a nice neat grid (with coordinates generated from a jit.gencoord ) but when I use noise to generate the particle positions, it reveals that there's something going wrong in the sampling process. In the attached patch, I'd like the image from the camera to remain readable even as the particles positions become randomly spread: the jit.gl.pix is supposed to be sampling the camera texture based on the x and y positions of each particle. Try crossfading from the noise to the grid and see what happens.
Any input on this would be very much appreciated! I'm planning to combine this with a stam fluid implementation on the GPU and will share here if I get it all working!
Tom
code for texturetogeo shader:
Geometry from texture
OK. Why is it that 5 minutes after asking for help here, I always seem to find the solution? At least, I think I've found the solution. The problem was not with the jit.gl.pix at all, but with the shader that was transforming the texture data into vertex data: it was interpolating the incoming texture, and I found on the forums that sending a message 'sendoutput filter none' to the jit.gl.pix immediately before the jit.gl.mesh solved the problem. Hope this might be useful for somebody else in the future...
Asking for help teaches humility. Finding the solution for yourself 5 minutes after being humble teaches you patience and impulse control. :-)
I'm sure it also has something to do with turning my usual chaotic patchers into something that I hope that other people might be able to understand: in doing so, I begin to understand them a little more myself...
Hey Tom, I saw on your Vimeo page that you got this particle system linked up to a GPU fluid solver. The clips that you posted look fantastic! Would you consider posting your patch here? I've been looking at other GPU-based fluid sims (most recently, the one posted over here: https://cycling74.com/forums/sharing-is-fun-fluid-simulation-on-the-gpu/ ) and would love to see what's going on in your implementation.
Hi Syrinx,
I'm planning to share the patch here, but will need to do some tidying up first... once my current project calms down a bit I hope to get on to it.
Cheers
Tom
Hey Tom w,
It's a great looking patch, I was wondering have you gotten any further with it? I'm currently trying to implement GPU based fluid dynamics :)
I'm just about to revisit this work, so hopefully will be able to post an update next week...
Hi
For the sake of learning , does anyone have a copy of the tovertices.jxs file this patcher references?
Cheers,
Graham
Hi again,
As promised, here is a patcher with the fluid simulation and particle system combined, using the GPU for just about everything.
It's not pretty, and it may take some work to get it to do anything on your systems, but here are some general notes:
- The source video is important. I've built this whilst using a video of a dancer shot against a pure black background, which works well. If you use a webcam or other video source, you might have to do some fiddling to get anything worth watching out of the other end...
- The video engine for the mov is HAP - so you'll need to install this package (through the package manager) or specify a different engine.
- Some of the parameters are extremely sensitive, but I haven't set limits for most of the float boxes, so you may find the whole thing spirals out of control very quickly when adjusting things
- I've set the resolution of the fluid simulation and particle system quite low for speed - experiment with higher resolutions if your GPU lets you.
- Only ever tested on Max 7 on a Mac...
I hope that (with the above caveats), this might prove interesting to some people.
Much of the code is based on Jos Stam's stable fluids implementation, and much credit to JMC on these forums https://cycling74.com/forums/sharing-is-fun-fluid-simulation-on-the-gpu for the jit.gl.pix implementation of Jos Stam's stable fluids simulation
Thanks so much for posting this, Tom! Really looking forward to trying it out and learning from your work on this.
I was getting an Access Denied page when I tried downloading the patcher due to the "+" in the filename. This URL should work: https://cycling74-web-uploads.s3.amazonaws.com/57a99b6a5ec84d5f58dab497/2017-06-22T13:29:53Z/GPU_fluid%2Bparticles_simplified.maxpat . Here it is pasted inline as well:
Tom, when you have a chance, could you please post particleshader_2.jxs ? Thank you!
This is brilliant well done!
If you don't me asking, what GPU were you using?
Sorry for the late reply, I seem to have turned off email notifications!
My GPU is an AMD Radeon R9 M370X 2048 MB on a MBP Retina (2015)
Tom
This is really phenomenal work, Tom! Thank you for sharing the patch and shader. I'm looking forward to digging into the workings.
As a side note, I'm wondering whether this technique could be adapted to other vertex processing tasks? As an example, I'm trying to figure out how to use your shader to feed the matrix output from jit.gl.gridshape into jit.gl.mesh as a texture. This would allow subsequent geometry processing to be done on the GPU, which would be nice for efficiency. With my naive approach (just sending the matrix output of jit.gl.gridshape directly into a chunk of your patch), it seems like z-axis is getting mangled, however:
Anyone have any thoughts on how to accomplish this?
Well, this is embarrassing - turns out the problem was just that the matrix plane order (ARGB) needed to be swizzled to the texture plane order (RGBA). Here it is working, with brcosa and slide to demonstrate using fragment shaders to process geometry from jit.gl.gridshape (or whatever) on the GPU:
There's a little bug with this - the planes of the cylinder and the corresponding faces of the cube are missing, as is the mid-section of the capsule. The points are being drawn at the centre of the plane where they should be, just not dispersed across the whole surface. Anyone know why/how to fix it?
Hi Syrinx,
This is great, really interesting work. From what I can see, the shapes are all being drawn correctly - it's just that some shapes don't have vertices that evenly cover the surface (e.g. on the end of a cylinder the surface is made up of triangular polygons each with one point in the centre of the surface and two adjacent points on the rim). Have a look at the shapes in poly_mode 2 2 and they appear in the same way as yours.
To create a more representative pattern of 'dots' you would need to subdivide the geometry. I would probably choose to do this is a 3D package and import as obj files, but perhaps somebody has built shaders that can do the same thing on the fly?
Cheers
Tom