GLSL 3D LUT
Christophe Burkman
12月 05 2019 | 4:57 午後
Hi there,
Trying to get my hands on 3D lut in max jitter, i spend last days searching the web findin some help.
The best i could fin is this topic on isadora forum :
https://community.troikatronix.com/topic/4869/using-lut-s-in-isadora/2
For short, mark creating a nice and working GLSL importing a 3DLUT as png image.
Here is the code as you find it in the glsl object :
I tried for some hours now to adapt this shader to max but didn't get any result.
As it is my first dive in GLSL, i thought a little help wouldnt hurt.
Any tips for getting it right plz ?
Thanks !
;)
Christophe Burkman
12月 06 2019 | 8:10 午後
I think the vertex shader might be the problem. Isadora side, this part is blank ( see attached ). So i get a basic one from max with : "<program name="vp" type="vertex" source="sh.passthru.xform.vp.glsl" />". But i did it pretty blindly.
I'm also trying to understand the fragment shader but after the declaration of the "textureColor", i'm lost. I dont understand why "textureColor.b" is multiply by 63 next line, while "textureColor" is from the video input and not the LUT input. I get that 0 to 63 means 64 and 8*8 = 64, which is the LUT size. But beyond that, it's a black hole for me.
Yet i do confirm it's working great from isadora despite a 2 years old topic/code ( see attached too )
So for, no real progress.
Any help plz ?
Thanks !
Christophe Burkman
12月 06 2019 | 8:48 午後
I should add that there is more than simple LUT work beyond this topic. And if we manage to get it right, i might have a really good surprise even for those not interested in LUT color.
To be continued...
;)
Andrew Benson
12月 06 2019 | 8:54 午後
Christophe Burkman
12月 06 2019 | 9:24 午後
Hi Andrew,
Thanks for your feed back. It's a very different approach than the original GLSL. I'm really aiming 3dLUT files from dedicated software like resolve, that could be then used inside max for movies or live video feed.
But the levels shader is a great tool for other works.
Thanks again o/
Andrew Benson
12月 07 2019 | 12:36 午前
I think it helps to look at the format of the 3DLUT images themselves. The image represents a 64x64x64 pixel cube across a 512x512 grid. Basically the Z-axis is represented by unwrapping the 64x64 boxes. So, the shader is looking up a value between 2 successive squares, and then interpolating between those values to using the Z (blue) value. I can't see anything obvious about the shader itself, but if you post your patcher and JXS file here, I'd be happy to take a look.
Christophe Burkman
12月 07 2019 | 10:09 午前
I did not purchase Max yet so cannot save and share patch but here is a sreen. ( 30 days patch saving trial is too short for me ) It's really simple.
At full intensity i get black image. At 0 intensity, i get unmodified video source ( bball.mov here ).
And as you say, the code divide the png to acces the slices of the LUT cube in the z axis aka blue axis. But for me, blueColor, comes from textureColor, which comes from tex0, texcoord0, which is my video input. I must miss something ^^.
And here is the code :
Christophe Burkman
12月 08 2019 | 4:12 午後
Andrew Benson
12月 08 2019 | 7:21 午後
I took a closer look and I think I know what’s happening. The original shader uses normalized texture coordinates (0...1) whereas texture2DRect() expects actual pixel coordinates. This means you are only ever sampling the first pixel from either texture. A simple multiply at the end would fix it
Christophe Burkman
12月 09 2019 | 1:44 午後
Indeed, i tried to rotate my png LUT to get red in the top left corner and i get red image.
This confirms your theory.
As i get the general idea, i dont see where the multiplication should happen.
Thanks for your help Andrew.
Andrew Benson
12月 09 2019 | 5:54 午後
First, I would use the "texcoord0" varying variable passed from the passthru vertex shader here for the original texture lookup:
vec4 textureColor = texture2DRect(tex0, texcoord);
Note that in your shader it's defined as "texcoord" which also needs to be changed to "varying vec2 texcoord0". After that first texture lookup, a lot of the shader is just establishing the texture lookup coordinates for the 3D LUT image. These end up as normalized vec2 values texpos1 and texpos2. You'll need to multiply those vec2 values by the size of the lookup texture, so if it's a 64x64x64, it should be 512x512 dimensions:vec2 tex1size = vec2(512. , 512.);
vec4 newColor1 = texture2DRect(tex1, texPos1*tex1size);
vec4 newColor2 = texture2DRect(tex1, texPos2*tex1size);
Technically, since the LUT texture is power of 2, you could safely use sampler2D/texture2D - which use normalized coordinates - instead of the Rect functions, but it's probably easiest to just stick with this for now.
Christophe Burkman
12月 09 2019 | 7:09 午後
I entierly replaced my vertex shader by your "sh.passthrudim.vp.glsl ", to be sure not missing anything.
I updated the variable texcoord0 accordingly and add the 512 scaling as you suggested.
We got something better, but out of space colors ^^ :
Intensity 0 = video unchanged
Intensity 1 = see below
Noting that for this test i used a based png LUT with no change, which should gives us unmodified video even at 1 intensity. Trying other png LUT give different results though.
Despite the color problem, i think we get close and i thank you for your help.
Andrew Benson
12月 09 2019 | 7:34 午後
Just remove the 2 lines that flip the y axis of the lookup coordinates:
//texPos1.y = 1.0 - texPos1.y;
//texPos2.y = 1.0 - texPos2.y;
Christophe Burkman
12月 09 2019 | 7:52 午後
That's it !
I thought it might be some inverted value so i first tried to negate intensity. But your suggestion is THE solution. Perfect.
Thousand thanks to you Andrew.
I'll be back shortly with more results.
Christophe Burkman
12月 09 2019 | 9:25 午後
The great thing about this is double :
One, you can get your video in any color grading software, do your grade, apply it to your png LUT reference, export it as a second png and use the shader on video files or live camera as you wish. You can then fade between different grade for different effect etc...
Two, the surprise. And that is interesting.
Now imagine that you dont use it to get a color grade but a color key. Rather than getting your video in a color grade tool, you get it in a compositing tool and pull the best chroma key you could have with it*. You apply the key to your reference png LUT and export it as a black and white matte.
Back to max, you can use the shader to apply this LUT to your video and get a super clean and efficient matte from your green/blue screen, video files AND live camera. You do the basic alpha blend to apply this matte to your source video and you get a high end GPU chroma key in no time. Blurring and softing your matte is easy patching if you need.
*note that as it is all based on color LUT, every non color aspect of a chroma key ( shrink, grow, blur, despill ) wont be exported properly. But all of those operations can easily be done in max.
The final jxs shader, thanks to ppl from isadora and Andrew for the Max translation :
Andrew Benson
12月 10 2019 | 12:07 午前
Thanks for sharing back. Really cool idea, and I've been enjoying the push to look more at LUT grading. There's some good info on generating hald png images over here: https://streamshark.io/obs-guide/converting-cube-3dl-lut-to-image
Mark Coniglio
9月 23 2021 | 8:45 午前
Hey there. Someone just alerted me to this thread and I'm happy that my code could generate such a discussion.
The idea of using a LUT to do Chroma Key is really interesting. I'm going to look into that myself.
Finally, thanks to Christophe Burkman for crediting my original code. It's a bit frustrating when I've seen others present my work as their own, so I am very grateful for the attribution. ;-)