Re: Re: RE: Depth of field style blur shader + global illum/soft shadow shader?

Axiom-Crux's icon

Havent had the chance to try it out yet, been busy with freelance and this installation im working on for MOCA. Anyone else?

I was thinking it might work to use a modified blur shader that detects distance from camera and blurs each object based on that, but im still a baby at this.

Axiom-Crux's icon

I was looking at the different messages that you can send to jit.gl.render, and I saw something that I cant find again, but it was somethign along the lines of capture depth buffer... or something like that.. is there a way to get the depth buffer into a texture or matrix?

http://farm4.static.flickr.com/3523/3297621481_1419f26916.jpg?v=0

wow finally links and images... I could go without the smilies...

____. .____
,.__________________._._. <<____/ |::| ____>>._._.__________________.,
-~=:!{[,,___________ ,.-~=:!ili!({[{[{[ www.axiom-crux.net ]}]}]})!ili!:=~-.,___________,,]}!:=~-
._/-_/-_/,.-----------' " \_, ,_, ,_// / ''' ' ' `----------.,__-_-_.
..----------------------../

wallace winfrey's icon

Axiom-Crux wrote on Sat, 21 February 2009 19:07
wow finally links and images... I could go without the smilies...

It's not permanent. Remember that you can turn them off in your post ("Disable smilies in this post").

After it got to 24 hours, I figured people would rather have an imperfect, yet being worked on forum back rather than no forum at all.

w

Axiom-Crux's icon

are you referring to the smilies or the other tools as well? I quite like having images and html links... I think those speed up things alot, no more copy paste, just click. Images are nice too so we can show patch stills and such without also having to copy paste a link.

Axiom-Crux's icon
nesa's icon

Axiom-Crux wrote on Sun, 22 February 2009 03:07is there a way to get the depth buffer into a texture or matrix

Yes, there's a depth_grab message that copies the contents of the depth buffer to a named texture.

We've been using it to create depth of field type effect where the blur amount is based on the distance of the pixel from the focal point - this is a common technique for most games, imo.

After some investigation, we found out that depth_grab is pretty expensive operation, so we made a shader that writes depth info to gl_FragColor and render the whole scene twice, which in our case proved to be far more efficient that depth_grab.

Axiom-Crux's icon

can I check out your depth of field effect? I have been looking for something like that. I am trying to learn shaders to do it myself but its taking a while to get up to speed.

Thank you either way nesa, your always a big help!

also, any thoughts on the technique on this page near the bottom:http://fly.srk.fer.hr/~unreal/theredbook/chapter10.html

looks like they use the frame accumulation buffer to draw the frame a few times from slight different camera positions and that makes it slightly blurred ish.. I dont know if thats realistic for complex scenes in reatime.. maybe your technique with shaders is best.

Wesley Smith's icon

I would definitely recommend writing out depth information via a shader. This is how most real-time graphics engines do it these days (as opposed to using the depth buffer). Aside from potential performance issues, the reason for going this route is to have a linear encoding of distance. The depth buffer is non-linear for reasons I won't go into suffice to say that it's difficult/expensive to apply certain operations when the data is nonlinear.

If you're going to try this out, be sure to set your texture type to float32 otherwise depth values will be clamped to [0, 1].

wes

wallace winfrey's icon

Axiom-Crux wrote on Sat, 21 February 2009 21:54are you referring to the smilies or the other tools as well?

Just the smilies.

nesa's icon

Axiom-Crux wrote on Mon, 23 February 2009 01:24can I check out your depth of field effect?

Well, the problem is that our shaders do a lot of stuff that's particular to the system we're building that it would make more sense to write new shader. Unfortunately at the moment I don't have time to do it...
But I'd be glad to help you with building your own!

For the depth grabing shader you could use:

gl_FragCoord.z (more info here)

and use it to calculate gl_FragColor.

Then you apply this shader to whole scene and capture it to the texture that will contain depth info.

After this, you render&capture the whole scene again, but without 'depthshader'

Now you can send both textures to a modified blur shader that changes the width of blur kernel based on the values from depth texture.

So there's a lot of things to do to make DOF shader, even more to make it look good, but for start I would suggest just writing a simple shader that displays depth of each pixel, and when that is working we can continue for next step.

Quote:also, any thoughts on the technique on this page near the bottom:http://fly.srk.fer.hr/~unreal/theredbook/chapter10.html

This is how my grandpa would do it. And he would still get one WHAO! from me.

Axiom-Crux's icon

cool, Ill get to work on it as soon as I am done with my current installation project for MOCA.

I had figured that it would work like what you mentioned, though I wasn't aware I would need to do 3 seperate renders for the whole thing. Also, to make the whole scene 1 shader(for the depth), do I have to assign each object to that shader seperately, or is there a way to send the whole render context to a shader, then bang that to a capture, then turn that shader off, bang that to another capture, and then apply the final blur kernel shader combinding the two and yet another render bang.

Is this crazy low fps?? can you do this at hd resolution or will it just be way too slow? do you need a really good graphics card to be able to do this at realtime smooth speeds?

Thanks for all the help! Id love to see any documentation (pictures or videos or anything) of the project your working on, as Im curious to see how this looks in the end. I am also just curious to see your work, as Ive seen many of your patches through the list and they are always quite innovative and clever.

Thanks again nesa, and wes!

Axiom-Crux's icon

I was looking around and I remembered seeing a shader for vvvv that does depth of field. I went to their site and found it, its attached here. It says on their info page that they use HLSL, high level shader language. Is this usable in jitter? I remember hearing GLSL and CG and other abbreviations for shader types but I dont recall HLSL. I am curious if the attached files can be converted to work with jitter. Ive also added some links to the vvvv pages I referenced.

Axiom-Crux's icon

Ok I have some time to work on this now, Im making the basic depth shader right now. I am curious if I can do the entire depth of field inside 1 shader? or do I have to do as I mentioned before and use a bunch of textures and combine them?

Axiom-Crux's icon

    Shader using color imported from opengl
        //________________Variables

//________________vertex shader
//__________________end vertex shader

//________________fragment shader
//__________________end fragment shader

So here is my first shader, its a basic depth shader, you can set the focus distance by setting the brightness of the object. Any thoughts on where to go from here? Id like to slowly build it up into a full fledged crazy shader with textures for specular, bump, normal, displacement, depth of field, fog, etc... but obviously I need to start small. I want to try and make a nice depth of field shader first. So any recommendations from here?

Axiom-Crux's icon

I think the one thing I am quite lost at is making a blur that references a texture for the blur amount. I understand it conceptually, but I don't know where to start with making a blur that responds to each pixel and blurs based on the brightness of that pixel.

nesa's icon

Hi N,

Axiom-Crux wrote on Fri, 27 March 2009 05:54I think the one thing I am quite lost at is making a blur that references a texture for the blur amount.

Basically, you need multiple render passes to display the final image:

grab depth
grab color
postproces+display

your blur/DOF shader would be in a slab used in postprocess part.
One input to slab would be the color texture, other input is the depth texture.

That shader can be a modified gaussian blur - cf.gaussian.2p.jxs

You would need to add additional texture input to it, then (for a start) you could just use the value of your depth texture to do a dry/wet mix between blurred and input texture, perhaps with something like this:

modify this line:

gl_FragColor = 0.1752 * sampleM + 0.1658 * (sampleB0 + sampleF0) + 0.1403 * (sampleB1 + sampleF1) + 0.1063 * (sampleB2 + sampleF2);

to

vec4 blurred = 0.1752 * sampleM + 0.1658 * (sampleB0 + sampleF0) + 0.1403 * (sampleB1 + sampleF1) + 0.1063 * (sampleB2 + sampleF2);

with this you have stored blurred pixel, and in next step you can sample the depth texture and use that value to do the final mix:

float depth = texture2DRect(depth,texcoordM);
gl_FragColor = mix(sampleM, blurred, depth);

I don't know if this code is 100% correct as I typed it directly in browser, but it should work as pseudocode:)

For a nicer effect, you could use depth information to modify the width of the kernel instead of just mix amount.

Axiom-Crux's icon

Yeah thats what I was thinking of doing, modifying the blur width rather then the mix, but I am not sure how to make a blur based on a texture. Ill look around and see if I can find something on the net in that realm, but for now your mix method seems like a good way to get a quick proof of concept version done.

Thanks again

Axiom-Crux's icon

Is the code you typed glsl? I wasn't sure if it was just your pseudocoding, it seemed like it was a bit different.

Man I been reading the orange book and studying tutorials for 2 days, and I can say that I am way geeked to finally be getting into this, after seeing various demos on the net I think that shaders are really the only way to go for creating lush high resolution imagery in jitter. Its really cool to know that they will be adaptable into a higher level language if I ever have the need too!

Andrew Benson's icon

Not sure how much this helps your cause, but I had written a luminance-based version of the built-in cf.gaussian.2p.jxs awhile back and finally got around to making it a little more presentable. It will probably be a lot more effective in a multi-pass setup (a la jit.gl.slab.gauss6x). HTH.

Andrew B.

Axiom-Crux's icon

perfect, sounds like exactly what I needed, thanks andrew! Ill check it out and try and hook up a depth of field shader in the mornin.

on a side note, Im also trying to figure out how to do shadows. I found very little documentation on the things I need in jitter. In the texture reference here is what I found:

>>>>this part talks about shadow mapping, only reference to shadows in all of the jitter distro!!!

compare_func     symbol         Sets the comparison function for shadow mapping when the texture is a depth texture.
compare_mode     symbol         Sets the comparison mode for shadow mapping when the texture is a depth texture.

other relevant info_____________________

capture_buffer     symbol          The word capture_buffer, followed by a symbol, sets which buffer to capture when the target of an ob3d @capture attribute. Valid options are:
color
depth
capture_depthbits     int         The word capture_depthbits, followed by a number, sets the number of bits of the depth buffer for render-to-texture mode.
capture_source     symbol         The word capture_source, followed by a symbol, sets the name of the target buffer to read from (such as the depth buffer) of an RTT texture.
colormode     symbol         The colorspace mode to use. (default = auto) Supported modes are:
alpha
depth
intensity
luminance
lumalpha
rgb
argb
uyvy
auto

so what symbol should I send? how do I set something up to make a shadow texture buffer? has anyone done it ever?? Ive looked a ton!!

thanks guys!

Axiom-Crux's icon

so here is a basic implementation with just jitter depth and andrews blur, make sure you download the file andrew posted if your going to try this out. Im curious if others get the same error I do, which is that hardware doesn't support depth textures, even though it works fine, maybe its a loadbang order thing.

going to put in my shader version shortly, but that wont take long, more concerned with figuring out how to calculate accessability in realtime for the ambient occlusion shader, and tryng really hard to figure out how the heck to get shadows working, I cant believe I cant find anyone who has done shadows in jitter before..??!!? I found one thing from a long time ago where wes or someone did a basic shadow type thing, but it wasn't real shadows, just a flat capture texture that was emulating a shadow, and it doesn't seem to work in my computer. Is there anyone who has done shadows in jitter? I know im not the first to try it.

strimbob's icon

i get
jit.gl.texture: unsupported texture format!
in the max window, but also the shader done not work not my graphics card, if i load the patch with out the shader in the search pat i get the same error

Axiom-Crux's icon

the blur shader doesn't work on your card? thats odd.. does anyone else know what this might be from? which card do you have and what computer/max version?

efe's icon

I had the same error, which is quite strange:
unsupported texture format

My specs:
MCP 10.5.6, 2.53 GHz+4 GB
NVIDIA 9400M+NVIDIA 9600M
Max 5.0.6

Emmanuel

strimbob's icon

i have a macbook
2 ghz intel core 2 duo
1 gb ram
graphic card GMA 950
osx 10.4.11
max 5.0.2

any ideas?

thanks ben

Axiom-Crux's icon

but did it work? did you try clicking the :jitter fbo or rtt boxes?

strimbob's icon

no did not work, tried the the box but nothing, when i change the shader in the slab i get a torus in the window

Léopold Frey's icon

Hello,

Can someone re-post Andrew's luminance-based blur shader ?
It seems it has been lost with the new forum.

Thanks

Léo

Dg's icon

Hi Léo

Here it is.

253.ab.lumagauss.jxs-help.maxpat
Max Patch
Léopold Frey's icon

Thanks

Léo

marleynoe's icon

Hi,

Could someone re-post Axiom-Crux's shader? Would be great, it seems it got lost with the new forum (or is it somewhere)?

Cheers,
M