how to depth_grab

Aug 27, 2006 at 11:31am

how to depth_grab

Hello e-family,

I’ve downloaded new betas for xp,(c74:thanks for this nice summer delight!),
and trying new jitter features.

I was wondering how could one capture depth buffer into texture and then to
matrix. Here’s what I’ve come up with, but it doesn’t seem to work as I
expected(I would like to see gray image showing depth of each pixel, but I
get rendered image(colors) as with ordinary render to texture).

Also, captured matrix shows strange values until I turn off rectangle
attribute of jit.gl.texture.

Here’s the patch:

#P window setfont Geneva 9.;
#P window linecount 1;
#N vpatcher 25 70 547 401;
#P window setfont Geneva 9.;
#P newex 50 146 113 9961481 jit.gl.gridshape out @smooth_shading 1
@lighting_enable 1 @depth_enable 1 @scale 0.2 0.2 0.2 @position -0.5 -0.5
0.;
#P newex 50 50 95 9961481 jit.gl.gridshape out @smooth_shading 1
@lighting_enable 1 @depth_enable 1 @scale 0.4 0.4 0.4 @color 0.5 0 0 1;
#P pop;
#P newobj 181 54 73 9961481 p gl.gridshapes;
#P message 313 95 58 9961481 rectangle 0;
#P user jit.cellblock 431 76 549 194 152 9 256 256 45 17 0 1 1 0 0 0 1 1 1 0
0 0 255 255 255 0 0 0 0 0 0 191 191 191 0 0 0 215 215 240 1 1 1 0 4 0 0 0;
#P comment 306 226 100 9961481 depth buffer:;
#P button 271 95 15 0;
#P button 293 95 15 0;
#P user jit.fpsgui 156 119 60 9961481 0;
#P newex 293 156 124 9961481 jit.matrix 4 char 256 256;
#P user jit.pwindow 292 241 258 258 0 1 0 0 1 0;
#X name tex;
#P user jit.pwindow 13 241 258 258 0 1 0 0 1 1;
#X name out;
#P newex 293 127 128 9961481 jit.gl.texture out @name de;
#P message 42 136 71 9961481 depth_grab de;
#P flonum 87 34 35 9 0 0 0 152 0 0 0 221 221 221 222 222 222 0 0 0;
#P toggle 42 34 15 0;
#P newex 42 56 55 9961481 qmetro 10;
#P newex 42 108 56 9961481 t b b erase;
#P newex 65 169 157 9961481 jit.gl.render out @depth_enable 1;
#P comment 27 226 100 9961481 gl scene:;
#P connect 10 0 9 0;
#P fasten 10 0 15 0 298 180 423 180 423 66 436 66;
#P fasten 13 0 10 0 276 149 298 149;
#P connect 7 0 10 0;
#P connect 2 2 1 0;
#P connect 6 0 1 0;
#P connect 2 1 1 0;
#P connect 16 0 7 0;
#P connect 3 0 2 0;
#P fasten 3 0 11 0 47 101 161 101;
#P connect 12 0 7 0;
#P connect 2 0 6 0;
#P connect 4 0 3 0;
#P connect 5 0 3 1;
#P window clipboard copycount 18;

bye,
n

#27320
Aug 28, 2006 at 8:41pm

On Aug 27, 2006, at 4:31 AM, nesa wrote:

> I was wondering how could one capture depth buffer into texture and
> then to matrix. Here’s what I’ve come up with, but it doesn’t seem
> to work as I expected(I would like to see gray image showing depth
> of each pixel, but I get rendered image(colors) as with ordinary
> render to texture).

Hi Nesa,

There’s two problems exposed here:

1. we don’t currently support copying depth textures back to a jitter
matrix directly. You’ll need to render to either a char or floating
point pbuffer with something like slab.

2. sometimes when changing texture parameters which cause a texture
to be recreated internally (like sending the message “rectangle 0″),
the @capture_buffer is set back to the default color buffer and
remains that way.

I’ve included a revised version of your patch which demonstrates how
you can use depth textures and read them back to host memory if
necessary. However, I’ve discovered a texture coordinate flipping
issues with reading back as char data to RAM, and an initial readback
issue where the first attempt to readback of a texture which has
never been bound will be all black.

Thanks for taking the time to investigate this new (and largely
undocumented) feature. Some aspects of it may be changing in the next
few months as we resolve these and other issues.

-Joshua

#P window setfont “Sans Serif” 9.;
#P number 335 36 35 9 0 0 0 3 0 0 0 221 221 221 222 222 222 0 0 0;
#P window linecount 1;
#P message 335 57 56 196617 dim $1 $1;
#P window linecount 4;
#P comment 344 163 177 196617 note that depth buffer resolution is
normalized across the range near clip to far clip. try making this
distance smaller to get more depth resolution;
#P window linecount 1;
#P message 414 218 14 196617 3;
#P message 341 220 23 196617 1.5;
#P flonum 341 241 35 9 0 0 0 3 0 0 0 221 221 221 222 222 222 0 0 0;
#P flonum 414 239 35 9 0 0 0 3 0 0 0 221 221 221 222 222 222 0 0 0;
#P message 414 262 61 196617 far_clip $1;
#P message 341 263 67 196617 near_clip $1;
#P window linecount 4;
#P comment 858 289 285 196617 note that some HW may not permit
ordinary texture sampling of depth textures within GLSL shaders ,
and that most jitter shaders assume rectangular textures in order for
the shader to work properly;
#P flonum 774 254 35 9 0 0 0 3 0 0 0 221 221 221 222 222 222 0 0 0;
#P flonum 675 253 35 9 0 0 0 3 0 0 0 221 221 221 222 222 222 0 0 0;
#P window linecount 1;
#P message 774 276 74 196617 param bias $1;
#P message 675 275 79 196617 param scale $1;
#P user gswitch2 141 130 39 32 0 0;
#P newex 193 175 40 196617 t swap;
#P window linecount 3;
#P comment 17 128 100 196617 toggle viewing depth texture rendered on
a plane;
#P user jit.fpsgui 554 365 60 196617 2;
#P window linecount 1;
#P message 606 275 53 196617 type char;
#P message 547 275 54 196617 type float;
#P window linecount 4;
#P comment 627 352 155 196617 oops. looks like there’s some issue
with texure coordinate flipping and readback with type char. will
investigate;
#P window linecount 1;
#P newex 482 75 106 196617 bgcolor 127 127 127;
#P user jit.pwindow 528 405 258 258 0 0 0 0 1 1;
#X name u335000181;
#P newex 529 339 55 196617 jit.matrix;
#P newex 529 304 171 196617 jit.gl.slab out @file cc.scalebias.jxs;
#P window linecount 3;
#P comment 856 245 285 196617 reading back depth textures directly to
a jitter matrix is not currently supported , you will need to render
to a color buffer texture (e.g. through slab) , prior to texture
readback.;
#P window linecount 1;
#P message 623 202 105 196617 capture_buffer depth;
#P window linecount 2;
#P newex 138 323 239 196617 jit.gl.gridshape out @color 1. 1. 1. 1.
@shape plane @texture de @transform_reset 2 @automatic 0;
#P window linecount 1;
#P newex 170 94 30 196617 t b b;
#P newex 105 175 76 196617 t swap b erase;
#N vpatcher 25 70 547 401;
#P inlet 172 32 15 0;
#P window setfont “Sans Serif” 9.;
#P newex 50 146 113 196617 jit.gl.gridshape out @smooth_shading 1
@lighting_enable 1 @depth_enable 1 @scale 0.2 0.2 0.2 @position -0.5
-0.5 0.;
#P newex 50 50 95 196617 jit.gl.gridshape out @smooth_shading 1
@lighting_enable 1 @depth_enable 1 @scale 0.4 0.4 0.4 @color 0.5 0 0 1;
#P connect 2 0 0 0;
#P connect 2 0 1 0;
#P pop;
#P newobj 335 76 73 196617 p gl.gridshapes;
#P message 549 202 58 196617 rectangle 0;
#P button 529 202 15 0;
#P user jit.fpsgui 291 104 60 196617 0;
#P user jit.pwindow 121 401 258 258 0 1 0 0 1 1;
#X name out;
#P newex 529 236 133 196617 jit.gl.texture out @name de;
#P message 238 202 71 196617 depth_grab de;
#P flonum 215 49 35 9 0 0 0 3 0 0 0 221 221 221 222 222 222 0 0 0;
#P toggle 170 49 15 0;
#P newex 170 71 55 196617 qmetro 10;
#P newex 238 175 106 196617 t b drawclients erase;
#P newex 216 296 157 196617 jit.gl.render out @depth_enable 1;
#P comment 126 377 100 196617 gl scene:;
#P window linecount 3;
#P comment 579 149 261 196617 sometimes needs to be explicitly set to
capture_buffer depth when changing texture attributes which
internally require a recreation of the texture;
#P connect 29 0 14 0;
#P connect 14 1 16 0;
#P connect 5 0 4 0;
#P connect 4 0 15 0;
#P connect 15 0 29 1;
#P connect 29 1 28 0;
#P connect 6 0 4 1;
#P connect 35 0 2 0;
#P connect 36 0 2 0;
#P connect 28 0 2 0;
#P connect 14 0 2 0;
#P connect 14 2 2 0;
#P connect 7 0 2 0;
#P connect 3 1 2 0;
#P connect 3 2 2 0;
#P fasten 15 1 3 0 195 143 243 143;
#P connect 3 0 7 0;
#P connect 15 1 10 0;
#P connect 43 0 42 0;
#P connect 42 0 13 0;
#P connect 39 0 38 0;
#P connect 38 0 35 0;
#P connect 40 0 37 0;
#P connect 37 0 36 0;
#P hidden connect 33 0 11 0;
#P hidden connect 32 0 11 0;
#P connect 12 0 8 0;
#P connect 11 0 8 0;
#P connect 17 0 8 0;
#P fasten 31 0 19 0 779 297 534 297;
#P fasten 30 0 19 0 680 297 534 297;
#P fasten 8 0 19 0 534 278 534 278;
#P fasten 24 0 19 0 552 297 534 297;
#P fasten 25 0 19 0 611 297 534 297;
#P connect 19 0 20 0;
#P connect 20 0 21 0;
#P connect 20 0 26 0;
#P connect 32 0 30 0;
#P connect 33 0 31 0;
#P window clipboard copycount 44;

#82564
Aug 28, 2006 at 9:43pm

Hi Nesa,
Here’s a patch I’ve been using recently to work with depth textures.
Currently this stuff works best with the FBO backend (as opposed to
the RTT backend). How to set this is demonstrated in the patch

best,
wes

#P window setfont “Sans Serif” 9.;
#P window linecount 1;
#P comment 105 333 165 9109513 depth buffer = the FBO’s depth buffer;
#P newex 19 325 63 9109513 s RTT_toggle;
#P toggle 19 306 15 0;
#P newex 19 283 38 9109513 sel 114;
#P newex 19 257 40 9109513 key;
#P hidden newex 156 120 28 9109513 sel 1;
#P newex 138 155 69 9109513 r RTT_toggle;
#P comment 410 312 59 9109513 Randomize;
#P newex 293 350 48 9109513 s render;
#P newex 25 187 48 9109513 r render;
#P flonum 293 312 35 9 0 0 0 139 0 0 0 221 221 221 222 222 222 0 0 0;
#P message 293 331 61 9109513 far_clip $1;
#P button 424 330 15 0;
#P newex 372 329 27 9109513 + 1;
#P toggle 372 308 15 0;
#P newex 336 85 48 9109513 loadbang;
#N vpatcher 425 245 929 504;
#P outlet 36 214 15 0;
#P window setfont “Sans Serif” 9.;
#P window linecount 0;
#P message 111 163 82 9109513 texture depth_buf;
#P newex 88 128 27 9109513 t b i;
#P newex 25 164 27 9109513 + 1;
#P inlet 88 109 15 0;
#P newex 36 191 50 9109513 switch 2 2;
#P newex 56 163 51 9109513 r surfaces;
#P comment 297 40 75 9109513 Depth Texture;
#P newex 24 78 248 9109513 jit.gl.texture RTT @name tex_scene2 @dim 720 480;
#P newex 24 56 248 9109513 jit.gl.texture RTT @name tex_scene1 @dim 720 480;
#P window linecount 1;
#P newex 297 54 171 9109513 jit.gl.texture RTT @name depth_buf;
#P comment 24 40 119 9109513 RTT Ping-Pong Textures;
#P fasten 9 1 8 0 110 158 30 158;
#P fasten 8 0 6 0 30 187 41 187;
#P connect 6 0 11 0;
#P connect 5 0 6 1;
#P fasten 10 0 6 2 116 185 81 185;
#P connect 7 0 9 0;
#P fasten 9 0 10 0 93 153 116 153;
#P pop;
#P newobj 124 379 62 9109513 p Texturize;
#N vpatcher 10 59 444 446;
#P origin -79 0;
#P inlet 60 166 15 0;
#P window setfont “Sans Serif” 9.;
#P window linecount 0;
#P newex 60 186 47 9109513 gate 2 2;
#P comment 258 91 100 9109513 set random speeds;
#N vpatcher 10 59 422 313;
#P origin -4 -4;
#P window setfont “Sans Serif” 9.;
#P window linecount 0;
#P newex 37 47 30 9109513 t b b;
#P newex 37 26 48 9109513 loadbang;
#P newex 57 179 143 9109513 jit.op @op + @val 0.1 0.1 0.1;
#P newex 57 158 151 9109513 jit.normalize @amp 0.5 0.5 0.5;
#P newex 57 109 105 9109513 prepend setattr basis;
#P message 57 89 232 9109513 detail 7. , roughness 0.8 , lacunarity
2.1 , rotate 1;
#P newex 57 136 292 9109513 jit.bfg 3 float32 5 5 @basis fractal.multi
@offset 20. 20. -20.;
#P inlet 17 111 15 0;
#P outlet 57 201 15 0;
#P window linecount 1;
#P newex 0 75 55 9109513 delay 100;
#P connect 9 0 0 0;
#P connect 8 0 9 0;
#P connect 9 1 4 0;
#P connect 4 0 5 0;
#P connect 5 0 3 0;
#P fasten 2 0 3 0 22 131 62 131;
#P connect 0 0 3 0;
#P connect 3 0 6 0;
#P connect 6 0 7 0;
#P connect 7 0 1 0;
#P pop;
#P newobj 240 121 77 9109513 p RandomVals2;
#N vpatcher 10 59 335 279;
#P origin 0 -3;
#P window setfont “Sans Serif” 9.;
#P window linecount 1;
#P newex 4 58 55 9109513 delay 100;
#P window linecount 0;
#P newex 42 25 30 9109513 t b b;
#P newex 42 5 48 9109513 loadbang;
#P newex 62 158 149 9109513 jit.op @op – pass pass @val 20;
#P newex 62 137 139 9109513 jit.normalize @amp 40. 3. 1.;
#P newex 62 90 105 9109513 prepend setattr basis;
#P message 62 70 232 9109513 detail 7. , roughness 0.8 , lacunarity
2.1 , rotate 1;
#P newex 62 115 195 9109513 jit.bfg 3 float32 5 5 @basis fractal.multi;
#P inlet 16 89 15 0;
#P outlet 62 180 15 0;
#P connect 8 0 9 0;
#P connect 7 0 8 0;
#P connect 8 1 3 0;
#P connect 3 0 4 0;
#P connect 4 0 2 0;
#P fasten 1 0 2 0 21 110 67 110;
#P connect 9 0 2 0;
#P connect 2 0 5 0;
#P connect 5 0 6 0;
#P connect 6 0 0 0;
#P pop;
#P newobj 161 91 71 9109513 p RandomVals;
#P newex 97 237 149 9109513 jit.op @op pass – pass @val 20;
#P button 240 91 15 0;
#P button 161 72 15 0;
#P newex 97 166 153 9109513 jit.op @op pass % pass @val 40;
#P newex 97 209 73 9109513 jit.matrix pos;
#P newex 97 143 153 9109513 jit.op @op pass + pass @val 0.2;
#P newex 97 121 136 9109513 jit.matrix pos 3 float32 5 5;
#P newex 97 94 20 9109513 t b;
#P window linecount 2;
#P newex 97 261 268 9109513 jit.gl.multiple RTT 1 @targetname me
@glparams position @automatic 0 @scale 0.1 0.1 0.1 @targetmode 0;
#P window linecount 1;
#P newex 97 66 60 9109513 r draw_rtt;
#P comment 157 29 100 9109513 set random offsets;
#P inlet 161 46 15 0;
#P connect 16 0 15 0;
#P connect 2 0 4 0;
#P fasten 12 0 5 0 166 114 102 114;
#P connect 4 0 5 0;
#P connect 5 0 6 0;
#P connect 6 0 8 0;
#P connect 8 0 15 1;
#P connect 15 1 7 0;
#P connect 7 0 11 0;
#P fasten 15 0 11 0 65 231 102 231;
#P connect 11 0 3 0;
#P connect 0 0 9 0;
#P connect 9 0 12 0;
#P fasten 0 0 10 0 166 67 245 67;
#P connect 10 0 13 0;
#P connect 13 0 6 1;
#P pop;
#P newobj 372 350 62 9109513 p Geometry;
#P message 133 486 34 9109513 reset;
#P newex 133 506 84 9109513 jit.gl.handle RTT;
#P button 80 486 15 0;
#P newex 80 506 48 9109513 loadbang;
#P window linecount 3;
#P newex 80 533 308 9109513 jit.gl.gridshape RTT @shape torus
@lighting_enable 1 @blend_enable 0 @smooth_shading 1 @automatic 0
@depth_enable 1 @dim 10 10 @name me @scale 0.1 0.1 0.1 @shader zalpha;
#P window linecount 1;
#N vpatcher 110 475 378 714;
#P window setfont “Sans Serif” 9.;
#P window linecount 0;
#P newex 27 196 60 9109513 s draw_rtt;
#P inlet 169 33 15 0;
#P newex 27 129 27 9109513 + 1;
#P newex 27 152 47 9109513 gate 2 2;
#P newex 64 175 41 9109513 s draw;
#P window linecount 1;
#P message 87 76 76 9109513 capture , erase;
#P newex 41 53 56 9109513 t b draw b;
#P inlet 41 33 15 0;
#P outlet 87 96 15 0;
#P outlet 41 75 15 0;
#P fasten 8 0 7 0 174 125 32 125;
#P connect 7 0 6 0;
#P connect 6 0 9 0;
#P connect 2 0 3 0;
#P connect 3 0 0 0;
#P connect 3 1 6 1;
#P connect 6 1 5 0;
#P connect 3 2 4 0;
#P connect 4 0 1 0;
#P pop;
#P newobj 76 187 49 9109513 p Screen;
#N vpatcher 71 448 598 736;
#P origin -45 -32;
#P window setfont “Sans Serif” 9.;
#P newex 345 178 73 9109513 prepend texture;
#P newex 2 104 65 9109513 s RTT_swap;
#P window linecount 2;
#P comment 66 186 95 9109513 Grab the depth buffer.;
#P window linecount 0;
#P message 83 138 107 9109513 depth_grab depth_buf;
#P inlet 133 22 15 0;
#P newex 65 54 47 9109513 gate 1 1;
#P newex 345 201 57 9109513 s surfaces;
#P message 410 154 62 9109513 tex_scene2;
#P message 345 154 62 9109513 tex_scene1;
#P newex 209 176 82 9109513 prepend capture;
#P message 275 152 62 9109513 tex_scene1;
#P message 209 152 62 9109513 tex_scene2;
#P newex 209 117 57 9109513 jit.cycle 2;
#P newex 209 198 48 9109513 t erase l;
#P newex 65 75 64 9109513 t b b draw b;
#P newex 101 116 60 9109513 s draw_rtt;
#P inlet 102 24 15 0;
#P outlet 65 162 15 0;
#P outlet 209 223 15 0;
#P comment 222 55 242 9109513 This section Ping-Pongs the RTT buffer
because you can’t read/write to the same buffer simultaneously.;
#P fasten 3 0 18 0 107 43 7 43;
#P fasten 15 0 14 0 138 49 70 49;
#P connect 14 0 5 0;
#P fasten 16 0 2 0 88 158 70 158;
#P connect 5 0 2 0;
#P connect 5 1 16 0;
#P connect 5 2 4 0;
#P connect 3 0 14 1;
#P fasten 5 3 7 0 124 104 214 104;
#P connect 7 0 8 0;
#P lcolor 15;
#P fasten 9 0 10 0 280 172 214 172;
#P connect 8 0 10 0;
#P connect 10 0 6 0;
#P connect 6 0 1 0;
#P fasten 6 1 1 0 252 219 214 219;
#P fasten 7 1 9 0 237 139 280 139;
#P lcolor 14;
#P fasten 7 0 11 0 214 145 350 145;
#P lcolor 15;
#P connect 11 0 19 0;
#P fasten 12 0 19 0 415 174 350 174;
#P connect 19 0 13 0;
#P fasten 7 1 12 0 237 139 415 139;
#P lcolor 14;
#P pop;
#P newobj 129 187 36 9109513 p RTT;
#P newex 79 380 41 9109513 r draw;
#P newex 76 159 30 9109513 t b b;
#P window linecount 2;
#P message 336 137 102 9109513 ; jitter glreadback rtt;
#P message 336 106 107 9109513 ; jitter glreadback fbo;
#P newex 79 415 298 9109513 jit.gl.videoplane RTT @blend_enable 0
@color 1 1 1 1 @depth_enable 0 @automatic 0 @transform_reset 2
@texture depth_buf;
#P toggle 217 152 15 0;
#P window linecount 1;
#P message 217 172 68 9109513 fullscreen $1;
#N vpatcher 30 89 166 253;
#P window setfont “Sans Serif” 9.;
#P newex 50 71 35 9109513 sel 27;
#P newex 50 50 40 9109513 key;
#P outlet 50 93 15 0;
#P connect 1 0 2 0;
#P connect 2 0 0 0;
#P pop;
#P newobj 217 132 33 9109513 p Esc;
#P newex 217 190 222 9109513 jit.window RTT @depthbuffer 1 @size 720 480;
#P toggle 76 113 15 0;
#P newex 76 135 57 9109513 qmetro 40;
#P newex 76 219 382 9109513 jit.gl.render RTT @erase_color 0 0 0 1
@camera 0 0 4 @near_clip 1. @far_clip 5.;
#P comment 104 88 198 9109513 Use ‘r’ to toggle RTT and normal rendering;
#P window setfont “Sans Serif” 20.;
#P comment 54 43 206 9109524 RTT and Depth Buffer;
#P window setfont “Sans Serif” 9.;
#P window linecount 2;
#P comment 355 279 59 9109513 Start/Stop Movement;
#P user umenu 124 356 100 9109543 1 64 372 1;
#X add Scene;
#X add depth buffer;
#P window linecount 1;
#P comment 105 316 140 9109513 Scene = the FBO’s color buffer;
#P window linecount 2;
#P comment 326 53 117 9109513 Depth grabbing only works with FBOs right now;
#P connect 38 0 39 0;
#P connect 39 0 40 0;
#P connect 40 0 41 0;
#P connect 8 0 7 0;
#P connect 7 0 16 0;
#P connect 16 0 19 0;
#P fasten 18 1 6 0 160 213 81 213;
#P fasten 19 0 6 0 81 213 81 213;
#P fasten 19 1 6 0 120 213 81 213;
#P fasten 18 0 6 0 134 213 81 213;
#P fasten 33 0 6 0 30 213 81 213;
#P fasten 26 0 13 0 129 404 84 404;
#P connect 17 0 13 0;
#P connect 22 0 21 0;
#P connect 21 0 20 0;
#P fasten 23 0 20 0 138 528 85 528;
#P fasten 36 0 19 1 143 182 120 182;
#P lcolor 13;
#P connect 2 0 26 0;
#P fasten 16 1 18 0 101 179 134 179;
#P connect 24 0 23 0;
#P fasten 36 0 18 1 143 182 160 182;
#P lcolor 13;
#P hidden connect 8 0 37 0;
#P connect 10 0 12 0;
#P connect 12 0 11 0;
#P connect 11 0 9 0;
#P connect 32 0 31 0;
#P connect 31 0 34 0;
#P connect 27 0 14 0;
#P connect 28 0 29 0;
#P connect 29 0 25 0;
#P hidden connect 37 0 30 0;
#P connect 30 0 25 1;
#P window clipboard copycount 43;

#82565
Aug 29, 2006 at 6:57pm

Thank you Joshua and Wesley, I see how it works now!

Thanks for the ping-pong tip too!

#82566

You must be logged in to reply to this topic.