Forums > Jitter

blending/compositing whithout programmable shaders

September 9, 2006 | 8:25 pm

Hi y’all,

I’m having quite of pickle… i designed a video mixer using my GPU and everything works pretty well. I composed my video using jit.gl.slab objects and the nice array of included shaders, but now i’m porting my prototype to the pc it should be run on while performing, since that graphics card is quite better than the one on my laptop (on which i designed the patch)

Well here comes the issue. since the graphics card on the pc is somewhat older, it does not support programmable shaders… seen afterwards, kinda stupid of me not to think of that earlier but hey… at least i’m learning.

Now, is there a way to mix/blend/compose several movie (on a videoplane preferably) using only the GPU?
U see, I don’t wonna use the jit.op since that deliver’s a readback to a jit.matrix… using the CPU. And this all slows down the patch too much. I definitely need my CPU cycles elsewhere!…

I already tried out using the blend_enable parameter of the OB3D group, but this didn’t deliver the pixing as i would like…

Any help would be much appreciated, since i’m a little shy in time to get even some simple mixing to work…

thanks in advance!


September 9, 2006 | 8:31 pm

You can do this using the @blend_mode attribute and 2 videplanes with
@depth_enable 0 @blend_enable 1. See the OpenGL spec or the jitter
opengl HTML doc for more info on this.

wes

On 9/9/06, david wrote:
>
> Hi y’all,
>
> I’m having quite of pickle… i designed a video mixer using my GPU and everything works pretty well. I composed my video using jit.gl.slab objects and the nice array of included shaders, but now i’m porting my prototype to the pc it should be run on while performing, since that graphics card is quite better than the one on my laptop (on which i designed the patch)
>
> Well here comes the issue. since the graphics card on the pc is somewhat older, it does not support programmable shaders… seen afterwards, kinda stupid of me not to think of that earlier but hey… at least i’m learning.
>
> Now, is there a way to mix/blend/compose several movie (on a videoplane preferably) using only the GPU?
> U see, I don’t wonna use the jit.op since that deliver’s a readback to a jit.matrix… using the CPU. And this all slows down the patch too much. I definitely need my CPU cycles elsewhere!…
>
> I already tried out using the blend_enable parameter of the OB3D group, but this didn’t deliver the pixing as i would like…
>
> Any help would be much appreciated, since i’m a little shy in time to get even some simple mixing to work…
>
> thanks in advance!
>


September 9, 2006 | 8:48 pm

Hi,

thanks for your quick reply! but umm… i don’t get it to work. i’m not really seeing where i should insert the alpha value that should be used for blending…

below i inserted a simple patch containing two movies an o a plane, if someone could be so nice to check where i’m missing. that would be superb.

A link to the < < jitter opengl HTML reference -unless u mean the Gl group page from the jitter object reference- >> or any other information about this would be great too.

thanks a lot
d

#P window setfont "Sans Serif" 9.;
#P window linecount 1;
#P message 770 126 29 9109513 reset;
#P newex 767 200 134 9109513 pak position 0. 0. 0.;
#P flonum 808 179 35 9 0 0 0 139 0 0 0 221 221 221 222 222 222 0 0 0;
#P flonum 849 179 35 9 0 0 0 139 0 0 0 221 221 221 222 222 222 0 0 0;
#P flonum 890 178 35 9 0 0 0 139 0 0 0 221 221 221 222 222 222 0 0 0;
#P newex 770 149 113 9109513 jit.gl.handle motekScreen;
#P window linecount 2;
#P newex 679 227 220 9109513 jit.gl.videoplane motekScreen @scale 3.333 0.8 1. @depth_enable 0 @blend_mode 1 @blend_mode 2;
#P window linecount 1;
#P newex 614 57 60 9109513 prepend rate;
#P flonum 614 39 35 9 0 0 0 139 0 0 0 221 221 221 222 222 222 0 0 0;
#P message 782 65 26 9109513 stop;
#P message 749 65 26 9109513 read;
#P newex 679 57 45 9109513 metro 33;
#P toggle 679 32 15 0;
#P newex 679 97 91 9109513 jit.qt.movie 640 480;
#P message 425 127 29 9109513 reset;
#P newex 422 201 134 9109513 pak position 0. 0. 0.;
#P flonum 463 180 35 9 0 0 0 139 0 0 0 221 221 221 222 222 222 0 0 0;
#P flonum 504 180 35 9 0 0 0 139 0 0 0 221 221 221 222 222 222 0 0 0;
#P flonum 545 179 35 9 0 0 0 139 0 0 0 221 221 221 222 222 222 0 0 0;
#P newex 734 338 45 9109513 loadbang;
#P message 701 378 47 9109513 floating 1;
#P toggle 646 360 15 0;
#P message 646 378 45 9109513 sync $1;
#P toggle 573 360 15 0;
#P newex 534 359 35 9109513 sel 27;
#P message 573 378 68 9109513 fullscreen $1;
#P newex 491 402 174 9109513 jit.window motekScreen @depthbuffer 1;
#P newex 489 359 40 9109513 key;
#P user jit.fpsgui 80 130 60 9109513 0;
#P number 103 46 35 9 0 0 0 139 0 0 0 221 221 221 222 222 222 0 0 0;
#P toggle 58 46 15 0;
#P newex 58 72 55 9109513 qmetro 20;
#P newex 58 99 55 9109513 t b b erase;
#P newex 58 166 112 9109513 jit.gl.render motekScreen;
#P newex 425 150 113 9109513 jit.gl.handle motekScreen;
#P window linecount 2;
#P newex 334 228 220 9109513 jit.gl.videoplane motekScreen @scale 3.333 0.8 1. @depth_enable 0 @blend_mode 1 @blend_mode 2;
#P window linecount 1;
#P newex 269 58 60 9109513 prepend rate;
#P flonum 269 40 35 9 0 0 0 139 0 0 0 221 221 221 222 222 222 0 0 0;
#P message 437 66 26 9109513 stop;
#P message 404 66 26 9109513 read;
#P newex 334 58 45 9109513 metro 33;
#P toggle 334 33 15 0;
#P newex 334 98 91 9109513 jit.qt.movie 640 480;
#P connect 38 0 41 3;
#P connect 39 0 41 2;
#P connect 40 0 41 1;
#P connect 42 0 37 0;
#P fasten 23 0 22 0 739 368 706 368;
#P connect 41 0 36 0;
#P fasten 37 0 36 0 775 189 684 189;
#P connect 29 0 36 0;
#P fasten 33 0 29 0 787 88 684 88;
#P fasten 32 0 29 0 684 82;
#P connect 35 0 29 0;
#P connect 31 0 29 0;
#P connect 30 0 31 0;
#P connect 21 0 20 0;
#P connect 34 0 35 0;
#P connect 19 0 17 0;
#P fasten 18 0 19 0 539 377 570 377 570 357 578 357;
#P connect 24 0 27 3;
#P fasten 15 0 18 0 494 378 531 378 531 357 539 357;
#P connect 25 0 27 2;
#P fasten 22 0 16 0 706 397 496 397;
#P fasten 17 0 16 0 578 397 496 397;
#P fasten 20 0 16 0 651 397 496 397;
#P connect 26 0 27 1;
#P connect 28 0 8 0;
#P connect 0 0 7 0;
#P fasten 8 0 7 0 430 190 339 190;
#P connect 27 0 7 0;
#P connect 2 0 0 0;
#P connect 6 0 0 0;
#P fasten 3 0 0 0 339 83;
#P fasten 4 0 0 0 442 89 339 89;
#P connect 1 0 2 0;
#P connect 5 0 6 0;
#P connect 13 0 11 1;
#P connect 10 1 14 0;
#P fasten 10 2 9 0 107 125 63 125;
#P connect 10 0 9 0;
#P connect 11 0 10 0;
#P connect 12 0 11 0;
#P window clipboard copycount 43;


September 9, 2006 | 9:00 pm

Hi David,
There were a few problems with your patch. First You had 2 blend_mode
attributes and each ojnly had 1 value. @blend_mode takes 2 values.
Also, you need to enable blending with @blend_enable 1. Here are the
blend_mode from the docs:

The source and destination planes associated with the blend mode
(default = 6 7) Blend modes are specified in the form src_blend_mode
dst_blend_mode. The supported modes are:
0 = zero
1 = one
2 = destination color
3 = source color
4 = one minus destination color
5 = one minus source color
6 = source alpha
7 = one minus source alpha
8 = destination alpha
9 = one minus destination alpha
10 = source alpha saturate

For info on exactly what there mean, look for the OpenGL Red Book.
You can view it as HTML online or download a PDF of it.

Here’s a modified patch:

#P window setfont "Sans Serif" 9.;
#P window linecount 1;
#P message 888 175 29 196617 reset;
#P newex 885 249 134 196617 pak position 0. 0. 0.;
#P flonum 926 228 35 9 0 0 0 3 0 0 0 221 221 221 222 222 222 0 0 0;
#P flonum 967 228 35 9 0 0 0 3 0 0 0 221 221 221 222 222 222 0 0 0;
#P flonum 1008 227 35 9 0 0 0 3 0 0 0 221 221 221 222 222 222 0 0 0;
#P newex 888 198 132 196617 jit.gl.handle motekScreen;
#P window linecount 2;
#P newex 797 276 243 196617 jit.gl.videoplane motekScreen @scale 3.333
0.8 1. @depth_enable 0 @blend_enable 1 @blend_mode 1 2;
#P window linecount 1;
#P newex 732 106 74 196617 prepend rate;
#P flonum 732 88 35 9 0 0 0 3 0 0 0 221 221 221 222 222 222 0 0 0;
#P message 900 114 26 196617 stop;
#P message 895 69 83 196617 read dishes.mov;
#P newex 797 106 59 196617 metro 33;
#P toggle 797 81 15 0;
#P newex 797 146 110 196617 jit.qt.movie 640 480;
#P message 485 187 29 196617 reset;
#P newex 482 261 134 196617 pak position 0. 0. 0.;
#P flonum 523 240 35 9 0 0 0 3 0 0 0 221 221 221 222 222 222 0 0 0;
#P flonum 564 240 35 9 0 0 0 3 0 0 0 221 221 221 222 222 222 0 0 0;
#P flonum 605 239 35 9 0 0 0 3 0 0 0 221 221 221 222 222 222 0 0 0;
#P newex 794 398 45 196617 loadbang;
#P message 761 438 61 196617 floating 1;
#P toggle 706 420 15 0;
#P message 706 438 45 196617 sync $1;
#P toggle 633 420 15 0;
#P newex 594 419 35 196617 sel 27;
#P message 633 438 68 196617 fullscreen $1;
#P newex 551 462 198 196617 jit.window motekScreen @depthbuffer 1;
#P newex 549 419 40 196617 key;
#P user jit.fpsgui 140 190 60 196617 0;
#P number 163 106 35 9 0 0 0 3 0 0 0 221 221 221 222 222 222 0 0 0;
#P toggle 118 106 15 0;
#P newex 118 132 55 196617 qmetro 20;
#P newex 118 159 55 196617 t b b erase;
#P newex 118 226 131 196617 jit.gl.render motekScreen;
#P newex 485 210 132 196617 jit.gl.handle motekScreen;
#P window linecount 2;
#P newex 394 288 243 196617 jit.gl.videoplane motekScreen @scale 3.333
0.8 1. @depth_enable 0 @blend_enable 1 @blend_mode 1 2;
#P window linecount 1;
#P newex 282 129 74 196617 prepend rate;
#P flonum 282 111 35 9 0 0 0 3 0 0 0 221 221 221 222 222 222 0 0 0;
#P message 497 126 26 196617 stop;
#P message 480 85 80 196617 read dozer.mov;
#P newex 394 118 59 196617 metro 33;
#P toggle 394 93 15 0;
#P newex 394 158 110 196617 jit.qt.movie 640 480;
#P connect 29 0 36 0;
#P fasten 37 0 36 0 893 238 802 238;
#P connect 41 0 36 0;
#P connect 27 0 7 0;
#P fasten 8 0 7 0 490 250 399 250;
#P connect 0 0 7 0;
#P connect 12 0 11 0;
#P connect 11 0 10 0;
#P connect 10 0 9 0;
#P fasten 10 2 9 0 167 185 123 185;
#P connect 10 1 14 0;
#P connect 13 0 11 1;
#P connect 5 0 6 0;
#P connect 1 0 2 0;
#P fasten 4 0 0 0 502 149 399 149;
#P fasten 3 0 0 0 399 143;
#P connect 6 0 0 0;
#P connect 2 0 0 0;
#P connect 28 0 8 0;
#P connect 26 0 27 1;
#P fasten 20 0 16 0 711 457 556 457;
#P fasten 17 0 16 0 638 457 556 457;
#P fasten 22 0 16 0 766 457 556 457;
#P connect 25 0 27 2;
#P fasten 15 0 18 0 554 438 591 438 591 417 599 417;
#P connect 24 0 27 3;
#P fasten 18 0 19 0 599 437 630 437 630 417 638 417;
#P connect 19 0 17 0;
#P connect 34 0 35 0;
#P connect 21 0 20 0;
#P connect 30 0 31 0;
#P connect 31 0 29 0;
#P connect 35 0 29 0;
#P fasten 32 0 29 0 802 131;
#P fasten 33 0 29 0 905 137 802 137;
#P fasten 23 0 22 0 799 428 766 428;
#P connect 42 0 37 0;
#P connect 40 0 41 1;
#P connect 39 0 41 2;
#P connect 38 0 41 3;
#P window clipboard copycount 43;

wes


September 9, 2006 | 9:15 pm

Thanks! pretty dumb mistakes indeed… but still, i’m a bit puzzled about how i can adapt the ‘channel/plane’ used for blending so that i can implement some kind of crossfade between the two movies by manipulating the ‘channel/plane’ in question.

Anyhoo, I downloaded the red book and i’ll start reading straight away… cuz i really need to get this -probably basic functionality – to work…
(actually it was sitting here on my harddisk for quite some time, but just didnt get around to it…)


September 9, 2006 | 9:21 pm

On 9/9/06, david wrote:
>
> Thanks! pretty dumb mistakes indeed… but still, i’m a bit puzzled about how i can adapt the ‘channel/plane’ used for blending so that i can implement some kind of crossfade between the two movies by manipulating the ‘channel/plane’ in question.
>

For alpha blending, the alpha based blend modes will probably be your best bet.

wes


September 11, 2006 | 1:03 pm

I’ve had to work in the same situation, with an older graphics card.
However, I have 4 separate images and 4 separate alpha channels, all
of which eventually blend together seamlessly. I use jit.op. Some
operators are faster than others, so it works. Maybe I was in a hurry,
but I did not see a way to handle multi-step compositing with GL just
using blend and alpha settings. How would I pipe the results of one
composite to another? By capturing the resulting texture map?

Here is the tree of operations, with "x" representing a composite
(though in actual practice I use both logical and arithmetic
operations based on what I know of the geometry of the images and
alphas):

image1 x alpha1 image2 x alpha2 image3 x alpha3
image4 x alpha4
composite11 x composite 22 composite33 x composite44
composite1122 x composite3344
final image

Could this be done with older GPUs using blend and alpha–and be done
faster than with jit.op on the CPU?

– Paul

On 9/9/06, Wesley Smith wrote:
> On 9/9/06, david wrote:
> >
> > Thanks! pretty dumb mistakes indeed… but still, i’m a bit puzzled about how i can adapt the ‘channel/plane’ used for blending so that i can implement some kind of crossfade between the two movies by manipulating the ‘channel/plane’ in question.
> >
>
> For alpha blending, the alpha based blend modes will probably be your best bet.
>
> wes
>


—– |(*,+,#,=)(#,=,*,+)(=,#,+,*)(+,*,=,#)| —–


September 12, 2006 | 2:58 pm

Hi Paul,

I’m afraid i’m not really getting the tree of operations you’re trying to depict in your post, but below i’ve pasted a patch wich shows the technique i’m using now.

This uses the alpha blend modes of the videoplanes ( 6 and 7 ), while i fill in the alphaplane of my movie myself. if you constructed any alphaplanes in your previous patch u could easily fill them in into the patch below. My guess is this could result in better performance since the if you’re really just doing alpha blending with jit.op…

One thing i didnt get out yet is the drawing order of the planes. Sometimes the rightmost plane is on top, sometimes the leftmost is. I guess i can control this myself by adding the @automatic 0 attribute to the videoplane and bang them myself whenever i want them to be drawn.

anyhoo, u’ll see what u can do.

btw, if anyone has a straightforward example of drawing orders, that would be nice too :)

greetz
david

#P button 1308 240 15 0;
#P window setfont "Sans Serif" 9.;
#P window linecount 1;
#P newex 1385 168 45 9109513 loadbang;
#P message 1385 192 23 9109513 255;
#P newex 1308 267 109 9109513 jit.matrix 1 char 320 240;
#P number 1385 216 35 9 0 0 0 139 0 0 0 221 221 221 222 222 222 0 0 0;
#P newex 1336 238 59 9109513 pak setall 0;
#P newex 1215 299 53 9109513 jit.pack;
#P newex 1215 218 66 9109513 jit.unpack;
#P message 1280 85 26 9109513 read;
#P window linecount 2;
#P newex 1215 342 232 9109513 jit.gl.videoplane screen @scale 1.333 1. 1. @depth_enable 0 @blend_enable 1 @blend_mode 6 7;
#P window linecount 1;
#P message 1312 84 26 9109513 stop;
#P newex 1215 84 59 9109513 metro 33;
#P toggle 1215 59 15 0;
#P newex 1215 124 55 9109513 jit.qt.movie;
#P comment 1431 217 122 9109513 < < Change alphamask here;
#P button 914 235 15 0;
#P newex 991 163 45 9109513 loadbang;
#P message 991 187 23 9109513 255;
#P newex 914 262 109 9109513 jit.matrix 1 char 320 240;
#P number 991 211 35 9 0 0 0 139 0 0 0 221 221 221 222 222 222 0 0 0;
#P newex 942 233 59 9109513 pak setall 0;
#P newex 821 294 53 9109513 jit.pack;
#P newex 821 213 66 9109513 jit.unpack;
#P message 886 80 26 9109513 read;
#P window linecount 2;
#P newex 821 337 232 9109513 jit.gl.videoplane screen @scale 1.333 1. 1. @depth_enable 0 @blend_enable 1 @blend_mode 6 7;
#P window linecount 1;
#P message 918 79 26 9109513 stop;
#P newex 821 79 59 9109513 metro 33;
#P toggle 821 54 15 0;
#P newex 821 119 55 9109513 jit.qt.movie;
#P comment 1037 212 122 9109513 < < Change alphamask here;
#P message 1274 62 70 9109513 read bball.mov;
#P message 888 54 78 9109513 read dishes.mov;
#P button 478 246 15 0;
#P newex 555 174 45 9109513 loadbang;
#P message 555 198 23 9109513 255;
#P newex 478 273 109 9109513 jit.matrix 1 char 320 240;
#P number 555 224 35 9 0 0 0 139 0 0 0 221 221 221 222 222 222 0 0 0;
#P newex 506 244 59 9109513 pak setall 0;
#P newex 385 305 53 9109513 jit.pack;
#P newex 385 224 66 9109513 jit.unpack;
#P message 450 91 26 9109513 read;
#P newex 641 450 45 9109513 loadbang;
#P message 608 490 61 9109513 floating 1;
#P toggle 553 472 15 0;
#P message 553 490 45 9109513 sync $1;
#P toggle 480 472 15 0;
#P newex 441 471 35 9109513 sel 27;
#P message 480 490 68 9109513 fullscreen $1;
#P newex 398 514 147 9109513 jit.window screen @depthbuffer 1;
#P newex 396 471 40 9109513 key;
#P user jit.fpsgui 168 146 60 9109513 0;
#P number 191 62 35 9 0 0 0 139 0 0 0 221 221 221 222 222 222 0 0 0;
#P toggle 146 62 15 0;
#P newex 146 88 55 9109513 qmetro 20;
#P newex 146 115 55 9109513 t b b erase;
#P newex 146 182 85 9109513 jit.gl.render screen;
#P window linecount 2;
#P newex 385 348 232 9109513 jit.gl.videoplane screen @scale 1.333 1. 1. @depth_enable 0 @blend_enable 1 @blend_mode 6 7;
#P window linecount 1;
#P message 482 90 26 9109513 stop;
#P message 451 66 80 9109513 read dozer.mov;
#P newex 385 90 59 9109513 metro 33;
#P toggle 385 65 15 0;
#P newex 385 130 55 9109513 jit.qt.movie;
#P comment 601 223 122 9109513 < < Change alphamask here;
#P connect 10 0 9 0;
#P connect 9 0 8 0;
#P fasten 8 2 7 0 195 141 151 141;
#P connect 8 0 7 0;
#P connect 8 1 12 0;
#P connect 11 0 9 1;
#P connect 2 0 3 0;
#P fasten 5 0 1 0 487 120 390 120;
#P fasten 22 0 1 0 455 120 390 120;
#P fasten 4 0 1 0 456 86 447 86 447 120 390 120;
#P connect 3 0 1 0;
#P connect 1 0 23 0;
#P fasten 27 0 24 0 483 296 390 296;
#P connect 24 0 6 0;
#P fasten 20 0 14 0 613 509 403 509;
#P fasten 15 0 14 0 485 509 403 509;
#P fasten 18 0 14 0 558 509 403 509;
#P connect 23 1 24 1;
#P connect 23 2 24 2;
#P connect 23 3 24 3;
#P fasten 13 0 16 0 401 490 438 490 438 469 446 469;
#P fasten 1 0 30 0 390 184 483 184;
#P fasten 25 0 27 0 511 267 483 267;
#P connect 30 0 27 0;
#P fasten 16 0 17 0 446 489 477 489 477 469 485 469;
#P connect 17 0 15 0;
#P connect 19 0 18 0;
#P connect 29 0 28 0;
#P connect 28 0 26 0;
#P connect 26 0 25 1;
#P fasten 21 0 20 0 646 480 613 480;
#P connect 35 0 36 0;
#P connect 36 0 34 0;
#P fasten 39 0 34 0 891 107 826 107;
#P fasten 37 0 34 0 923 108 826 108;
#P fasten 31 0 34 0 893 76 883 76 883 108 826 108;
#P connect 34 0 40 0;
#P fasten 44 0 41 0 919 285 826 285;
#P connect 41 0 38 0;
#P connect 40 1 41 1;
#P connect 40 2 41 2;
#P connect 40 3 41 3;
#P fasten 34 0 47 0 826 173 919 173;
#P connect 47 0 44 0;
#P fasten 42 0 44 0 947 256 919 256;
#P connect 46 0 45 0;
#P connect 45 0 43 0;
#P connect 43 0 42 1;
#P connect 50 0 51 0;
#P connect 51 0 49 0;
#P fasten 54 0 49 0 1285 114 1220 114;
#P fasten 52 0 49 0 1317 114 1220 114;
#P fasten 32 0 49 0 1279 114 1220 114;
#P connect 49 0 55 0;
#P fasten 59 0 56 0 1313 290 1220 290;
#P connect 56 0 53 0;
#P connect 55 1 56 1;
#P connect 55 2 56 2;
#P connect 55 3 56 3;
#P fasten 49 0 62 0 1220 178 1313 178;
#P connect 62 0 59 0;
#P fasten 57 0 59 0 1341 261 1313 261;
#P connect 61 0 60 0;
#P connect 60 0 58 0;
#P connect 58 0 57 1;
#P window clipboard copycount 63;


Viewing 8 posts - 1 through 8 (of 8 total)