first shader, first troubles

May 15, 2008 at 11:39am

first shader, first troubles

I’m trying to write a shader that will basically extract the alpha channel from a video, invert it, then spit it out just the alpha layer so I can glue it onto other videos later on in the processing chain. This is my first shader, so if this code looks hopelessly stupid you can tell me to go back to the manuals, but I’m hoping it’s something simple.



samples the alpha channel, inverts it, and spits out single plane(??)

language name=”glsl” version=”1.0″>

< ![CDATA[

uniform sampler2DRect tex0;
varying vec2 texcoord0;

void main()
{
vec4 at = texture2DRect(tex0, texcoord0);
float pl = at.a – 1.0;
gl_FragColor = abs(pl);
}
]]>

#37797
May 15, 2008 at 12:51pm

Hi,

your code looks ok shader-vise, that’s good:)

but the math:

> float pl = at.a – 1.0;

should probably be: 1.0 – at.a (if incoming values are < = 1.0)

hth,
nesa

#130772
May 15, 2008 at 1:01pm

thanks nesa. The values are in fact less than or equal to 1.0, but I’m trying to invert the values of the black and white matrix. To do this I’m trying to subtract 1 to get a value between -1.0 and 0.0, and then take the absolute value before spitting it out to make it positive again. Is this illegal in glsl? If so, how else could I go about this?

#130773
May 15, 2008 at 1:27pm

ok, actually it turns out the negative number wasn’t an issue. I had forgotten to enclose the “language name” tag in an opening ‘< '. DOH!

However, now there is another problem. When I try to run the shader, I get the following error:

– START GLSL INFO LOG: fp –
ERROR: 0:8: ‘assign’ : cannot convert from ‘float’ to ‘FragColor 4-component vector of float’
– END GLSL INFO LOG: fp –
jit.gl.shader: GLSL program failed to compile.

This seems to imply that I can’t set the gl_FragColor to a float. I only want/need to spit out the alpha channel. The shader does work if I express it if I write it as:

void main()
{
vec4 at = texture2DRect(tex0, texcoord0);
vec4 pl = vec4(at.a – 1.0);
gl_FragColor = abs(pl);
}

but then to use the alpha channel for another video later on in the slab chain, I’d have to make a shader that would just extract the layer again anyway which seems redundant to me. Clearly I need to work through the orange book a bit more, but if someone could tip me off on how to output just the single channel it would help me a lot with this current patch I’m working on.

#130774
May 15, 2008 at 2:08pm

you cannot have output a single “planes” from shaders. gl_FragColor is a
vec4 type.

On Thu, May 15, 2008 at 4:27 PM, bryanteoh wrote:

>
> ok, actually it turns out the negative number wasn’t an issue. I had
> forgotten to enclose the “language name” tag in an opening ‘< '. DOH!
>
> However, now there is another problem. When I try to run the shader, I get
> the following error:
>
> — START GLSL INFO LOG: fp –
> ERROR: 0:8: ‘assign’ : cannot convert from ‘float’ to ‘FragColor
> 4-component vector of float’
> — END GLSL INFO LOG: fp –
> jit.gl.shader: GLSL program failed to compile.
>
> This seems to imply that I can’t set the gl_FragColor to a float. I only
> want/need to spit out the alpha channel. The shader does work if I express
> it if I write it as:
>
> void main()
> {
> vec4 at = texture2DRect(tex0, texcoord0);
> vec4 pl = vec4(at.a – 1.0);
> gl_FragColor = abs(pl);
> }
>
> but then to use the alpha channel for another video later on in the slab
> chain, I’d have to make a shader that would just extract the layer again
> anyway which seems redundant to me. Clearly I need to work through the
> orange book a bit more, but if someone could tip me off on how to output
> just the single channel it would help me a lot with this current patch I’m
> working on.
>

#130775
May 15, 2008 at 2:39pm

thanks yair! A single float is no longer necessary anyway. It recently dawned on me that it was kind of a dumb idea to pass out a single value only to use it in another shader later. I’m currently working on one shader that will do all the necessary processing so a final vec4 output will be fine.

I’d like to thank both of you for such speedy responses.

#130776
May 15, 2008 at 3:11pm

>
> ok, actually it turns out the negative number wasn’t an issue. I
> had forgotten to enclose the “language name” tag in an opening ‘< '.
> DOH!
>

btw, did the jit.gl.shader complian when you tried to read that file?

> vec4 pl = vec4(at.a – 1.0);

ah, yes, sorry for that oversight.

#130777
May 15, 2008 at 3:47pm

GLSL doesn’t do well with implicit type casting. You have to always be explicit about moving from one data-type to another. In this case, gl_FragColor is expecting a vec4. If you want to pass the alpha channel and set all other channels to 0., you could do the following:
gl_FragColor = vec4(pl,0.,0.,0.);

But it sounds like you are looking to add this alpha channel to another texture, which would be better accomplished using a shader similar to cc.alphaglue.jxs. You should be able to alter that shader to your needs fairly easily.

AB

#130778
May 16, 2008 at 3:31am

Thanks Andrew. I was able to successfully alter the alphaglue shader. In an attempt to rip you off more completely, I attempted to add a lumagate to the shader as well so I could specify values to use. The material I’ll be using for the alpha channel in this patch is comprised of several shades of white. As far as I can tell the code is ok but I got the following error:

jit_xml_document: error reading file at byte offset 2392 / mismatched tag
line (80): jittershader>
jit.gl.shader: error reading shader xml file…

I first assumed that I didn’t close one of my tags correctly, so I picked through it several times to make sure that wasn’t the issue. I apologize for posting so much on this topic but if anyone has any ideas of what I did wrong, please let me know.



alphaglue, lumagate, and their forbidden love

luminance low hi thresholds

select plane to use as alpha

use luminence of second texture

brightness threshold

fade amount for threshold

Luminance coefficients (RGBA)









< ![CDATA[

// Andrew Benson – andrewb@cycling74.com
//Copyright 2006 – Cycling ’74

//fragment shader for applying alpha channel from second texture

//setup for 2 texture
varying vec2 texcoord0;
varying vec2 texcoord1;
uniform sampler2DRect tex0;
uniform sampler2DRect tex1;

//wipe state
uniform vec2 lum;
uniform int plane;
uniform float fade;
uniform float lum2alpha;
uniform float thresh;
uniform vec4 lumcoeff;

void main()
{
vec4 at = texture2DRect(tex0, texcoord0);//texture for blend
vec4 bt = texture2DRect(tex1, texcoord1);//texture for alpha

//lumagate the intended alpha channel
float luminance = dot(bt,lumcoeff);
float clo = smoothstep(lum.x,lum.x+fade.x,luminance);
float chi = 1.-smoothstep(lum.y-fade.y,lum.y,luminance);
float amask = float (clo * chi);

// decide which plane to use
float lumi = dot(bt,lumcoeff);//calc the luminance
amask = mix(amask,lumi,lum2alpha);//pick plane or luma
amask = smoothstep(thresh,thresh+fade,amask);//apply threshold

at = vec4(at*amask);//map video onto mask

// output texture along with alpha channel
gl_FragColor = vec4 (at.rgb,amask);
}
]]>

#130779
May 16, 2008 at 7:28am

Hello from italy,

I’m installing a video server system that will run on a number of
macmini in network using jitter of course.

I’m trying to establish the best video compression setting in order to
load off the most from the cpu and macmini videocard that is quite
cheap.

It will be only one video for each machine that will playback direct
to the videoprojector in fullscreen (I’ll try to go 1024 or 800).

Any experience and suggestions will be very valuable.

Thanks in advance.

ciao ciao
riccardo mazza

#130780
May 16, 2008 at 2:49pm

The “plane” param wasn’t closed. Also, you aren’t using a vec2 for “fade”, so you can’t write “fade.x”/”fade.y”. Fixing these things will allow your shader to compile. I also notice that you are calculating luminance twice (luminance, lumi), and that you aren’t using the “plane” variable at all.

I don’t understand the purpose of this line:

at = vec4(at*amask);//map video onto mask

Since you are trying to create an alpha channel, why are you multiplying the RGBA values by the “amask”? The vec4 constructor in the final line is adding the alpha mask to the color vector already. The multiplication is just going to add weird dark halos around your mask and make your image darker. Of course, maybe that’s what you’re looking for…

ab

#130781

You must be logged in to reply to this topic.