Forums > Jitter

first shader, first troubles

May 15, 2008 | 11:39 am

I’m trying to write a shader that will basically extract the alpha channel from a video, invert it, then spit it out just the alpha layer so I can glue it onto other videos later on in the processing chain. This is my first shader, so if this code looks hopelessly stupid you can tell me to go back to the manuals, but I’m hoping it’s something simple.



samples the alpha channel, inverts it, and spits out single plane(??)

language name="glsl" version="1.0">

< ![CDATA[

uniform sampler2DRect tex0;
varying vec2 texcoord0;

void main()
{
vec4 at = texture2DRect(tex0, texcoord0);
float pl = at.a – 1.0;
gl_FragColor = abs(pl);
}
]]>


May 15, 2008 | 12:51 pm

Hi,

your code looks ok shader-vise, that’s good:)

but the math:

> float pl = at.a – 1.0;

should probably be: 1.0 – at.a (if incoming values are < = 1.0)

hth,
nesa


May 15, 2008 | 1:01 pm

thanks nesa. The values are in fact less than or equal to 1.0, but I’m trying to invert the values of the black and white matrix. To do this I’m trying to subtract 1 to get a value between -1.0 and 0.0, and then take the absolute value before spitting it out to make it positive again. Is this illegal in glsl? If so, how else could I go about this?


May 15, 2008 | 1:27 pm

ok, actually it turns out the negative number wasn’t an issue. I had forgotten to enclose the "language name" tag in an opening ‘< '. DOH!

However, now there is another problem. When I try to run the shader, I get the following error:

– START GLSL INFO LOG: fp –
ERROR: 0:8: ‘assign’ : cannot convert from ‘float’ to ‘FragColor 4-component vector of float’
– END GLSL INFO LOG: fp –
jit.gl.shader: GLSL program failed to compile.

This seems to imply that I can’t set the gl_FragColor to a float. I only want/need to spit out the alpha channel. The shader does work if I express it if I write it as:

void main()
{
vec4 at = texture2DRect(tex0, texcoord0);
vec4 pl = vec4(at.a – 1.0);
gl_FragColor = abs(pl);
}

but then to use the alpha channel for another video later on in the slab chain, I’d have to make a shader that would just extract the layer again anyway which seems redundant to me. Clearly I need to work through the orange book a bit more, but if someone could tip me off on how to output just the single channel it would help me a lot with this current patch I’m working on.


May 15, 2008 | 2:08 pm

you cannot have output a single "planes" from shaders. gl_FragColor is a
vec4 type.

On Thu, May 15, 2008 at 4:27 PM, bryanteoh wrote:

>
> ok, actually it turns out the negative number wasn’t an issue. I had
> forgotten to enclose the "language name" tag in an opening ‘< '. DOH!
>
> However, now there is another problem. When I try to run the shader, I get
> the following error:
>
> — START GLSL INFO LOG: fp –
> ERROR: 0:8: ‘assign’ : cannot convert from ‘float’ to ‘FragColor
> 4-component vector of float’
> — END GLSL INFO LOG: fp –
> jit.gl.shader: GLSL program failed to compile.
>
> This seems to imply that I can’t set the gl_FragColor to a float. I only
> want/need to spit out the alpha channel. The shader does work if I express
> it if I write it as:
>
> void main()
> {
> vec4 at = texture2DRect(tex0, texcoord0);
> vec4 pl = vec4(at.a – 1.0);
> gl_FragColor = abs(pl);
> }
>
> but then to use the alpha channel for another video later on in the slab
> chain, I’d have to make a shader that would just extract the layer again
> anyway which seems redundant to me. Clearly I need to work through the
> orange book a bit more, but if someone could tip me off on how to output
> just the single channel it would help me a lot with this current patch I’m
> working on.
>


May 15, 2008 | 2:39 pm

thanks yair! A single float is no longer necessary anyway. It recently dawned on me that it was kind of a dumb idea to pass out a single value only to use it in another shader later. I’m currently working on one shader that will do all the necessary processing so a final vec4 output will be fine.

I’d like to thank both of you for such speedy responses.


May 15, 2008 | 3:11 pm

>
> ok, actually it turns out the negative number wasn’t an issue. I
> had forgotten to enclose the "language name" tag in an opening ‘< '.
> DOH!
>

btw, did the jit.gl.shader complian when you tried to read that file?

> vec4 pl = vec4(at.a – 1.0);

ah, yes, sorry for that oversight.


May 15, 2008 | 3:47 pm

GLSL doesn’t do well with implicit type casting. You have to always be explicit about moving from one data-type to another. In this case, gl_FragColor is expecting a vec4. If you want to pass the alpha channel and set all other channels to 0., you could do the following:
gl_FragColor = vec4(pl,0.,0.,0.);

But it sounds like you are looking to add this alpha channel to another texture, which would be better accomplished using a shader similar to cc.alphaglue.jxs. You should be able to alter that shader to your needs fairly easily.

AB


May 16, 2008 | 3:31 am

Thanks Andrew. I was able to successfully alter the alphaglue shader. In an attempt to rip you off more completely, I attempted to add a lumagate to the shader as well so I could specify values to use. The material I’ll be using for the alpha channel in this patch is comprised of several shades of white. As far as I can tell the code is ok but I got the following error:

jit_xml_document: error reading file at byte offset 2392 / mismatched tag
line (80): jittershader>
jit.gl.shader: error reading shader xml file…

I first assumed that I didn’t close one of my tags correctly, so I picked through it several times to make sure that wasn’t the issue. I apologize for posting so much on this topic but if anyone has any ideas of what I did wrong, please let me know.



alphaglue, lumagate, and their forbidden love

luminance low hi thresholds

select plane to use as alpha

use luminence of second texture

brightness threshold

fade amount for threshold

Luminance coefficients (RGBA)









< ![CDATA[

// Andrew Benson – andrewb@cycling74.com
//Copyright 2006 – Cycling ’74

//fragment shader for applying alpha channel from second texture

//setup for 2 texture
varying vec2 texcoord0;
varying vec2 texcoord1;
uniform sampler2DRect tex0;
uniform sampler2DRect tex1;

//wipe state
uniform vec2 lum;
uniform int plane;
uniform float fade;
uniform float lum2alpha;
uniform float thresh;
uniform vec4 lumcoeff;

void main()
{
vec4 at = texture2DRect(tex0, texcoord0);//texture for blend
vec4 bt = texture2DRect(tex1, texcoord1);//texture for alpha

//lumagate the intended alpha channel
float luminance = dot(bt,lumcoeff);
float clo = smoothstep(lum.x,lum.x+fade.x,luminance);
float chi = 1.-smoothstep(lum.y-fade.y,lum.y,luminance);
float amask = float (clo * chi);

// decide which plane to use
float lumi = dot(bt,lumcoeff);//calc the luminance
amask = mix(amask,lumi,lum2alpha);//pick plane or luma
amask = smoothstep(thresh,thresh+fade,amask);//apply threshold

at = vec4(at*amask);//map video onto mask

// output texture along with alpha channel
gl_FragColor = vec4 (at.rgb,amask);
}
]]>


May 16, 2008 | 7:28 am

Hello from italy,

I’m installing a video server system that will run on a number of
macmini in network using jitter of course.

I’m trying to establish the best video compression setting in order to
load off the most from the cpu and macmini videocard that is quite
cheap.

It will be only one video for each machine that will playback direct
to the videoprojector in fullscreen (I’ll try to go 1024 or 800).

Any experience and suggestions will be very valuable.

Thanks in advance.

ciao ciao
riccardo mazza


May 16, 2008 | 2:49 pm

The "plane" param wasn’t closed. Also, you aren’t using a vec2 for "fade", so you can’t write "fade.x"/"fade.y". Fixing these things will allow your shader to compile. I also notice that you are calculating luminance twice (luminance, lumi), and that you aren’t using the "plane" variable at all.

I don’t understand the purpose of this line:

at = vec4(at*amask);//map video onto mask

Since you are trying to create an alpha channel, why are you multiplying the RGBA values by the "amask"? The vec4 constructor in the final line is adding the alpha mask to the color vector already. The multiplication is just going to add weird dark halos around your mask and make your image darker. Of course, maybe that’s what you’re looking for…

ab


Viewing 11 posts - 1 through 11 (of 11 total)