bryanteoh

I'm trying to write a shader that will basically extract the alpha channel from a video, invert it, then spit it out just the alpha layer so I can glue it onto other videos later on in the processing chain. This is my first shader, so if this code looks hopelessly stupid you can tell me to go back to the manuals, but I'm hoping it's something simple.

    samples the alpha channel, inverts it, and spits out single plane(??)

    language name="glsl" version="1.0">

I'm trying to write a shader that will basically extract the alpha channel from a video, invert it, then spit it out just the alpha layer so I can glue it onto other videos later on in the processing chain.  This is my first shader, so if this code looks hopelessly stupid you can tell me to go back to the manuals, but I'm hoping it's something simple.  

	samples the alpha channel, inverts it, and spits out single plane(??)
	language name="glsl" version="1.0">

first-shader-first-troubles

your code looks ok shader-vise, that's good:)

should probably be: 1.0 - at.a (if incoming values are

thanks nesa. The values are in fact less than or equal to 1.0, but I'm trying to invert the values of the black and white matrix. To do this I'm trying to subtract 1 to get a value between -1.0 and 0.0, and then take the absolute value before spitting it out to make it positive again. Is this illegal in glsl? If so, how else could I go about this?

thanks nesa.  The values are in fact less than or equal to 1.0, but I'm trying to invert the values of the black and white matrix.  To do this I'm trying to subtract 1 to get a value between -1.0 and 0.0, and then take the absolute value before spitting it out to make it positive again.  Is this illegal in glsl?  If so, how else could I go about this?


ok, actually it turns out the negative number wasn't an issue. I had forgotten to enclose the "language name" tag in an opening '

However, now there is another problem. When I try to run the shader, I get the following error:

ERROR: 0:8: 'assign' : cannot convert from 'float' to 'FragColor 4-component vector of float'

jit.gl.shader: GLSL program failed to compile.

This seems to imply that I can't set the gl_FragColor to a float. I only want/need to spit out the alpha channel. The shader does work if I express it if I write it as:

vec4 at = texture2DRect(tex0, texcoord0);

but then to use the alpha channel for another video later on in the slab chain, I'd have to make a shader that would just extract the layer again anyway which seems redundant to me. Clearly I need to work through the orange book a bit more, but if someone could tip me off on how to output just the single channel it would help me a lot with this current patch I'm working on.

ok, actually it turns out the negative number wasn't an issue.  I had forgotten to enclose the "language name" tag in an opening '

However, now there is another problem.  When I try to run the shader, I get the following error:

-- START GLSL INFO LOG: fp --
ERROR: 0:8: 'assign' :  cannot convert from 'float' to 'FragColor 4-component vector of float'
-- END GLSL INFO LOG: fp --
jit.gl.shader: GLSL program failed to compile.

This seems to imply that I can't set the gl_FragColor to a float.  I only want/need to spit out the alpha channel.  The shader does work if I express it if I write it as:

void main()
{
vec4 at = texture2DRect(tex0, texcoord0);
vec4 pl = vec4(at.a - 1.0);
gl_FragColor = abs(pl);
}

but then to use the alpha channel for another video later on in the slab chain, I'd have to make a shader that would just extract the layer again anyway which seems redundant to me.  Clearly I need to work through the orange book a bit more, but if someone could tip me off on how to output just the single channel it would help me a lot with this current patch I'm working on.

you cannot have output a single "planes" from shaders. gl_FragColor is a

On Thu, May 15, 2008 at 4:27 PM, bryanteoh wrote:

> ok, actually it turns out the negative number wasn't an issue. I had

> forgotten to enclose the "language name" tag in an opening '

> However, now there is another problem. When I try to run the shader, I get

> ERROR: 0:8: 'assign' : cannot convert from 'float' to 'FragColor

> jit.gl.shader: GLSL program failed to compile.

> This seems to imply that I can't set the gl_FragColor to a float. I only

> want/need to spit out the alpha channel. The shader does work if I express

> vec4 at = texture2DRect(tex0, texcoord0);

> but then to use the alpha channel for another video later on in the slab

> chain, I'd have to make a shader that would just extract the layer again

> anyway which seems redundant to me. Clearly I need to work through the

> orange book a bit more, but if someone could tip me off on how to output

> just the single channel it would help me a lot with this current patch I'm

you cannot have output a single "planes" from shaders.  gl_FragColor is a
vec4 type.

On Thu, May 15, 2008 at 4:27 PM, bryanteoh  wrote:

>
> ok, actually it turns out the negative number wasn't an issue.  I had
> forgotten to enclose the "language name" tag in an opening '
>
> However, now there is another problem.  When I try to run the shader, I get
> the following error:
>
> -- START GLSL INFO LOG: fp --
> ERROR: 0:8: 'assign' :  cannot convert from 'float' to 'FragColor
> 4-component vector of float'
> -- END GLSL INFO LOG: fp --
> jit.gl.shader: GLSL program failed to compile.
>
> This seems to imply that I can't set the gl_FragColor to a float.  I only
> want/need to spit out the alpha channel.  The shader does work if I express
> it if I write it as:
>
> void main()
> {
> vec4 at = texture2DRect(tex0, texcoord0);
> vec4 pl = vec4(at.a - 1.0);
> gl_FragColor = abs(pl);
> }
>
> but then to use the alpha channel for another video later on in the slab
> chain, I'd have to make a shader that would just extract the layer again
> anyway which seems redundant to me.  Clearly I need to work through the
> orange book a bit more, but if someone could tip me off on how to output
> just the single channel it would help me a lot with this current patch I'm
> working on.
>

thanks yair! A single float is no longer necessary anyway. It recently dawned on me that it was kind of a dumb idea to pass out a single value only to use it in another shader later. I'm currently working on one shader that will do all the necessary processing so a final vec4 output will be fine.

I'd like to thank both of you for such speedy responses.

thanks yair!  A single float is no longer necessary anyway.  It recently dawned on me that it was kind of a dumb idea to pass out a single value only to use it in another shader later.  I'm currently working on one shader that will do all the necessary processing so a final vec4 output will be fine.  

I'd like to thank both of you for such speedy responses.


> ok, actually it turns out the negative number wasn't an issue. I 

> had forgotten to enclose the "language name" tag in an opening '

btw, did the jit.gl.shader complian when you tried to read that file?

>
> ok, actually it turns out the negative number wasn't an issue.  I  
> had forgotten to enclose the "language name" tag in an opening '
> DOH!
>

GLSL doesn't do well with implicit type casting. You have to always be explicit about moving from one data-type to another. In this case, gl_FragColor is expecting a vec4. If you want to pass the alpha channel and set all other channels to 0., you could do the following:

But it sounds like you are looking to add this alpha channel to another texture, which would be better accomplished using a shader similar to cc.alphaglue.jxs. You should be able to alter that shader to your needs fairly easily.

GLSL doesn't do well with implicit type casting.  You have to always be explicit about moving from one data-type to another.  In this case, gl_FragColor is expecting a vec4.  If you want to pass the alpha channel and set all other channels to 0., you could do the following:
gl_FragColor = vec4(pl,0.,0.,0.);

But it sounds like you are looking to add this alpha channel to another texture, which would be better accomplished using a shader similar to cc.alphaglue.jxs.  You should be able to alter that shader to your needs fairly easily.

Thanks Andrew. I was able to successfully alter the alphaglue shader. In an attempt to rip you off more completely, I attempted to add a lumagate to the shader as well so I could specify values to use. The material I'll be using for the alpha channel in this patch is comprised of several shades of white. As far as I can tell the code is ok but I got the following error:

jit_xml_document: error reading file at byte offset 2392 / mismatched tag

jit.gl.shader: error reading shader xml file...

I first assumed that I didn't close one of my tags correctly, so I picked through it several times to make sure that wasn't the issue. I apologize for posting so much on this topic but if anyone has any ideas of what I did wrong, please let me know.

    alphaglue, lumagate, and their forbidden love

        luminance low hi thresholds

        select plane to use as alpha

        use luminence of second texture

        fade amount for threshold

        Luminance coefficients (RGBA)

Thanks Andrew.  I was able to successfully alter the alphaglue shader.  In an attempt to rip you off more completely, I attempted to add a lumagate to the shader as well so I could specify values to use.  The material I'll be using for the alpha channel in this patch is comprised of several shades of white.  As far as I can tell the code is ok but I got the following error:

jit_xml_document: error reading file at byte offset 2392 / mismatched tag
line (80): jittershader>
jit.gl.shader: error reading shader xml file...

I first assumed that I didn't close one of my tags correctly, so I picked through it several times to make sure that wasn't the issue.  I apologize for posting so much on this topic but if anyone has any ideas of what I did wrong, please let me know.

	alphaglue, lumagate, and their forbidden love
		luminance low hi thresholds
		select plane to use as alpha
		use luminence of second texture
		brightness threshold
		fade amount for threshold
		Luminance coefficients (RGBA)
	

I'm installing a video server system that will run on a number of 

macmini in network using jitter of course.

I'm trying to establish the best video compression setting in order to 

load off the most from the cpu and macmini videocard that is quite 

It will be only one video for each machine that will playback direct 

to the videoprojector in fullscreen (I'll try to go 1024 or 800).

Any experience and suggestions will be very valuable.

I'm installing a video server system that will run on a number of  
macmini in network using jitter of course.

I'm trying to establish the best video compression setting in order to  
load off the most from the cpu and macmini videocard that is quite  
cheap.

It will be only one video for each machine that will playback direct  
to the videoprojector in fullscreen (I'll try to go 1024 or 800).

The "plane" param wasn't closed. Also, you aren't using a vec2 for "fade", so you can't write "fade.x"/"fade.y". Fixing these things will allow your shader to compile. I also notice that you are calculating luminance twice (luminance, lumi), and that you aren't using the "plane" variable at all.

I don't understand the purpose of this line:

at = vec4(at*amask);//map video onto mask

Since you are trying to create an alpha channel, why are you multiplying the RGBA values by the "amask"? The vec4 constructor in the final line is adding the alpha mask to the color vector already. The multiplication is just going to add weird dark halos around your mask and make your image darker. Of course, maybe that's what you're looking for...

The "plane" param wasn't closed.   Also, you aren't using a vec2 for "fade", so you can't write "fade.x"/"fade.y".  Fixing these things will allow your shader to compile.  I also notice that you are calculating luminance twice (luminance, lumi), and that you aren't using the "plane" variable at all.  

Since you are trying to create an alpha channel, why are you multiplying the RGBA values by the "amask"?  The vec4 constructor in the final line is adding the alpha mask to the color vector already.  The multiplication is just going to add weird dark halos around your mask and make your image darker.  Of course, maybe that's what you're looking for...

first shader, first troubles