CVOpenGLTextureRef to jit_gl_texture
hi.
i brew my own movieplayback external, now i would like to connect to jitter-world, is this possible?
so, what i have is a QTVisualContextRef, where i am fetching the texture-data into a CVOpenGLTextureRef... can i pass the information from the CVOpenGLTextureRef further into a jit_gl_texture object? or can i use a jit_gl_texture object directly instead of the CVOpenGLTextureRef-object? - would be the best i think...
is there anything i can read about jit_gl_texture? could not find anything... which datatype are used internally..etc
what i do in the draw method:
//getting the image form the movie, where x->mytexture is a CVOpenGLTextureRef and x->myref is the QTVisualContextRef:
QTVisualContextCopyImageForTime(x->myref, NULL, NULL, &x->mytexture);
thanks for help&advice...
io
Hi Ingo,
I believe the easiest strategy to start with would be to render your
input CVOpenGLTextureRef to a "capturing" jit_gl_texture object. I'll
try to find some time in the next few days to draw up a simple example
for you in C, but for the time being if you can take one of the
jit.gl.* SDK examples and establish how to render your texture inside
the current OpenGL context (probably means you need to figure out how
to make your QTVisualContextRef make use of the Jitter OpenGL
context), then you can use the standard "capture" attribute to render
it to a jit.gl.texture object rather than to the main context. This
might be a hassle, as we don't have tons of documentation on these
types of internals (and they're subject to change FWIW), but I'll try
to give you enough support for you to get your object working if I can.
-Joshua
On Dec 3, 2008, at 6:42 PM, ingo randolf wrote:
>
> hi.
>
> i brew my own movieplayback external, now i would like to connect to
> jitter-world, is this possible?
>
> so, what i have is a QTVisualContextRef, where i am fetching the
> texture-data into a CVOpenGLTextureRef... can i pass the information
> from the CVOpenGLTextureRef further into a jit_gl_texture object? or
> can i use a jit_gl_texture object directly instead of the
> CVOpenGLTextureRef-object? - would be the best i think...
>
> is there anything i can read about jit_gl_texture? could not find
> anything... which datatype are used internally..etc
>
>
> what i do in the draw method:
>
> //getting the image form the movie, where x->mytexture is a
> CVOpenGLTextureRef and x->myref is the QTVisualContextRef:
> QTVisualContextCopyImageForTime(x->myref, NULL, NULL, &x->mytexture);
>
> http://developer.apple.com/qa/qa2005/qa1443.html
>
> thanks for help&advice...
> io
Hi Joshua, thanks a lot!
well i make use of the jitter-openGl context allready, so the QTVisualContextRef is set to the context i get out of this piece of code:
//get the context, interpret it as an AGLContext
jit_ctx = jit_gl_get_context();
ctx = (AGLContext) jit_ctx->context;
//get the CGL context ou of it
aglGetCGLContext(ctx, &x->cglCtx);
//get the pixelformat out of it
fmt = (AGLPixelFormat) jit_ctx->pixelformat;
aglGetCGLPixelFormat(fmt, &x->cglFmt);
//create the QTOpenGLTexture, where x->myref is the QTVisualContextRef
err = QTOpenGLTextureCreat(NULL, x->cglCtx, x->cglFmt, NULL, &x->myref);
io
addition to the post before:
do you mean it would be possible to pass the jit_gl_textures-context to QTOpenGLTextureCreate (miss-spelled in the post before...) instead ot the jit_gl_context? then it would render directly into the texture... hmm...
io
On Dec 4, 2008, at 1:36 PM, ingo randolf wrote:
> do you mean it would be possible to pass the jit_gl_textures-context
> to QTOpenGLTextureCreate (miss-spelled in the post before...)
> instead ot the jit_gl_context? then it would render directly into
> the texture... hmm...
Actually, I would suggest you use that texture context to texture a
quad filling the entire viewport, and set the object's "capture"
attribute to be some jit.gl.texture in a patcher. This should render
the QT texture into a Jitter texture. Once you have that working, I
can help you work with the jit.gl.texture object from C. You'll want
to resize the jit.gl.texture object to be the same size as the QT movie.
Let me know if this is confusing, or if you have specific questions.
-Joshua
Hi Ingo,
I've stripped out the relevant functions from jit.gl.imageunit, and uploaded this excerpt as a C file here on the jitter-dev forum (if you're using email, then access via https://cycling74.com/forums/index.php?t=msg&th=36969 ).
Here's the summary. This jit.gl.imageunit excerpt, while not a complete example, demonstrates how to use texture inputs and outputs for a jit.gl.* object. For your purposes, you would just need to look at how the object makes use of x->output, but I include the other methods which are relevant for other developers who might want to do something similar with texture inputs.
- jit_gl_texture_new(): demonstrates how to allocate your texture objects from C, and set relevant texture attributes
- jit_gl_texture_destroy(): demonstrates how to free your texture objects from C
- jit_gl_imageunit_recalc(): demonstrates setting input list via OB3D "texture" attribute, setting output texture via OB3D "capture" attribute
- jit_gl_imageunit_jit_matrix(): demonstrates how to take an incoming matrix and copy it into a texture input, resizing dependencies if necessary
- jit_gl_imageunit_jit_gl_texture(): demonstrates how to take an incoming texture for use instead of an internal texture, resizing dependencies if necessary
- jit_gl_imageunit_dest_changed(): demonstrates flags to make use of for any necessary recreation within your draw method. there should be no need to dispatch anything to internal texture objects here as they will receive this message themselves
- jit_gl_imageunit_drawto(): demonstrates the need to dispatch drawto renderer changes to internal inputs and outputs
- jit_gl_imageunit_draw(): demonstrates some simple OpenGL state setup which we use for our image unit example. when this function is called, all of the input textures in the "texture" attribute will be bound for use by geometry, and the output texture having been set as the "capture" attribute will be set as the rendering context, so everything in this function will render to the texture (which will also be the current context returned by jit_gl_get_context(). for your example, you would probably just render a textured quad here that spans the view port, textured with your CVOpenGLTextureRef. You will probably want to have additional texture setup which needs to take place. Another useful thing that this function demonstrates is how to obtain the OpenGL texture ID for a texture with the *get only* "glid" attribute.
Okay, that should hopefully at least get you started. Please feel free to post specific questions as you encounter them with this project.
-Joshua
oh yeah!
thank you very much!!... this keeps me busy for a while.. :)
cheers
io
On Dec 9, 2008, at 3:01 PM, Joshua Kit Clayton wrote:
>
> - jit_gl_imageunit_draw(): demonstrates some simple OpenGL state
> setup which we use for our image unit example. when this function is
> called, all of the input textures in the "texture" attribute will be
> bound for use by geometry, and the output texture having been set as
> the "capture" attribute will be set as the rendering context, so
> everything in this function will render to the texture (which will
> also be the current context returned by jit_gl_get_context()...
One more thing I realize I should have mentioned. If you were using
the current context in your draw method to set up your
CVOpenGLTextureRef, there's a chance you may want to actually create
it in the dest_changed method, since that will have the main context
set when calling, rather than the capturing jit.gl.texture's context
set. With imageunits we want the jit.gl.texture's context to be our
output context, but for your purposes it might be different. Kinda
confusing, due to the way capture to texture works, but let me know if
you have any questions.
-Joshua
posting the source here inline for the new forums without attachments:
t_jit_gl_imageunit *jit_gl_imageunit_new(t_symbol * dest_name)
{
long i;
t_jit_gl_imageunit *x;
if (x=(t_jit_gl_imageunit *)jit_object_alloc(_jit_gl_imageunit_class)) {
x->adapt = TRUE;
x->inputcount = 2;
x->type = ps_auto;
x->dim[0] = 256;
x->dim[1] = 256;
x->dim[2] = 0;
x->dimscale[0] = 1.;
x->dimscale[1] = 1.;
x->activeinput = 0;
for(i = 0; i < JIT_GL_MAX_TEXTUREUNITS; i++)
{
x->input[i] = NULL;
x->useinternal[i] = FALSE;
x->internal[i] = jit_object_new(ps_jit_gl_texture,dest_name);
jit_attr_setsym(x->internal[i],_jit_sym_name,jit_symbol_unique());
jit_attr_setsym(x->internal[i],gensym("defaultimage"),gensym("black"));
}
x->colormode = ps_argb;
x->firsttime = 1;
x->verbose = 0;
x->output = jit_object_new(ps_jit_gl_texture,dest_name);
jit_attr_setsym(x->output,_jit_sym_name,jit_symbol_unique());
jit_attr_setlong(x->output, ps_flip, 0);
jit_attr_setsym(x->output,gensym("defaultimage"),gensym("black"));
x->file = gensym("Color Matrix");
// create and attach ob3d
jit_ob3d_new(x, dest_name);
x->update = TRUE;
x->jiupupdate = TRUE;
jit_gl_imageunit_recalc(x);
} else {
x = NULL;
}
return x;
}
void jit_gl_imageunit_destroy(t_jit_gl_imageunit *x)
{
long i;
// destroy inputs
for(i = 0; i < JIT_GL_MAX_TEXTUREUNITS; i++)
{
x->input[i] = NULL;
x->useinternal[i] = FALSE;
if (x->internal[i])
jit_object_free(x->internal[i]);
x->internal[i] = NULL;
}
// destroy outputs
if (x->output)
jit_object_free(x->output);
x->output = NULL;
}
void jit_gl_imageunit_recalc( t_jit_gl_imageunit *x )
{
long i;
t_symbol *name;
// get input texture names for external and internal textures
for(i = 0; i < x->inputcount; i++)
{
if (x->input[i] && !x->useinternal[i]) {
name = x->input[i];
//jit_object_post((t_object *)x,"external texture");
}
else if (x->useinternal[i])
name = jit_attr_getsym(x->internal[i], ps_name);
else
name = NULL;
x->textures[i] = name ? name : _jit_sym_nothing;
}
// add input textures to ob3d texture list
jit_attr_setsym_array(x, ps_texture, x->inputcount, x->textures);
// add output texture to ob3d capture list
jit_attr_setsym(x,ps_capture,jit_attr_getsym(x->output, ps_name));
// disable ob3d automatic mode
jit_attr_setlong(x,ps_automatic,0);
if (x->update)
x->jiupupdate = TRUE;
// reset update flag
x->update = FALSE;
}
t_jit_err jit_gl_imageunit_jit_matrix(t_jit_gl_imageunit *x, t_symbol *s, int argc, t_atom *argv)
{
t_jit_object *matrix;
t_jit_matrix_info info;
t_symbol *name = jit_atom_getsym(argv);
if (name)
{
matrix = (t_jit_object*)jit_object_findregistered(name);
if (!matrix)
{
post ("jit.gl.imageunit: couldn't get matrix object!");
return JIT_ERR_GENERIC;
}
if (x->activeinput >= 0 && x->activeinput < JIT_GL_MAX_TEXTUREUNITS)
{
// toggle use flag
x->useinternal[x->activeinput] = TRUE;
// get matrix info
jit_object_method(matrix, _jit_sym_getinfo, &info);
// adapt to leftmost input dim
if((x->activeinput == 0 || x->firsttime))
{
if (x->adapt)
{
long lastdim[2];
lastdim[0] = x->dim[0];
lastdim[1] = x->dim[1];
// use current info and resize output texture
if (x->colormode==ps_uyvy)
info.dim[0] *= JIT_GL_IMAGEUNIT_COLORMODE_SCALAR;
x->dim[0] = info.dim[0];
x->dim[1] = info.dim[1];
x->dim[0] = (long)(ABS((double)x->dim[0]*x->dimscale[0]) + 0.5);
x->dim[1] = (long)(ABS((double)x->dim[1]*x->dimscale[1]) + 0.5);
if (x->dim[0]!=lastdim[0]||x->dim[1]!=lastdim[1]) {
x->update = TRUE;
}
if (x->update) {
jit_attr_setlong_array(x->output, _jit_sym_dim, 2, x->dim);
}
}
}
// reset external referenced texture
x->input[x->activeinput] = NULL;
// submit matrix to active texture
jit_object_method(x->internal[x->activeinput], s, s, argc, argv);
x->firsttime = 0;
// recalc
jit_gl_imageunit_recalc(x);
}
}
return JIT_ERR_NONE;
}
t_jit_err jit_gl_imageunit_jit_gl_texture(t_jit_gl_imageunit *x, t_symbol *s, int argc, t_atom *argv)
{
t_jit_object *texture;
t_symbol *name = jit_atom_getsym(argv);
if (name)
{
texture = (t_jit_object*)jit_object_findregistered(name);
if (!texture)
{
post ("jit.gl.imageunit: couldn't get texture object!");
return JIT_ERR_GENERIC;
}
if (x->activeinput >= 0 && x->activeinput < JIT_GL_MAX_TEXTUREUNITS)
{
// cache input texture name
x->input[x->activeinput] = name;
// disable internal texture object
x->useinternal[x->activeinput] = FALSE;
// adapt to leftmost input dim
if(x->activeinput == 0)
{
if (x->adapt) {
jit_attr_getlong_array(texture,_jit_sym_dim,2,x->dim);
x->dim[0] = (long)(ABS((double)x->dim[0]*x->dimscale[0]) + 0.5);
x->dim[1] = (long)(ABS((double)x->dim[1]*x->dimscale[1]) + 0.5);
x->dim[2] = 0;
if (x->output) {
long wasdim[2] = {0,0};
jit_attr_getlong_array(x->output,_jit_sym_dim,2,wasdim);
if ((wasdim[0]!=x->dim[0])||(wasdim[1]!=x->dim[1])) {
jit_attr_setlong_array(x->output, _jit_sym_dim, 2, x->dim);
}
}
}
}
jit_gl_imageunit_recalc(x);
}
}
return JIT_ERR_NONE;
}
t_jit_err jit_gl_imageunit_dest_changed(t_jit_gl_imageunit *x)
{
x->update = TRUE;
x->sendcached = TRUE;
return JIT_ERR_NONE;
}
t_jit_err jit_gl_imageunit_drawto(t_jit_gl_imageunit *x, t_symbol *s, int argc, t_atom *argv)
{
long i;
if (x->output)
object_attr_setvalueof(x->output,s,argc,argv);
for (i=0;iinputcount;i++)
{
if (x->internal[i])
object_attr_setvalueof(x->internal[i],s,argc,argv);
}
jit_ob3d_dest_name_set((t_jit_object *)x, NULL, argc, argv);
return JIT_ERR_NONE;
}
t_jit_err jit_gl_imageunit_draw(t_jit_gl_imageunit *x)
{
t_jit_err err = JIT_ERR_NONE;
t_jit_gl_context ctx;
t_jit_object *texture;
GLuint texname;
long i,width,height,flip,rv;
t_atom a;
t_jiup_filter_description *d=NULL;
if (!x)
return JIT_ERR_INVALID_PTR;
if (x->update)
jit_gl_imageunit_recalc(x);
// double check our context dim
width = jit_attr_getlong(x->output,ps_width);
if (width!=x->dim[0]) {
x->dim[0] = width;
x->jiupupdate = 1;
}
height = jit_attr_getlong(x->output,ps_height);
if (height!=x->dim[1]) {
x->dim[1] = height;
x->jiupupdate = 1;
}
// CODE REMOVED ... IMAGE UNIT (RE) CREATION WOULD BE HERE
if (x->jiup) {
// CODE REMOVED ... ADDITIONAL IMAGE UNIT SETUP WOULD BE HERE
for (i=0;ijiup->description->imginputcount&&iinputcount;i++) {
if (x->textures[i]&&x->textures[i]!=_jit_sym_nothing) {
texture = (t_jit_object*)jit_object_findregistered(x->textures[i]);
if (!texture) {
texname = width = height = flip = 0;
} else {
// texname is out OpenGL texture ID
// this attribute is *only* gettable
texname = jit_attr_getlong(texture,ps_glid);
if (texname
texname = 0;
width = jit_attr_getlong(texture,ps_width);
height = jit_attr_getlong(texture,ps_height);
flip = jit_attr_getlong(texture,ps_flip);
}
jiup_input_set(x->jiup,i,texname,width,height,flip);
}
}
glPushAttrib(GL_ALL_ATTRIB_BITS);
// setup state
glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);
// need to set texture matrix transform to identity, else problems with Image Units
// for other applications, this may not be necessary, and the state setup might be different
glMatrixMode(GL_TEXTURE);
glPushMatrix();
glLoadIdentity();
glMatrixMode(GL_PROJECTION);
glPushMatrix();
glLoadIdentity();
glOrtho (0, x->dim[0], 0, x->dim[1], -1, 1);
glMatrixMode(GL_MODELVIEW);
glPushMatrix();
glLoadIdentity();
glBlendFunc (GL_ONE, GL_ONE_MINUS_SRC_ALPHA);
glEnable (GL_BLEND);
// CODE REMOVED ... IMAGE UNIT RENDERING WOULD BE HERE
// restore state
glClear(GL_DEPTH_BUFFER_BIT);
glMatrixMode(GL_PROJECTION);
glPopMatrix();
glMatrixMode(GL_MODELVIEW);
glPopMatrix();
glMatrixMode(GL_TEXTURE);
glPopMatrix();
glPopAttrib();
} else {
glClear (GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);
}
return err;
}
Thanks Joshua, that was exactly what I was looking for!
For the moment I have only one question:
My guess is that ps_glid should be a symbol, but which one?
I'll try with gensym("glid") and see how far it takes me:)
Yes, exactly. Following the ps_ convention for "pointer to symbol", ps_glid is gensym("glid"). Sorry for not including that sym def in this sample code.
-Joshua
Hi again,
after a bit of a delay, I've returned to opencl external.
So far I've tried to render a fullscreen quad containing the internal texture to a capturing jit.gl.texture - this works ok.
I've also tried to pass a jit.gl.texture's glid directly to opencl as a 'compute result' and this is where it fails.
I do get the glid of jit.gl.texture properly (it's 1), but if I pass it to opencl's clCreateFromGLTexture2D:
I get this error:
CL_INVALID_GL_OBJECT if texture is not a GL texture object whose type matches texture_target, if the specified miplevel of texture is not defined, or if the width or height of the specified miplevel is zero.
this is how the texture is created in the example code:
glTexParameteri(TextureTarget, GL_TEXTURE_WRAP_S, GL_CLAMP);
glTexParameteri(TextureTarget, GL_TEXTURE_WRAP_T, GL_CLAMP);
glTexParameteri(TextureTarget, GL_TEXTURE_MAG_FILTER, GL_NEAREST);
glTexParameteri(TextureTarget, GL_TEXTURE_MIN_FILTER, GL_NEAREST);
TextureTarget is GL_TEXTURE_2D
and the jit.gl.texture has attributes @rectangle 0, @filter nearest, @wrap clamp clamp
Now I don't know where to look next. Any pointers?
Thanks,
nesa
I haven't gotten into OpenCL code yet and am not certain how you're integrating OpenGL/OpenGL, but I imagine that there's potentially some issues with contexts not being in the same share group and thus being able to share resources across contexts. Here is a thread on the Apple OpenGL mailing list which shows how to create an OpenCL context which shares the current OpenGL context. Something similar should work once you've set your jitter GL context to be current. You may also need to rebuild your CL context on GL context rebuild.
Let us know if there are specific questions...
-Joshua