Efficient way to display a mix of video, image and 3D
Hi all,
What is your most efficient way to play with Max/Jitter to mix a number of video, image and sometimes 3D render ?
For example, I am capturing a video and stretching it as the background. Now, I would like to add a static frame on top of it. Then a few images which can floating on top of it with 3D motions.
The thing is BIG and have a resolution no less than 1600 x 1200.
Way 1 : Go 3D
I guess the most natural way is to have all the things up there be represented as jit.videoplane and then it is easy to manipulate.
However, I am wondering if I am wasting resources on copying the captured video to texture memory constant, it is the background anyway.
Way 2 : Go 2D
OK. Jitter has all sort of method to bind video as matrix. So, what if we rendering the small clips into matrix and combine it ? Well, this method is very slow with the "to_texture" option.
Way 3 : Go 2D + 3D
Finally, is it possible to have the video and the static frame blended and draw as background; then render the small clips on top of the back-buffer? The method make sense, but I can't find a way to do it in max/jitter.
What's your idea ?
William
something like this?
On Nov 3, 2007, at 10:47 AM, williamshome wrote:
>
> Hi all,
>
> What is your most efficient way to play with Max/Jitter to mix a
> number of video, image and sometimes 3D render ?
>
> For example, I am capturing a video and stretching it as the
> background. Now, I would like to add a static frame on top of it.
> Then a few images which can floating on top of it with 3D motions.
>
> The thing is BIG and have a resolution no less than 1600 x 1200.
>
> Way 1 : Go 3D
>
> I guess the most natural way is to have all the things up there be
> represented as jit.videoplane and then it is easy to manipulate.
>
> However, I am wondering if I am wasting resources on copying the
> captured video to texture memory constant, it is the background
> anyway.
>
> Way 2 : Go 2D
>
> OK. Jitter has all sort of method to bind video as matrix. So, what
> if we rendering the small clips into matrix and combine it ? Well,
> this method is very slow with the "to_texture" option.
>
> Way 3 : Go 2D + 3D
>
> Finally, is it possible to have the video and the static frame
> blended and draw as background; then render the small clips on top
> of the back-buffer? The method make sense, but I can't find a way to
> do it in max/jitter.
>
> What's your idea ?
>
> William
>
draw pixels does not interpolate however, use a video plane with
@transform_reset 2 and @automatic 0 or @layer. migrating to slab
processing with that is easier too
:)
On Nov 3, 2007, at 11:04 AM, joshua goldberg wrote:
> something like this?
>
> #P window setfont "Sans Serif" 9.;
> #P window linecount 1;
> #P comment 152 483 100 196617 step 3b;
> #P window linecount 2;
> #P newex 87 510 277 196617 jit.gl.gridshape bork @shape torus @name
> shapey @texture cubetex @scale 0.1 0.1 0.1 @position -0.2 0.1 0
> @automatic 0;
> #P window linecount 1;
> #P message 252 392 30 196617 read;
> #P comment 299 416 100 196617 step 3a;
> #P newex 228 414 63 196617 jit.qt.movie;
> #P newex 228 439 164 196617 jit.gl.texture bork @name cubetex;
> #P comment 472 390 100 196617 step 2;
> #P window linecount 3;
> #P newex 417 414 169 196617 jit.gl.gridshape bork @shape plane
> @color 0. 0. 0. 0. @scale 0.7 0.7 0.7 @automatic 0;
> #P window linecount 1;
> #P message 178 115 30 196617 read;
> #P newex 151 164 104 196617 prepend draw_pixels;
> #P newex 151 138 107 196617 jit.qt.movie @adapt 1;
> #P message 335 181 34 196617 reset;
> #P newex 335 205 188 196617 jit.gl.handle bork @inherit_transform 1;
> #P message 444 107 70 196617 fullscreen $1;
> #P toggle 444 87 15 0;
> #P newex 444 64 38 196617 sel 27;
> #P newex 444 42 40 196617 key;
> #P number 91 41 35 9 0 0 0 3 0 0 0 221 221 221 222 222 222 0 0 0;
> #P user jit.fpsgui 28 274 60 196617 0;
> #P newex 31 365 416 196617 t b b b b b b b b b b b b b b b b b b b;
> #P newex 31 341 40 196617 r stuff;
> #P toggle 31 47 15 0;
> #P newex 60 130 40 196617 s stuff;
> #P newex 28 249 201 196617 jit.gl.render bork @erase_color 0. 0. 0.
> 1.;
> #P newex 31 98 138 196617 t b b b;
> #P newex 31 73 57 196617 qmetro 33;
> #P newex 444 128 207 196617 jit.window bork @floating 1 @depthbuffer
> 1;
> #P comment 267 137 100 196617 step 1;
> #P connect 8 5 26 0;
> #P connect 25 0 23 0;
> #P connect 23 0 22 0;
> #P connect 8 9 23 0;
> #P connect 8 18 20 0;
> #P connect 19 0 17 0;
> #P connect 18 0 4 0;
> #P connect 17 0 18 0;
> #P connect 3 2 17 0;
> #P connect 2 0 3 0;
> #P connect 3 0 4 0;
> #P connect 3 1 5 0;
> #P connect 14 0 1 0;
> #P connect 13 0 14 0;
> #P connect 12 0 13 0;
> #P connect 11 0 12 0;
> #P connect 16 0 15 0;
> #P connect 10 0 2 1;
> #P connect 7 0 8 0;
> #P connect 4 0 9 0;
> #P connect 15 0 4 0;
> #P connect 6 0 2 0;
> #P window clipboard copycount 28;
>
> On Nov 3, 2007, at 10:47 AM, williamshome wrote:
>
>>
>> Hi all,
>>
>> What is your most efficient way to play with Max/Jitter to mix a
>> number of video, image and sometimes 3D render ?
>>
>> For example, I am capturing a video and stretching it as the
>> background. Now, I would like to add a static frame on top of it.
>> Then a few images which can floating on top of it with 3D motions.
>>
>> The thing is BIG and have a resolution no less than 1600 x 1200.
>>
>> Way 1 : Go 3D
>>
>> I guess the most natural way is to have all the things up there be
>> represented as jit.videoplane and then it is easy to manipulate.
>>
>> However, I am wondering if I am wasting resources on copying the
>> captured video to texture memory constant, it is the background
>> anyway.
>>
>> Way 2 : Go 2D
>>
>> OK. Jitter has all sort of method to bind video as matrix. So, what
>> if we rendering the small clips into matrix and combine it ? Well,
>> this method is very slow with the "to_texture" option.
>>
>> Way 3 : Go 2D + 3D
>>
>> Finally, is it possible to have the video and the static frame
>> blended and draw as background; then render the small clips on top
>> of the back-buffer? The method make sense, but I can't find a way
>> to do it in max/jitter.
>>
>> What's your idea ?
>>
>> William
>>
>
vade is right.
On Nov 3, 2007, at 11:29 AM, vade wrote:
> draw pixels does not interpolate however, use a video plane with
> @transform_reset 2 and @automatic 0 or @layer. migrating to slab
> processing with that is easier too
>
> :)
> On Nov 3, 2007, at 11:04 AM, joshua goldberg wrote:
>
>> something like this?
>>
>> #P window setfont "Sans Serif" 9.;
>> #P window linecount 1;
>> #P comment 152 483 100 196617 step 3b;
>> #P window linecount 2;
>> #P newex 87 510 277 196617 jit.gl.gridshape bork @shape torus @name
>> shapey @texture cubetex @scale 0.1 0.1 0.1 @position -0.2 0.1 0
>> @automatic 0;
>> #P window linecount 1;
>> #P message 252 392 30 196617 read;
>> #P comment 299 416 100 196617 step 3a;
>> #P newex 228 414 63 196617 jit.qt.movie;
>> #P newex 228 439 164 196617 jit.gl.texture bork @name cubetex;
>> #P comment 472 390 100 196617 step 2;
>> #P window linecount 3;
>> #P newex 417 414 169 196617 jit.gl.gridshape bork @shape plane
>> @color 0. 0. 0. 0. @scale 0.7 0.7 0.7 @automatic 0;
>> #P window linecount 1;
>> #P message 178 115 30 196617 read;
>> #P newex 151 164 104 196617 prepend draw_pixels;
>> #P newex 151 138 107 196617 jit.qt.movie @adapt 1;
>> #P message 335 181 34 196617 reset;
>> #P newex 335 205 188 196617 jit.gl.handle bork @inherit_transform 1;
>> #P message 444 107 70 196617 fullscreen $1;
>> #P toggle 444 87 15 0;
>> #P newex 444 64 38 196617 sel 27;
>> #P newex 444 42 40 196617 key;
>> #P number 91 41 35 9 0 0 0 3 0 0 0 221 221 221 222 222 222 0 0 0;
>> #P user jit.fpsgui 28 274 60 196617 0;
>> #P newex 31 365 416 196617 t b b b b b b b b b b b b b b b b b b b;
>> #P newex 31 341 40 196617 r stuff;
>> #P toggle 31 47 15 0;
>> #P newex 60 130 40 196617 s stuff;
>> #P newex 28 249 201 196617 jit.gl.render bork @erase_color 0. 0. 0.
>> 1.;
>> #P newex 31 98 138 196617 t b b b;
>> #P newex 31 73 57 196617 qmetro 33;
>> #P newex 444 128 207 196617 jit.window bork @floating 1
>> @depthbuffer 1;
>> #P comment 267 137 100 196617 step 1;
>> #P connect 8 5 26 0;
>> #P connect 25 0 23 0;
>> #P connect 23 0 22 0;
>> #P connect 8 9 23 0;
>> #P connect 8 18 20 0;
>> #P connect 19 0 17 0;
>> #P connect 18 0 4 0;
>> #P connect 17 0 18 0;
>> #P connect 3 2 17 0;
>> #P connect 2 0 3 0;
>> #P connect 3 0 4 0;
>> #P connect 3 1 5 0;
>> #P connect 14 0 1 0;
>> #P connect 13 0 14 0;
>> #P connect 12 0 13 0;
>> #P connect 11 0 12 0;
>> #P connect 16 0 15 0;
>> #P connect 10 0 2 1;
>> #P connect 7 0 8 0;
>> #P connect 4 0 9 0;
>> #P connect 15 0 4 0;
>> #P connect 6 0 2 0;
>> #P window clipboard copycount 28;
>>
>> On Nov 3, 2007, at 10:47 AM, williamshome wrote:
>>
>>>
>>> Hi all,
>>>
>>> What is your most efficient way to play with Max/Jitter to mix a
>>> number of video, image and sometimes 3D render ?
>>>
>>> For example, I am capturing a video and stretching it as the
>>> background. Now, I would like to add a static frame on top of it.
>>> Then a few images which can floating on top of it with 3D motions.
>>>
>>> The thing is BIG and have a resolution no less than 1600 x 1200.
>>>
>>> Way 1 : Go 3D
>>>
>>> I guess the most natural way is to have all the things up there be
>>> represented as jit.videoplane and then it is easy to manipulate.
>>>
>>> However, I am wondering if I am wasting resources on copying the
>>> captured video to texture memory constant, it is the background
>>> anyway.
>>>
>>> Way 2 : Go 2D
>>>
>>> OK. Jitter has all sort of method to bind video as matrix. So,
>>> what if we rendering the small clips into matrix and combine it ?
>>> Well, this method is very slow with the "to_texture" option.
>>>
>>> Way 3 : Go 2D + 3D
>>>
>>> Finally, is it possible to have the video and the static frame
>>> blended and draw as background; then render the small clips on top
>>> of the back-buffer? The method make sense, but I can't find a way
>>> to do it in max/jitter.
>>>
>>> What's your idea ?
>>>
>>> William
>>>
>>
>
>
and shouldn't the dim of the video plane be changed from the default of 20x20 to 2x2 for efficiency?
yeah, I was never sure why its 20x20 :)
On Nov 3, 2007, at 3:00 PM, Robert Ramirez wrote:
>
> and shouldn't the dim of the video plane be changed from the default
> of 20x20 to 2x2 for efficiency?
Thanks joshua, your example is nice and neat.
While, I am not sure what vade means in "migrating to slab processing with that is easier". I wish so much to accelerate what I am doing in GPU, but doesn't seems to work.
Thanks all ppl. ^_^
Have a look at the patches in jitter-examples/render/slab/. They ought to give you some idea of how to integrate your slab processing into your patches.
best,
dan