Forums > Jitter

cc.uyvy2rgba.jxs and optimization

November 23, 2007 | 1:42 am

Upon vades suggestion and searching the archives….

http://www.cycling74.com/forums/index.php?t=msg&goto=113246&rid=0&S=12672e78d4a9aad871b2693f9a7bbb5b#msg_113246

I tried implementing this conversion on the gpu. The problem is I get funny colored purpley/greeny/red distorted lines on the output window… and when I try to use either dx | qt.grab the screen is distorted.

The good news is my fps is WAY BETTER!!! without using @colormode uyvy on the qt.movie I can only get 100fps on the patch I’m posting whereas with the shader doing the conversion I get up to 160 fps !!???!! So weird. Any help would greatly be appreciated.

I’m using windows XP PRO on an AMD s939 4800X2 with the latest max and jitter.and a Viper ATI Radeon HD 3870 gpu.

#P window setfont "Sans Serif" 9.;
#P number 418 72 35 9 1 3 3 139 0 0 0 221 221 221 222 222 222 0 0 0;
#P window linecount 1;
#P newex 403 193 214 9109513 jit.qt.movie 720 480 @unique 1 @colormode argb;
#P newex 403 228 77 9109513 jit.gl.slab window;
#P user jit.fpsgui 155 341 60 9109513 0;
#P message 621 156 30 9109513 close;
#P newex 623 121 57 9109513 select 1 2 3;
#P message 652 156 28 9109513 open;
#P newex 388 154 42 9109513 gate 3 1;
#P newex 623 193 267 9109513 jit.dx.grab 720 480 @vdevice 0 @unique 1 @colormode uyvy;
#P message 80 398 62 9109513 fullscreen $1;
#P toggle 80 380 15 0;
#P newex 58 330 40 9109513 key;
#P newex 80 357 46 9109513 select 27;
#P newex 141 227 250 9109513 jit.gl.slab window @file cc.uyvy2rgba.jxs @dimscale 2. 1.;
#P newex 308 30 48 9109513 loadbang;
#P newex 139 84 30 9109513 t b b;
#P toggle 138 40 15 0;
#P newex 138 64 45 9109513 qmetro 5;
#P newex 75 121 111 9109513 t b b erase;
#P newex 193 308 192 9109513 jit.gl.videoplane window @transform_reset 2;
#P newex 142 188 219 9109513 jit.qt.movie 720 480 @unique 1 @colormode uyvy;
#P message 310 117 76 9109513 read bball.mov;
#P newex 69 426 199 9109513 jit.window window @depthbuffer 1 @floating 1;
#P newex 75 299 86 9109513 jit.gl.render window;
#P connect 18 2 17 0;
#P connect 16 2 15 0;
#P connect 19 0 15 0;
#P connect 17 0 15 0;
#P connect 23 0 16 0;
#P connect 23 0 18 0;
#P connect 18 1 19 0;
#P connect 18 0 19 0;
#P connect 5 1 16 1;
#P connect 22 0 21 0;
#P connect 16 1 22 0;
#P fasten 2 0 3 0 315 144 147 144;
#P connect 2 0 22 0;
#P connect 9 0 2 0;
#P connect 10 0 4 0;
#P connect 21 0 4 0;
#P connect 0 0 20 0;
#P connect 16 0 3 0;
#P fasten 3 0 10 0 147 218 146 218;
#P fasten 15 0 10 0 628 218 146 218;
#P connect 6 0 8 0;
#P connect 7 0 6 0;
#P connect 13 0 14 0;
#P connect 11 0 13 0;
#P connect 12 0 11 0;
#P connect 5 0 0 0;
#P connect 5 2 0 0;
#P fasten 8 0 5 0 144 108 80 108;
#P connect 14 0 1 0;
#P window clipboard copycount 24;


November 23, 2007 | 9:31 am

Hi Tyler,
I don’t know about the funny colors you see, but on the fps side, i
understand that jit.qt.movie is doing the conversion from argb to uyvy
internally. on some bigger movies I get even better performance when I
transform the source movie to a half sized "quasi" uyvy movie
beforehand. (see attahced patch). I only see some "trailing" on that
kind of material. this is on MacBook Pro, OSX 10.4.10, Quicktime 7.2

John.

max v2;
#N vpatcher 461 240 1347 803;
#P window setfont "Sans Serif" 9.;
#P window linecount 1;
#P comment 638 116 100 196617 5. read new movie;
#P comment 119 209 100 196617 4. stop;
#P window linecount 2;
#P comment 116 179 100 196617 2. select write location;
#P window linecount 1;
#P message 597 119 30 196617 read;
#P newex 554 193 297 196617 jit.gl.videoplane content @colormode uyvy
@transform_reset 2;
#P newex 554 167 155 196617 jit.qt.movie @adapt 1 @unique 1;
#P newex 490 307 94 196617 jit.window content;
#P user jit.fpsgui 435 125 60 196617 0;
#P newex 417 348 101 196617 jit.gl.render content;
#P newex 417 97 66 196617 t b b b erase;
#P toggle 417 48 15 0;
#P newex 417 73 57 196617 qmetro 20;
#P user jit.pwindow 250 161 82 62 0 1 0 0 1 0;
#P message 90 197 29 196617 stop;
#P message 91 160 154 196617 write 25. animation normal 600;
#P newex 57 233 66 196617 jit.qt.record;
#P message 122 66 30 196617 read;
#P message 57 66 60 196617 framedump;
#P newex 57 123 190 196617 jit.qt.movie @adapt 1 @colormode uyvy;
#P comment 125 45 100 196617 1. read movie;
#P comment 21 45 100 196617 3. framedump;
#P fasten 4 0 2 0 127 99 62 99;
#P connect 3 0 2 0;
#P connect 2 0 5 0;
#P fasten 7 0 5 0 95 224 62 224;
#P fasten 6 0 5 0 96 181 62 181;
#P fasten 2 0 8 0 62 147 256 147;
#P connect 10 0 9 0;
#P connect 9 0 11 0;
#P connect 11 0 12 0;
#P fasten 11 3 12 0 476 160 422 160;
#P connect 11 1 13 0;
#P connect 17 0 15 0;
#P fasten 11 2 15 0 458 121 559 121;
#P connect 15 0 16 0;
#P pop;

Tyler Nitsch schrieb:
> Upon vades suggestion and searching the archives….
>
> http://www.cycling74.com/forums/index.php?t=msg&goto=113246&rid=0&S=12672e78d4a9aad871b2693f9a7bbb5b#msg_113246
>
> I tried implementing this conversion on the gpu. The problem is I get funny colored purpley/greeny/red distorted lines on the output window… and when I try to use either dx | qt.grab the screen is distorted.
>
> The good news is my fps is WAY BETTER!!! without using @colormode uyvy on the qt.movie I can only get 100fps on the patch I’m posting whereas with the shader doing the conversion I get up to 160 fps !!???!! So weird. Any help would greatly be appreciated.
>
> I’m using windows XP PRO on an AMD s939 4800X2 with the latest max and jitter.and a Viper ATI Radeon HD 3870 gpu.
>
> #P window setfont "Sans Serif" 9.;
> #P number 418 72 35 9 1 3 3 139 0 0 0 221 221 221 222 222 222 0 0 0;
> #P window linecount 1;
> #P newex 403 193 214 9109513 jit.qt.movie 720 480 @unique 1 @colormode argb;
> #P newex 403 228 77 9109513 jit.gl.slab window;
> #P user jit.fpsgui 155 341 60 9109513 0;
> #P message 621 156 30 9109513 close;
> #P newex 623 121 57 9109513 select 1 2 3;
> #P message 652 156 28 9109513 open;
> #P newex 388 154 42 9109513 gate 3 1;
> #P newex 623 193 267 9109513 jit.dx.grab 720 480 @vdevice 0 @unique 1 @colormode uyvy;
> #P message 80 398 62 9109513 fullscreen $1;
> #P toggle 80 380 15 0;
> #P newex 58 330 40 9109513 key;
> #P newex 80 357 46 9109513 select 27;
> #P newex 141 227 250 9109513 jit.gl.slab window @file cc.uyvy2rgba.jxs @dimscale 2. 1.;
> #P newex 308 30 48 9109513 loadbang;
> #P newex 139 84 30 9109513 t b b;
> #P toggle 138 40 15 0;
> #P newex 138 64 45 9109513 qmetro 5;
> #P newex 75 121 111 9109513 t b b erase;
> #P newex 193 308 192 9109513 jit.gl.videoplane window @transform_reset 2;
> #P newex 142 188 219 9109513 jit.qt.movie 720 480 @unique 1 @colormode uyvy;
> #P message 310 117 76 9109513 read bball.mov;
> #P newex 69 426 199 9109513 jit.window window @depthbuffer 1 @floating 1;
> #P newex 75 299 86 9109513 jit.gl.render window;
> #P connect 18 2 17 0;
> #P connect 16 2 15 0;
> #P connect 19 0 15 0;
> #P connect 17 0 15 0;
> #P connect 23 0 16 0;
> #P connect 23 0 18 0;
> #P connect 18 1 19 0;
> #P connect 18 0 19 0;
> #P connect 5 1 16 1;
> #P connect 22 0 21 0;
> #P connect 16 1 22 0;
> #P fasten 2 0 3 0 315 144 147 144;
> #P connect 2 0 22 0;
> #P connect 9 0 2 0;
> #P connect 10 0 4 0;
> #P connect 21 0 4 0;
> #P connect 0 0 20 0;
> #P connect 16 0 3 0;
> #P fasten 3 0 10 0 147 218 146 218;
> #P fasten 15 0 10 0 628 218 146 218;
> #P connect 6 0 8 0;
> #P connect 7 0 6 0;
> #P connect 13 0 14 0;
> #P connect 11 0 13 0;
> #P connect 12 0 11 0;
> #P connect 5 0 0 0;
> #P connect 5 2 0 0;
> #P fasten 8 0 5 0 144 108 80 108;
> #P connect 14 0 1 0;
> #P window clipboard copycount 24;
>
>
>


November 23, 2007 | 5:40 pm

On Nov 23, 2007, at 1:31 AM, John Dekron wrote:

> I don’t know about the funny colors you see, but on the fps side, i
> understand that jit.qt.movie is doing the conversion from argb to
> uyvy internally.

FWIW, QT only has to convert from argb to uyvy if the format is
natively an argb format, but John, maybe you had these reversed
above, since it sounds like Tyler is getting better framerates with
UYVY, as should be expected.

If these objects needs to do internal rescaling of the video (i.e. if
the movie/camera dimensions do not match the requested dimensions),
there is scaling of the UYVY data on the CPU which is both expensive,
and can produce chroma artifacts, which is perhaps what you are
experiencing, Tyler. I would suggest also using @adapt 1 in
jit.qt.movie *and* jit.dx/qt.grab, and using the texture’s HW
rescaling (which also happens in rgba, after the uyvy to argb
conversion).

-Joshua


November 23, 2007 | 6:00 pm

@adapt 1 in jit.qt.movie *and* jit.dx/qt.grab,

Have I really, until now, not known that jit.qt.grab could use @adapt
1 to get native dimensions of the capture device? I love you.

On Nov 23, 2007, at 12:40 PM, Joshua Kit Clayton wrote:

>
> On Nov 23, 2007, at 1:31 AM, John Dekron wrote:
>
>> I don’t know about the funny colors you see, but on the fps side, i
>> understand that jit.qt.movie is doing the conversion from argb to
>> uyvy internally.
>
> FWIW, QT only has to convert from argb to uyvy if the format is
> natively an argb format, but John, maybe you had these reversed
> above, since it sounds like Tyler is getting better framerates with
> UYVY, as should be expected.
>
> If these objects needs to do internal rescaling of the video (i.e.
> if the movie/camera dimensions do not match the requested
> dimensions), there is scaling of the UYVY data on the CPU which is
> both expensive, and can produce chroma artifacts, which is perhaps
> what you are experiencing, Tyler. I would suggest also using @adapt
> 1 in jit.qt.movie *and* jit.dx/qt.grab, and using the texture’s HW
> rescaling (which also happens in rgba, after the uyvy to argb
> conversion).
>
> -Joshua
>
>


November 23, 2007 | 7:23 pm

On Nov 23, 2007, at 10:00 AM, vade wrote:

> @adapt 1 in jit.qt.movie *and* jit.dx/qt.grab,
>
> Have I really, until now, not known that jit.qt.grab could use
> @adapt 1 to get native dimensions of the capture device? I love you.

Oops. I misremembered, and was totally wrong here, so it’s up to you
to make sure that the dimensions match the input source of the camera
to prevent the video scaling I mentioned.

Sorry about that.

-Joshua


November 24, 2007 | 6:58 pm

Ok even using the above sample patch and changing the attribute within qt.movie to @adapt 1, I still get the funny colored lines… with dx.grab I’ve tried several resolutions and same results also.

This is wierd, I was using the same patch on an intel P4 with an ati radeon 1650 everything worked as should be and the performance was far superior even though the hardware wasnt’


November 24, 2007 | 7:34 pm

can you send a screencap/pict / patch?

On Nov 24, 2007, at 1:58 PM, Tyler Nitsch wrote:

>
> Ok even using the above sample patch and changing the attribute
> within qt.movie to @adapt 1, I still get the funny colored lines…
> with dx.grab I’ve tried several resolutions and same results also.
>
> This is wierd, I was using the same patch on an intel P4 with an ati
> radeon 1650 everything worked as should be and the performance was
> far superior even though the hardware wasnt’


November 24, 2007 | 7:39 pm

I’m not quite sure what you mean buy textures HW rescaling.. are you talking about setting autoscale to 1? This is an area I’m actually confused about… dimensional scaling.. texture scaling???


November 24, 2007 | 9:16 pm

Hey Vade here is a simplified version of the patch I previously posted.

t

#P window setfont "Sans Serif" 9.;
#P user jit.fpsgui 31 179 60 9109513 0;
#P window linecount 1;
#P message 58 289 62 9109513 fullscreen $1;
#P toggle 58 271 15 0;
#P newex 36 221 40 9109513 key;
#P newex 58 248 46 9109513 select 27;
#P newex 229 191 250 9109513 jit.gl.slab window @file cc.uyvy2rgba.jxs @dimscale 2. 1.;
#P newex 308 30 48 9109513 loadbang;
#P toggle 138 38 15 0;
#P newex 138 64 45 9109513 qmetro 5;
#P newex 138 89 111 9109513 t b b erase;
#P newex 230 216 192 9109513 jit.gl.videoplane window @transform_reset 2;
#P newex 230 152 226 9109513 jit.qt.movie @unique 1 @colormode uyvy @adapt 1;
#P message 310 117 76 9109513 read bball.mov;
#P newex 47 317 199 9109513 jit.window window @depthbuffer 1 @floating 1;
#P newex 31 154 86 9109513 jit.gl.render window;
#P fasten 5 0 0 0 143 131 36 131;
#P fasten 5 2 0 0 243 131 36 131;
#P connect 0 0 14 0;
#P connect 13 0 1 0;
#P connect 11 0 10 0;
#P connect 10 0 12 0;
#P connect 12 0 13 0;
#P connect 7 0 6 0;
#P connect 6 0 5 0;
#P connect 3 0 9 0;
#P connect 5 1 3 0;
#P fasten 2 0 3 0 315 144 235 144;
#P connect 9 0 4 0;
#P connect 8 0 2 0;
#P window clipboard copycount 15;

and the picture is attached.

[img]index.php?t=getfile&id=1000&private=0[/img][img]index.php?t=getfile&id=1000&private=0[/img]


November 25, 2007 | 5:22 pm

On Nov 24, 2007, at 1:17 PM, Tyler Nitsch wrote:

>
> and the picture is attached.
> [img]index.php?t=getfile&id=1000&private=0[/img][img]index.php?
> t=getfile&id=1000&private=0[/img]
>

Very strange. Might be some driver issue with the UYVY shader, or
some floating point error in the math. One thing this might be is
accidentally grabbing the wrong input planes for the chroma
conversion. Another thing this might be is fold over for over/
undersaturation of the color channels. For this, you could try
clamping all your output colors to 0-1 before setting the
gl_fragcolor. If looking at those possible fixes, perhaps updating to
latest drivers may solve it. Unfortunately, kind of hard for us to
investigate without a similar setup.

-Joshua


November 25, 2007 | 7:38 pm

Can you try to cc.uyvy2rgba lite shader? I was thinking is some sort
of rounding issue with the shader, but I honestly have no idea.

On Nov 25, 2007, at 12:22 PM, Joshua Kit Clayton wrote:

>
> On Nov 24, 2007, at 1:17 PM, Tyler Nitsch wrote:
>
>>
>> and the picture is attached.
>> [img]index.php?t=getfile&id=1000&private=0[/img][img]index.php?
>> t=getfile&id=1000&private=0[/img]
>>

>
> Very strange. Might be some driver issue with the UYVY shader, or
> some floating point error in the math. One thing this might be is
> accidentally grabbing the wrong input planes for the chroma
> conversion. Another thing this might be is fold over for over/
> undersaturation of the color channels. For this, you could try
> clamping all your output colors to 0-1 before setting the
> gl_fragcolor. If looking at those possible fixes, perhaps updating
> to latest drivers may solve it. Unfortunately, kind of hard for us
> to investigate without a similar setup.
>
> -Joshua
>


December 6, 2007 | 9:31 am

thank you mr. Dekron, your pre uyvy method has gotten me a 768×768 movie run
a lot nicer on an xp.

btw. is there a method to force other post enviroments (aftereffects, shake
etc.) to export to uyvy.
most of the post friends i asked looked at me funny.

On Nov 23, 2007 11:31 AM, John Dekron

wrote:

> Hi Tyler,
> I don’t know about the funny colors you see, but on the fps side, i
> understand that jit.qt.movie is doing the conversion from argb to uyvy
> internally. on some bigger movies I get even better performance when I
> transform the source movie to a half sized "quasi" uyvy movie
> beforehand. (see attahced patch). I only see some "trailing" on that
> kind of material. this is on MacBook Pro, OSX 10.4.10, Quicktime 7.2
>
> John.
>
> max v2;
> #N vpatcher 461 240 1347 803;
> #P window setfont "Sans Serif" 9.;
> #P window linecount 1;
> #P comment 638 116 100 196617 5. read new movie;
> #P comment 119 209 100 196617 4. stop;
> #P window linecount 2;
> #P comment 116 179 100 196617 2. select write location;
> #P window linecount 1;
> #P message 597 119 30 196617 read;
> #P newex 554 193 297 196617 jit.gl.videoplane content @colormode uyvy
> @transform_reset 2;
> #P newex 554 167 155 196617 jit.qt.movie @adapt 1 @unique 1;
> #P newex 490 307 94 196617 jit.window content;
> #P user jit.fpsgui 435 125 60 196617 0;
> #P newex 417 348 101 196617 jit.gl.render content;
> #P newex 417 97 66 196617 t b b b erase;
> #P toggle 417 48 15 0;
> #P newex 417 73 57 196617 qmetro 20;
> #P user jit.pwindow 250 161 82 62 0 1 0 0 1 0;
> #P message 90 197 29 196617 stop;
> #P message 91 160 154 196617 write 25. animation normal 600;
> #P newex 57 233 66 196617 jit.qt.record;
> #P message 122 66 30 196617 read;
> #P message 57 66 60 196617 framedump;
> #P newex 57 123 190 196617 jit.qt.movie @adapt 1 @colormode uyvy;
> #P comment 125 45 100 196617 1. read movie;
> #P comment 21 45 100 196617 3. framedump;
> #P fasten 4 0 2 0 127 99 62 99;
> #P connect 3 0 2 0;
> #P connect 2 0 5 0;
> #P fasten 7 0 5 0 95 224 62 224;
> #P fasten 6 0 5 0 96 181 62 181;
> #P fasten 2 0 8 0 62 147 256 147;
> #P connect 10 0 9 0;
> #P connect 9 0 11 0;
> #P connect 11 0 12 0;
> #P fasten 11 3 12 0 476 160 422 160;
> #P connect 11 1 13 0;
> #P connect 17 0 15 0;
> #P fasten 11 2 15 0 458 121 559 121;
> #P connect 15 0 16 0;
> #P pop;
>
>
> Tyler Nitsch schrieb:
> > Upon vades suggestion and searching the archives….
> >
> >
> http://www.cycling74.com/forums/index.php?t=msg&goto=113246&rid=0&S=12672e78d4a9aad871b2693f9a7bbb5b#msg_113246
> >
> > I tried implementing this conversion on the gpu. The problem is I get
> funny colored purpley/greeny/red distorted lines on the output window… and
> when I try to use either dx | qt.grab the screen is distorted.
> >
> > The good news is my fps is WAY BETTER!!! without using @colormode uyvy
> on the qt.movie I can only get 100fps on the patch I’m posting whereas
> with the shader doing the conversion I get up to 160 fps !!???!! So weird.
> Any help would greatly be appreciated.
> >
> > I’m using windows XP PRO on an AMD s939 4800X2 with the latest max and
> jitter.and a Viper ATI Radeon HD 3870 gpu.
> >
> > #P window setfont "Sans Serif" 9.;
> > #P number 418 72 35 9 1 3 3 139 0 0 0 221 221 221 222 222 222 0 0 0;
> > #P window linecount 1;
> > #P newex 403 193 214 9109513 jit.qt.movie 720 480 @unique 1 @colormode
> argb;
> > #P newex 403 228 77 9109513 jit.gl.slab window;
> > #P user jit.fpsgui 155 341 60 9109513 0;
> > #P message 621 156 30 9109513 close;
> > #P newex 623 121 57 9109513 select 1 2 3;
> > #P message 652 156 28 9109513 open;
> > #P newex 388 154 42 9109513 gate 3 1;
> > #P newex 623 193 267 9109513 jit.dx.grab 720 480 @vdevice 0 @unique 1
> @colormode uyvy;
> > #P message 80 398 62 9109513 fullscreen $1;
> > #P toggle 80 380 15 0;
> > #P newex 58 330 40 9109513 key;
> > #P newex 80 357 46 9109513 select 27;
> > #P newex 141 227 250 9109513 jit.gl.slab window @file cc.uyvy2rgba.jxs@dimscale 2. 1.;
> > #P newex 308 30 48 9109513 loadbang;
> > #P newex 139 84 30 9109513 t b b;
> > #P toggle 138 40 15 0;
> > #P newex 138 64 45 9109513 qmetro 5;
> > #P newex 75 121 111 9109513 t b b erase;
> > #P newex 193 308 192 9109513 jit.gl.videoplane window @transform_reset
> 2;
> > #P newex 142 188 219 9109513 jit.qt.movie 720 480 @unique 1 @colormode
> uyvy;
> > #P message 310 117 76 9109513 read bball.mov;
> > #P newex 69 426 199 9109513 jit.window window @depthbuffer 1 @floating
> 1;
> > #P newex 75 299 86 9109513 jit.gl.render window;
> > #P connect 18 2 17 0;
> > #P connect 16 2 15 0;
> > #P connect 19 0 15 0;
> > #P connect 17 0 15 0;
> > #P connect 23 0 16 0;
> > #P connect 23 0 18 0;
> > #P connect 18 1 19 0;
> > #P connect 18 0 19 0;
> > #P connect 5 1 16 1;
> > #P connect 22 0 21 0;
> > #P connect 16 1 22 0;
> > #P fasten 2 0 3 0 315 144 147 144;
> > #P connect 2 0 22 0;
> > #P connect 9 0 2 0;
> > #P connect 10 0 4 0;
> > #P connect 21 0 4 0;
> > #P connect 0 0 20 0;
> > #P connect 16 0 3 0;
> > #P fasten 3 0 10 0 147 218 146 218;
> > #P fasten 15 0 10 0 628 218 146 218;
> > #P connect 6 0 8 0;
> > #P connect 7 0 6 0;
> > #P connect 13 0 14 0;
> > #P connect 11 0 13 0;
> > #P connect 12 0 11 0;
> > #P connect 5 0 0 0;
> > #P connect 5 2 0 0;
> > #P fasten 8 0 5 0 144 108 80 108;
> > #P connect 14 0 1 0;
> > #P window clipboard copycount 24;
> >
> >
> >
>
>


December 30, 2007 | 3:08 am

yes dekron, yair,

I have been looking into this method of doing the conversion before,
and it does help, doubling the frame rate of the display patch and
cutting down on cpu/gpu load.

Problem is, it seems only some codecs work with this method.
Animation which you use I guess compresses over time, because when I
try to display particular frames I get digital artifacts. For Jpeg,
motion-jpeg the colors aren’t right when the film is loading back in.
I had success with PNG codec, but the file sizes after the
compression are quite large.

Any other codecs you could recommend that compress each frame
separately and work with this method of creating "quasi" uyvy to be
decoded as such on the GPU?

To get Photo-Jpeg/Motion-Jpeg to work I tried to map the planes
differently, but couldn’t get the colors to match. Anyone throw some
light on this? Is there a particular website that might describe all
the different standard quicktime codecs? Detailing which compress
every frame separately and which compress over time? Jitter does
better with the former.

cheers

(())_n

On Dec 6, 2007, at 4:31 AM, yair reshef wrote:

> thank you mr. Dekron, your pre uyvy method has gotten me a 768×768
> movie run a lot nicer on an xp.
>
> btw. is there a method to force other post enviroments
> (aftereffects, shake etc.) to export to uyvy.
> most of the post friends i asked looked at me funny.
>
>
>
> On Nov 23, 2007 11:31 AM, John Dekron

wrote:
> Hi Tyler,
> I don’t know about the funny colors you see, but on the fps side, i
> understand that jit.qt.movie is doing the conversion from argb to uyvy
> internally. on some bigger movies I get even better performance when I
> transform the source movie to a half sized "quasi" uyvy movie
> beforehand. (see attahced patch). I only see some "trailing" on that
> kind of material. this is on MacBook Pro, OSX 10.4.10, Quicktime 7.2
>
> John.
>
> max v2;
> #N vpatcher 461 240 1347 803;
> #P window setfont "Sans Serif" 9.;
> #P window linecount 1;
> #P comment 638 116 100 196617 5. read new movie;
> #P comment 119 209 100 196617 4. stop;
> #P window linecount 2;
> #P comment 116 179 100 196617 2. select write location;
> #P window linecount 1;
> #P message 597 119 30 196617 read;
> #P newex 554 193 297 196617 jit.gl.videoplane content @colormode uyvy
> @transform_reset 2;
> #P newex 554 167 155 196617 jit.qt.movie @adapt 1 @unique 1;
> #P newex 490 307 94 196617 jit.window content;
> #P user jit.fpsgui 435 125 60 196617 0;
> #P newex 417 348 101 196617 jit.gl.render content;
> #P newex 417 97 66 196617 t b b b erase;
> #P toggle 417 48 15 0;
> #P newex 417 73 57 196617 qmetro 20;
> #P user jit.pwindow 250 161 82 62 0 1 0 0 1 0;
> #P message 90 197 29 196617 stop;
> #P message 91 160 154 196617 write 25. animation normal 600;
> #P newex 57 233 66 196617 jit.qt.record;
> #P message 122 66 30 196617 read;
> #P message 57 66 60 196617 framedump;
> #P newex 57 123 190 196617 jit.qt.movie @adapt 1 @colormode uyvy;
> #P comment 125 45 100 196617 1. read movie;
> #P comment 21 45 100 196617 3. framedump;
> #P fasten 4 0 2 0 127 99 62 99;
> #P connect 3 0 2 0;
> #P connect 2 0 5 0;
> #P fasten 7 0 5 0 95 224 62 224;
> #P fasten 6 0 5 0 96 181 62 181;
> #P fasten 2 0 8 0 62 147 256 147;
> #P connect 10 0 9 0;
> #P connect 9 0 11 0;
> #P connect 11 0 12 0;
> #P fasten 11 3 12 0 476 160 422 160;
> #P connect 11 1 13 0;
> #P connect 17 0 15 0;
> #P fasten 11 2 15 0 458 121 559 121;
> #P connect 15 0 16 0;
> #P pop;
>
>
> Tyler Nitsch schrieb:
> > Upon vades suggestion and searching the archives….
> >
> > http://www.cycling74.com/forums/index.php?
> t=msg&goto=113246&rid=0&S=12672e78d4a9aad871b2693f9a7bbb5b#msg_113246
> >
> > I tried implementing this conversion on the gpu. The problem is
> I get funny colored purpley/greeny/red distorted lines on the
> output window… and when I try to use either dx | qt.grab the
> screen is distorted.
> >
> > The good news is my fps is WAY BETTER!!! without using @colormode
> uyvy on the qt.movie I can only get 100fps on the patch I’m posting
> whereas with the shader doing the conversion I get up to 160
> fps !!???!! So weird. Any help would greatly be appreciated.
> >
> > I’m using windows XP PRO on an AMD s939 4800X2 with the latest
> max and jitter.and a Viper ATI Radeon HD 3870 gpu.
> >
> > #P window setfont "Sans Serif" 9.;
> > #P number 418 72 35 9 1 3 3 139 0 0 0 221 221 221 222 222 222 0 0 0;
> > #P window linecount 1;
> > #P newex 403 193 214 9109513 jit.qt.movie 720 480 @unique 1
> @colormode argb;
> > #P newex 403 228 77 9109513 jit.gl.slab window;
> > #P user jit.fpsgui 155 341 60 9109513 0;
> > #P message 621 156 30 9109513 close;
> > #P newex 623 121 57 9109513 select 1 2 3;
> > #P message 652 156 28 9109513 open;
> > #P newex 388 154 42 9109513 gate 3 1;
> > #P newex 623 193 267 9109513 jit.dx.grab 720 480 @vdevice 0
> @unique 1 @colormode uyvy;
> > #P message 80 398 62 9109513 fullscreen $1;
> > #P toggle 80 380 15 0;
> > #P newex 58 330 40 9109513 key;
> > #P newex 80 357 46 9109513 select 27;
> > #P newex 141 227 250 9109513 jit.gl.slab window @file
> cc.uyvy2rgba.jxs @dimscale 2. 1.;
> > #P newex 308 30 48 9109513 loadbang;
> > #P newex 139 84 30 9109513 t b b;
> > #P toggle 138 40 15 0;
> > #P newex 138 64 45 9109513 qmetro 5;
> > #P newex 75 121 111 9109513 t b b erase;
> > #P newex 193 308 192 9109513 jit.gl.videoplane window
> @transform_reset 2;
> > #P newex 142 188 219 9109513 jit.qt.movie 720 480 @unique 1
> @colormode uyvy;
> > #P message 310 117 76 9109513 read bball.mov;
> > #P newex 69 426 199 9109513 jit.window window @depthbuffer 1
> @floating 1;
> > #P newex 75 299 86 9109513 jit.gl.render window;
> > #P connect 18 2 17 0;
> > #P connect 16 2 15 0;
> > #P connect 19 0 15 0;
> > #P connect 17 0 15 0;
> > #P connect 23 0 16 0;
> > #P connect 23 0 18 0;
> > #P connect 18 1 19 0;
> > #P connect 18 0 19 0;
> > #P connect 5 1 16 1;
> > #P connect 22 0 21 0;
> > #P connect 16 1 22 0;
> > #P fasten 2 0 3 0 315 144 147 144;
> > #P connect 2 0 22 0;
> > #P connect 9 0 2 0;
> > #P connect 10 0 4 0;
> > #P connect 21 0 4 0;
> > #P connect 0 0 20 0;
> > #P connect 16 0 3 0;
> > #P fasten 3 0 10 0 147 218 146 218;
> > #P fasten 15 0 10 0 628 218 146 218;
> > #P connect 6 0 8 0;
> > #P connect 7 0 6 0;
> > #P connect 13 0 14 0;
> > #P connect 11 0 13 0;
> > #P connect 12 0 11 0;
> > #P connect 5 0 0 0;
> > #P connect 5 2 0 0;
> > #P fasten 8 0 5 0 144 108 80 108;
> > #P connect 14 0 1 0;
> > #P window clipboard copycount 24;
> >
> >
> >
>
>


January 2, 2008 | 12:09 pm

Hi _n,
I get the digital artifacts as well and have no idea where they come from.
Animation and png are videocodecs that hold an alpha channel what is
necessary for the uyvy information. If I remember correctly it is
possible to limit the data rate on png compression. I did use png for a
previous project, but lately I experienced some crashes when playing
back png compressed movies. i switched to animation and it worked fine.
Sorry that I didn’t investigate deeper on the crashes that I had, so
this information might as well be treated as gossip- but eventually
there is some truth in it.

good luck. John.

(())_n schrieb:
> yes dekron, yair,
>
> I have been looking into this method of doing the conversion before, and
> it does help, doubling the frame rate of the display patch and cutting
> down on cpu/gpu load.
>
> Problem is, it seems only some codecs work with this method. Animation
> which you use I guess compresses over time, because when I try to
> display particular frames I get digital artifacts. For Jpeg, motion-jpeg
> the colors aren’t right when the film is loading back in. I had success
> with PNG codec, but the file sizes after the compression are quite large.
>
> Any other codecs you could recommend that compress each frame separately
> and work with this method of creating "quasi" uyvy to be decoded as such
> on the GPU?
>
> To get Photo-Jpeg/Motion-Jpeg to work I tried to map the planes
> differently, but couldn’t get the colors to match. Anyone throw some
> light on this? Is there a particular website that might describe all the
> different standard quicktime codecs? Detailing which compress every
> frame separately and which compress over time? Jitter does better with
> the former.
>
> cheers
>
> (())_n
>
>
>
>
>
>
> On Dec 6, 2007, at 4:31 AM, yair reshef wrote:
>
>> thank you mr. Dekron, your pre uyvy method has gotten me a 768×768
>> movie run a lot nicer on an xp.
>>
>> btw. is there a method to force other post enviroments (aftereffects,
>> shake etc.) to export to uyvy.
>> most of the post friends i asked looked at me funny.
>>
>>
>>
>> On Nov 23, 2007 11:31 AM, John Dekron

wrote:
>> Hi Tyler,
>> I don’t know about the funny colors you see, but on the fps side, i
>> understand that jit.qt.movie is doing the conversion from argb to uyvy
>> internally. on some bigger movies I get even better performance when I
>> transform the source movie to a half sized "quasi" uyvy movie
>> beforehand. (see attahced patch). I only see some "trailing" on that
>> kind of material. this is on MacBook Pro, OSX 10.4.10, Quicktime 7.2
>>
>> John.
>>
>> max v2;
>> #N vpatcher 461 240 1347 803;
>> #P window setfont "Sans Serif" 9.;
>> #P window linecount 1;
>> #P comment 638 116 100 196617 5. read new movie;
>> #P comment 119 209 100 196617 4. stop;
>> #P window linecount 2;
>> #P comment 116 179 100 196617 2. select write location;
>> #P window linecount 1;
>> #P message 597 119 30 196617 read;
>> #P newex 554 193 297 196617 jit.gl.videoplane content @colormode uyvy
>> @transform_reset 2;
>> #P newex 554 167 155 196617 jit.qt.movie @adapt 1 @unique 1;
>> #P newex 490 307 94 196617 jit.window content;
>> #P user jit.fpsgui 435 125 60 196617 0;
>> #P newex 417 348 101 196617 jit.gl.render content;
>> #P newex 417 97 66 196617 t b b b erase;
>> #P toggle 417 48 15 0;
>> #P newex 417 73 57 196617 qmetro 20;
>> #P user jit.pwindow 250 161 82 62 0 1 0 0 1 0;
>> #P message 90 197 29 196617 stop;
>> #P message 91 160 154 196617 write 25. animation normal 600;
>> #P newex 57 233 66 196617 jit.qt.record;
>> #P message 122 66 30 196617 read;
>> #P message 57 66 60 196617 framedump;
>> #P newex 57 123 190 196617 jit.qt.movie @adapt 1 @colormode uyvy;
>> #P comment 125 45 100 196617 1. read movie;
>> #P comment 21 45 100 196617 3. framedump;
>> #P fasten 4 0 2 0 127 99 62 99;
>> #P connect 3 0 2 0;
>> #P connect 2 0 5 0;
>> #P fasten 7 0 5 0 95 224 62 224;
>> #P fasten 6 0 5 0 96 181 62 181;
>> #P fasten 2 0 8 0 62 147 256 147;
>> #P connect 10 0 9 0;
>> #P connect 9 0 11 0;
>> #P connect 11 0 12 0;
>> #P fasten 11 3 12 0 476 160 422 160;
>> #P connect 11 1 13 0;
>> #P connect 17 0 15 0;
>> #P fasten 11 2 15 0 458 121 559 121;
>> #P connect 15 0 16 0;
>> #P pop;
>>
>>
>> Tyler Nitsch schrieb:
>> > Upon vades suggestion and searching the archives….
>> >
>> >
>> http://www.cycling74.com/forums/index.php?t=msg&goto=113246&rid=0&S=12672e78d4a9aad871b2693f9a7bbb5b#msg_113246
>>
>> >
>> > I tried implementing this conversion on the gpu. The problem is I
>> get funny colored purpley/greeny/red distorted lines on the output
>> window… and when I try to use either dx | qt.grab the screen is
>> distorted.
>> >
>> > The good news is my fps is WAY BETTER!!! without using @colormode
>> uyvy on the qt.movie I can only get 100fps on the patch I’m posting
>> whereas with the shader doing the conversion I get up to 160 fps
>> !!???!! So weird. Any help would greatly be appreciated.
>> >
>> > I’m using windows XP PRO on an AMD s939 4800X2 with the latest max
>> and jitter.and a Viper ATI Radeon HD 3870 gpu.
>> >
>> > #P window setfont "Sans Serif" 9.;
>> > #P number 418 72 35 9 1 3 3 139 0 0 0 221 221 221 222 222 222 0 0 0;
>> > #P window linecount 1;
>> > #P newex 403 193 214 9109513 jit.qt.movie 720 480 @unique 1
>> @colormode argb;
>> > #P newex 403 228 77 9109513 jit.gl.slab window;
>> > #P user jit.fpsgui 155 341 60 9109513 0;
>> > #P message 621 156 30 9109513 close;
>> > #P newex 623 121 57 9109513 select 1 2 3;
>> > #P message 652 156 28 9109513 open;
>> > #P newex 388 154 42 9109513 gate 3 1;
>> > #P newex 623 193 267 9109513 jit.dx.grab 720 480 @vdevice 0 @unique
>> 1 @colormode uyvy;
>> > #P message 80 398 62 9109513 fullscreen $1;
>> > #P toggle 80 380 15 0;
>> > #P newex 58 330 40 9109513 key;
>> > #P newex 80 357 46 9109513 select 27;
>> > #P newex 141 227 250 9109513 jit.gl.slab window @file
>> cc.uyvy2rgba.jxs @dimscale 2. 1.;
>> > #P newex 308 30 48 9109513 loadbang;
>> > #P newex 139 84 30 9109513 t b b;
>> > #P toggle 138 40 15 0;
>> > #P newex 138 64 45 9109513 qmetro 5;
>> > #P newex 75 121 111 9109513 t b b erase;
>> > #P newex 193 308 192 9109513 jit.gl.videoplane window
>> @transform_reset 2;
>> > #P newex 142 188 219 9109513 jit.qt.movie 720 480 @unique 1
>> @colormode uyvy;
>> > #P message 310 117 76 9109513 read bball.mov;
>> > #P newex 69 426 199 9109513 jit.window window @depthbuffer 1
>> @floating 1;
>> > #P newex 75 299 86 9109513 jit.gl.render window;
>> > #P connect 18 2 17 0;
>> > #P connect 16 2 15 0;
>> > #P connect 19 0 15 0;
>> > #P connect 17 0 15 0;
>> > #P connect 23 0 16 0;
>> > #P connect 23 0 18 0;
>> > #P connect 18 1 19 0;
>> > #P connect 18 0 19 0;
>> > #P connect 5 1 16 1;
>> > #P connect 22 0 21 0;
>> > #P connect 16 1 22 0;
>> > #P fasten 2 0 3 0 315 144 147 144;
>> > #P connect 2 0 22 0;
>> > #P connect 9 0 2 0;
>> > #P connect 10 0 4 0;
>> > #P connect 21 0 4 0;
>> > #P connect 0 0 20 0;
>> > #P connect 16 0 3 0;
>> > #P fasten 3 0 10 0 147 218 146 218;
>> > #P fasten 15 0 10 0 628 218 146 218;
>> > #P connect 6 0 8 0;
>> > #P connect 7 0 6 0;
>> > #P connect 13 0 14 0;
>> > #P connect 11 0 13 0;
>> > #P connect 12 0 11 0;
>> > #P connect 5 0 0 0;
>> > #P connect 5 2 0 0;
>> > #P fasten 8 0 5 0 144 108 80 108;
>> > #P connect 14 0 1 0;
>> > #P window clipboard copycount 24;
>> >
>> >
>> >
>>
>>
>
>


January 2, 2008 | 4:36 pm

Hm. Unless I am mistaken, UYVY does absolutely nothing with the alpha
channel what so ever. That information is lost to the ether unless you
use the alphaglue shader and read in the alpha via other means. UYVY
has no alpha channel, and I dont think that jit.qt.movie premulitplies
alpha for you while doing the conversion?

From my understanding of the UYVY shaders and colormode is:

*Some video codecs use an internal format that is something like ARGB
(or RGBA…), or YUV or some variant (Y, cr’ cb’ or whatnot)

*jit.qt.movie when left alone, will always do conversion to JITTERS
basic ARGB matrix format.

This means that any video whose codecs native internal pixel format is
YUV or some other needs to be converted to ARGB by either quicktime or
ji.qt.movie (I am unsure which internal mechanism does this). If the
pixel format is something like RGBA or ARGB, nothing really needs to
be done.

jit.qt.movie @colormode uyvy does the reverse of this, which means any
quicktime movie whose normal internal pixel format is YUV (or
similarly compatible) gets left alone, otherwise it is converted to
UYVY.

This means that for codecs which normally use YUV or whatnot
colorspace internally, using @colormode UYVY is a fastpath – as there
is no conversion overhead.

This also means that ARGB codecs (or anything else), using @colormode
UYVY is NOT a fastpath, as conversion must happen (somewhere – again I
do not know specifically where this happens).

Id love to know from Cycling if this is correct.

*feature request below!*

Ive been thinking about this myself, and it might be nice if there was
an attribute @colorspace native

which would also dumpout the colorspace on the rightmost outlet
(dumpout outlet) right before the movie plays its first frame, and
load the movie natively.

This would allow you to :

Have a fastpath GPU uploading solution that would retain alpha channel
should your quicktime movie have one, or skip right to UYVY those that
need it.

decide what to do with the alpha channel should you want to keep it.

* As far as the errors go, have you tried the other various uyvy
conversion shaders, the lite, and the one with gamma correction?

As far as animation is concerned I belive it is

ARGB and uses only run length encoding, which is basically like
uncompressed, and has not inter frame coding. Its like motion jpeg/
photo jpeg, just ARGB and lossless compression.

On Jan 2, 2008, at 7:09 AM, John Dekron wrote:

> Hi _n,
> I get the digital artifacts as well and have no idea where they come
> from.
> Animation and png are videocodecs that hold an alpha channel what is
> necessary for the uyvy information. If I remember correctly it is
> possible to limit the data rate on png compression. I did use png
> for a previous project, but lately I experienced some crashes when
> playing back png compressed movies. i switched to animation and it
> worked fine. Sorry that I didn’t investigate deeper on the crashes
> that I had, so this information might as well be treated as gossip-
> but eventually there is some truth in it.
>
> good luck. John.
>
> (())_n schrieb:
>> yes dekron, yair,
>> I have been looking into this method of doing the conversion
>> before, and it does help, doubling the frame rate of the display
>> patch and cutting down on cpu/gpu load.
>> Problem is, it seems only some codecs work with this method.
>> Animation which you use I guess compresses over time, because when
>> I try to display particular frames I get digital artifacts. For
>> Jpeg, motion-jpeg the colors aren’t right when the film is loading
>> back in. I had success with PNG codec, but the file sizes after the
>> compression are quite large.
>> Any other codecs you could recommend that compress each frame
>> separately and work with this method of creating "quasi" uyvy to be
>> decoded as such on the GPU?
>> To get Photo-Jpeg/Motion-Jpeg to work I tried to map the planes
>> differently, but couldn’t get the colors to match. Anyone throw
>> some light on this? Is there a particular website that might
>> describe all the different standard quicktime codecs? Detailing
>> which compress every frame separately and which compress over time?
>> Jitter does better with the former.
>> cheers
>> (())_n
>> On Dec 6, 2007, at 4:31 AM, yair reshef wrote:
>>> thank you mr. Dekron, your pre uyvy method has gotten me a 768×768
>>> movie run a lot nicer on an xp.
>>>
>>> btw. is there a method to force other post enviroments
>>> (aftereffects, shake etc.) to export to uyvy.
>>> most of the post friends i asked looked at me funny.
>>>
>>>
>>>
>>> On Nov 23, 2007 11:31 AM, John Dekron

wrote:
>>> Hi Tyler,
>>> I don’t know about the funny colors you see, but on the fps side, i
>>> understand that jit.qt.movie is doing the conversion from argb to
>>> uyvy
>>> internally. on some bigger movies I get even better performance
>>> when I
>>> transform the source movie to a half sized "quasi" uyvy movie
>>> beforehand. (see attahced patch). I only see some "trailing" on that
>>> kind of material. this is on MacBook Pro, OSX 10.4.10, Quicktime 7.2
>>>
>>> John.
>>>
>>> max v2;
>>> #N vpatcher 461 240 1347 803;
>>> #P window setfont "Sans Serif" 9.;
>>> #P window linecount 1;
>>> #P comment 638 116 100 196617 5. read new movie;
>>> #P comment 119 209 100 196617 4. stop;
>>> #P window linecount 2;
>>> #P comment 116 179 100 196617 2. select write location;
>>> #P window linecount 1;
>>> #P message 597 119 30 196617 read;
>>> #P newex 554 193 297 196617 jit.gl.videoplane content @colormode
>>> uyvy
>>> @transform_reset 2;
>>> #P newex 554 167 155 196617 jit.qt.movie @adapt 1 @unique 1;
>>> #P newex 490 307 94 196617 jit.window content;
>>> #P user jit.fpsgui 435 125 60 196617 0;
>>> #P newex 417 348 101 196617 jit.gl.render content;
>>> #P newex 417 97 66 196617 t b b b erase;
>>> #P toggle 417 48 15 0;
>>> #P newex 417 73 57 196617 qmetro 20;
>>> #P user jit.pwindow 250 161 82 62 0 1 0 0 1 0;
>>> #P message 90 197 29 196617 stop;
>>> #P message 91 160 154 196617 write 25. animation normal 600;
>>> #P newex 57 233 66 196617 jit.qt.record;
>>> #P message 122 66 30 196617 read;
>>> #P message 57 66 60 196617 framedump;
>>> #P newex 57 123 190 196617 jit.qt.movie @adapt 1 @colormode uyvy;
>>> #P comment 125 45 100 196617 1. read movie;
>>> #P comment 21 45 100 196617 3. framedump;
>>> #P fasten 4 0 2 0 127 99 62 99;
>>> #P connect 3 0 2 0;
>>> #P connect 2 0 5 0;
>>> #P fasten 7 0 5 0 95 224 62 224;
>>> #P fasten 6 0 5 0 96 181 62 181;
>>> #P fasten 2 0 8 0 62 147 256 147;
>>> #P connect 10 0 9 0;
>>> #P connect 9 0 11 0;
>>> #P connect 11 0 12 0;
>>> #P fasten 11 3 12 0 476 160 422 160;
>>> #P connect 11 1 13 0;
>>> #P connect 17 0 15 0;
>>> #P fasten 11 2 15 0 458 121 559 121;
>>> #P connect 15 0 16 0;
>>> #P pop;
>>>
>>>
>>> Tyler Nitsch schrieb:
>>> > Upon vades suggestion and searching the archives….
>>> >
>>> > http://www.cycling74.com/forums/index.php?t=msg&goto=113246&rid=0&S=12672e78d4a9aad871b2693f9a7bbb5b#msg_113246
>>> >
>>> > I tried implementing this conversion on the gpu. The problem is
>>> I get funny colored purpley/greeny/red distorted lines on the
>>> output window… and when I try to use either dx | qt.grab the
>>> screen is distorted.
>>> >
>>> > The good news is my fps is WAY BETTER!!! without using
>>> @colormode uyvy on the qt.movie I can only get 100fps on the patch
>>> I’m posting whereas with the shader doing the conversion I get up
>>> to 160 fps !!???!! So weird. Any help would greatly be
>>> appreciated.
>>> >
>>> > I’m using windows XP PRO on an AMD s939 4800X2 with the latest
>>> max and jitter.and a Viper ATI Radeon HD 3870 gpu.
>>> >
>>> > #P window setfont "Sans Serif" 9.;
>>> > #P number 418 72 35 9 1 3 3 139 0 0 0 221 221 221 222 222 222 0
>>> 0 0;
>>> > #P window linecount 1;
>>> > #P newex 403 193 214 9109513 jit.qt.movie 720 480 @unique 1
>>> @colormode argb;
>>> > #P newex 403 228 77 9109513 jit.gl.slab window;
>>> > #P user jit.fpsgui 155 341 60 9109513 0;
>>> > #P message 621 156 30 9109513 close;
>>> > #P newex 623 121 57 9109513 select 1 2 3;
>>> > #P message 652 156 28 9109513 open;
>>> > #P newex 388 154 42 9109513 gate 3 1;
>>> > #P newex 623 193 267 9109513 jit.dx.grab 720 480 @vdevice 0
>>> @unique 1 @colormode uyvy;
>>> > #P message 80 398 62 9109513 fullscreen $1;
>>> > #P toggle 80 380 15 0;
>>> > #P newex 58 330 40 9109513 key;
>>> > #P newex 80 357 46 9109513 select 27;
>>> > #P newex 141 227 250 9109513 jit.gl.slab window @file
>>> cc.uyvy2rgba.jxs @dimscale 2. 1.;
>>> > #P newex 308 30 48 9109513 loadbang;
>>> > #P newex 139 84 30 9109513 t b b;
>>> > #P toggle 138 40 15 0;
>>> > #P newex 138 64 45 9109513 qmetro 5;
>>> > #P newex 75 121 111 9109513 t b b erase;
>>> > #P newex 193 308 192 9109513 jit.gl.videoplane window
>>> @transform_reset 2;
>>> > #P newex 142 188 219 9109513 jit.qt.movie 720 480 @unique 1
>>> @colormode uyvy;
>>> > #P message 310 117 76 9109513 read bball.mov;
>>> > #P newex 69 426 199 9109513 jit.window window @depthbuffer 1
>>> @floating 1;
>>> > #P newex 75 299 86 9109513 jit.gl.render window;
>>> > #P connect 18 2 17 0;
>>> > #P connect 16 2 15 0;
>>> > #P connect 19 0 15 0;
>>> > #P connect 17 0 15 0;
>>> > #P connect 23 0 16 0;
>>> > #P connect 23 0 18 0;
>>> > #P connect 18 1 19 0;
>>> > #P connect 18 0 19 0;
>>> > #P connect 5 1 16 1;
>>> > #P connect 22 0 21 0;
>>> > #P connect 16 1 22 0;
>>> > #P fasten 2 0 3 0 315 144 147 144;
>>> > #P connect 2 0 22 0;
>>> > #P connect 9 0 2 0;
>>> > #P connect 10 0 4 0;
>>> > #P connect 21 0 4 0;
>>> > #P connect 0 0 20 0;
>>> > #P connect 16 0 3 0;
>>> > #P fasten 3 0 10 0 147 218 146 218;
>>> > #P fasten 15 0 10 0 628 218 146 218;
>>> > #P connect 6 0 8 0;
>>> > #P connect 7 0 6 0;
>>> > #P connect 13 0 14 0;
>>> > #P connect 11 0 13 0;
>>> > #P connect 12 0 11 0;
>>> > #P connect 5 0 0 0;
>>> > #P connect 5 2 0 0;
>>> > #P fasten 8 0 5 0 144 108 80 108;
>>> > #P connect 14 0 1 0;
>>> > #P window clipboard copycount 24;
>>> >
>>> >
>>> >
>>>
>>>
>


January 2, 2008 | 5:17 pm

Great Vade !


January 3, 2008 | 9:16 am

HI Vade,

vade schrieb:
> Hm. Unless I am mistaken, UYVY does absolutely nothing with the alpha
> channel what so ever. That information is lost to the ether unless you
> use the alphaglue shader and read in the alpha via other means. UYVY has
> no alpha channel, and I dont think that jit.qt.movie premulitplies alpha
> for you while doing the conversion?
That is right. I was referring to the method where I store an uyvy movie
in the argb animation codec. the alpha channel stores the u component.

What is the "official" codec for uyvy movies?

John.
>
> From my understanding of the UYVY shaders and colormode is:
>
> *Some video codecs use an internal format that is something like ARGB
> (or RGBA…), or YUV or some variant (Y, cr’ cb’ or whatnot)
>
> *jit.qt.movie when left alone, will always do conversion to JITTERS
> basic ARGB matrix format.
>
> This means that any video whose codecs native internal pixel format is
> YUV or some other needs to be converted to ARGB by either quicktime or
> ji.qt.movie (I am unsure which internal mechanism does this). If the
> pixel format is something like RGBA or ARGB, nothing really needs to be
> done.
>
> jit.qt.movie @colormode uyvy does the reverse of this, which means any
> quicktime movie whose normal internal pixel format is YUV (or similarly
> compatible) gets left alone, otherwise it is converted to UYVY.
>
> This means that for codecs which normally use YUV or whatnot colorspace
> internally, using @colormode UYVY is a fastpath – as there is no
> conversion overhead.
>
> This also means that ARGB codecs (or anything else), using @colormode
> UYVY is NOT a fastpath, as conversion must happen (somewhere – again I
> do not know specifically where this happens).
>
> Id love to know from Cycling if this is correct.
>
> *feature request below!*
>
> Ive been thinking about this myself, and it might be nice if there was
> an attribute @colorspace native
>
> which would also dumpout the colorspace on the rightmost outlet (dumpout
> outlet) right before the movie plays its first frame, and load the
> movie natively.
>
> This would allow you to :
>
> Have a fastpath GPU uploading solution that would retain alpha channel
> should your quicktime movie have one, or skip right to UYVY those that
> need it.
>
> decide what to do with the alpha channel should you want to keep it.
>
>
> * As far as the errors go, have you tried the other various uyvy
> conversion shaders, the lite, and the one with gamma correction?
>
> As far as animation is concerned I belive it is
>
> ARGB and uses only run length encoding, which is basically like
> uncompressed, and has not inter frame coding. Its like motion jpeg/photo
> jpeg, just ARGB and lossless compression.
>
>
> On Jan 2, 2008, at 7:09 AM, John Dekron wrote:
>
>> Hi _n,
>> I get the digital artifacts as well and have no idea where they come
>> from.
>> Animation and png are videocodecs that hold an alpha channel what is
>> necessary for the uyvy information. If I remember correctly it is
>> possible to limit the data rate on png compression. I did use png for
>> a previous project, but lately I experienced some crashes when playing
>> back png compressed movies. i switched to animation and it worked
>> fine. Sorry that I didn’t investigate deeper on the crashes that I
>> had, so this information might as well be treated as gossip- but
>> eventually there is some truth in it.
>>
>> good luck. John.
>>
>> (())_n schrieb:
>>> yes dekron, yair,
>>> I have been looking into this method of doing the conversion before,
>>> and it does help, doubling the frame rate of the display patch and
>>> cutting down on cpu/gpu load.
>>> Problem is, it seems only some codecs work with this method.
>>> Animation which you use I guess compresses over time, because when I
>>> try to display particular frames I get digital artifacts. For Jpeg,
>>> motion-jpeg the colors aren’t right when the film is loading back in.
>>> I had success with PNG codec, but the file sizes after the
>>> compression are quite large.
>>> Any other codecs you could recommend that compress each frame
>>> separately and work with this method of creating "quasi" uyvy to be
>>> decoded as such on the GPU?
>>> To get Photo-Jpeg/Motion-Jpeg to work I tried to map the planes
>>> differently, but couldn’t get the colors to match. Anyone throw some
>>> light on this? Is there a particular website that might describe all
>>> the different standard quicktime codecs? Detailing which compress
>>> every frame separately and which compress over time? Jitter does
>>> better with the former.
>>> cheers
>>> (())_n
>>> On Dec 6, 2007, at 4:31 AM, yair reshef wrote:
>>>> thank you mr. Dekron, your pre uyvy method has gotten me a 768×768
>>>> movie run a lot nicer on an xp.
>>>>
>>>> btw. is there a method to force other post enviroments
>>>> (aftereffects, shake etc.) to export to uyvy.
>>>> most of the post friends i asked looked at me funny.
>>>>
>>>>
>>>>
>>>> On Nov 23, 2007 11:31 AM, John Dekron

wrote:
>>>> Hi Tyler,
>>>> I don’t know about the funny colors you see, but on the fps side, i
>>>> understand that jit.qt.movie is doing the conversion from argb to uyvy
>>>> internally. on some bigger movies I get even better performance when I
>>>> transform the source movie to a half sized "quasi" uyvy movie
>>>> beforehand. (see attahced patch). I only see some "trailing" on that
>>>> kind of material. this is on MacBook Pro, OSX 10.4.10, Quicktime 7.2
>>>>
>>>> John.
>>>>
>>>> max v2;
>>>> #N vpatcher 461 240 1347 803;
>>>> #P window setfont "Sans Serif" 9.;
>>>> #P window linecount 1;
>>>> #P comment 638 116 100 196617 5. read new movie;
>>>> #P comment 119 209 100 196617 4. stop;
>>>> #P window linecount 2;
>>>> #P comment 116 179 100 196617 2. select write location;
>>>> #P window linecount 1;
>>>> #P message 597 119 30 196617 read;
>>>> #P newex 554 193 297 196617 jit.gl.videoplane content @colormode uyvy
>>>> @transform_reset 2;
>>>> #P newex 554 167 155 196617 jit.qt.movie @adapt 1 @unique 1;
>>>> #P newex 490 307 94 196617 jit.window content;
>>>> #P user jit.fpsgui 435 125 60 196617 0;
>>>> #P newex 417 348 101 196617 jit.gl.render content;
>>>> #P newex 417 97 66 196617 t b b b erase;
>>>> #P toggle 417 48 15 0;
>>>> #P newex 417 73 57 196617 qmetro 20;
>>>> #P user jit.pwindow 250 161 82 62 0 1 0 0 1 0;
>>>> #P message 90 197 29 196617 stop;
>>>> #P message 91 160 154 196617 write 25. animation normal 600;
>>>> #P newex 57 233 66 196617 jit.qt.record;
>>>> #P message 122 66 30 196617 read;
>>>> #P message 57 66 60 196617 framedump;
>>>> #P newex 57 123 190 196617 jit.qt.movie @adapt 1 @colormode uyvy;
>>>> #P comment 125 45 100 196617 1. read movie;
>>>> #P comment 21 45 100 196617 3. framedump;
>>>> #P fasten 4 0 2 0 127 99 62 99;
>>>> #P connect 3 0 2 0;
>>>> #P connect 2 0 5 0;
>>>> #P fasten 7 0 5 0 95 224 62 224;
>>>> #P fasten 6 0 5 0 96 181 62 181;
>>>> #P fasten 2 0 8 0 62 147 256 147;
>>>> #P connect 10 0 9 0;
>>>> #P connect 9 0 11 0;
>>>> #P connect 11 0 12 0;
>>>> #P fasten 11 3 12 0 476 160 422 160;
>>>> #P connect 11 1 13 0;
>>>> #P connect 17 0 15 0;
>>>> #P fasten 11 2 15 0 458 121 559 121;
>>>> #P connect 15 0 16 0;
>>>> #P pop;
>>>>
>>>>
>>>> Tyler Nitsch schrieb:
>>>> > Upon vades suggestion and searching the archives….
>>>> >
>>>> >
>>>> http://www.cycling74.com/forums/index.php?t=msg&goto=113246&rid=0&S=12672e78d4a9aad871b2693f9a7bbb5b#msg_113246
>>>>
>>>> >
>>>> > I tried implementing this conversion on the gpu. The problem is I
>>>> get funny colored purpley/greeny/red distorted lines on the output
>>>> window… and when I try to use either dx | qt.grab the screen is
>>>> distorted.
>>>> >
>>>> > The good news is my fps is WAY BETTER!!! without using @colormode
>>>> uyvy on the qt.movie I can only get 100fps on the patch I’m posting
>>>> whereas with the shader doing the conversion I get up to 160 fps
>>>> !!???!! So weird. Any help would greatly be appreciated.
>>>> >
>>>> > I’m using windows XP PRO on an AMD s939 4800X2 with the latest max
>>>> and jitter.and a Viper ATI Radeon HD 3870 gpu.
>>>> >
>>>> > #P window setfont "Sans Serif" 9.;
>>>> > #P number 418 72 35 9 1 3 3 139 0 0 0 221 221 221 222 222 222 0 0 0;
>>>> > #P window linecount 1;
>>>> > #P newex 403 193 214 9109513 jit.qt.movie 720 480 @unique 1
>>>> @colormode argb;
>>>> > #P newex 403 228 77 9109513 jit.gl.slab window;
>>>> > #P user jit.fpsgui 155 341 60 9109513 0;
>>>> > #P message 621 156 30 9109513 close;
>>>> > #P newex 623 121 57 9109513 select 1 2 3;
>>>> > #P message 652 156 28 9109513 open;
>>>> > #P newex 388 154 42 9109513 gate 3 1;
>>>> > #P newex 623 193 267 9109513 jit.dx.grab 720 480 @vdevice 0
>>>> @unique 1 @colormode uyvy;
>>>> > #P message 80 398 62 9109513 fullscreen $1;
>>>> > #P toggle 80 380 15 0;
>>>> > #P newex 58 330 40 9109513 key;
>>>> > #P newex 80 357 46 9109513 select 27;
>>>> > #P newex 141 227 250 9109513 jit.gl.slab window @file
>>>> cc.uyvy2rgba.jxs @dimscale 2. 1.;
>>>> > #P newex 308 30 48 9109513 loadbang;
>>>> > #P newex 139 84 30 9109513 t b b;
>>>> > #P toggle 138 40 15 0;
>>>> > #P newex 138 64 45 9109513 qmetro 5;
>>>> > #P newex 75 121 111 9109513 t b b erase;
>>>> > #P newex 193 308 192 9109513 jit.gl.videoplane window
>>>> @transform_reset 2;
>>>> > #P newex 142 188 219 9109513 jit.qt.movie 720 480 @unique 1
>>>> @colormode uyvy;
>>>> > #P message 310 117 76 9109513 read bball.mov;
>>>> > #P newex 69 426 199 9109513 jit.window window @depthbuffer 1
>>>> @floating 1;
>>>> > #P newex 75 299 86 9109513 jit.gl.render window;
>>>> > #P connect 18 2 17 0;
>>>> > #P connect 16 2 15 0;
>>>> > #P connect 19 0 15 0;
>>>> > #P connect 17 0 15 0;
>>>> > #P connect 23 0 16 0;
>>>> > #P connect 23 0 18 0;
>>>> > #P connect 18 1 19 0;
>>>> > #P connect 18 0 19 0;
>>>> > #P connect 5 1 16 1;
>>>> > #P connect 22 0 21 0;
>>>> > #P connect 16 1 22 0;
>>>> > #P fasten 2 0 3 0 315 144 147 144;
>>>> > #P connect 2 0 22 0;
>>>> > #P connect 9 0 2 0;
>>>> > #P connect 10 0 4 0;
>>>> > #P connect 21 0 4 0;
>>>> > #P connect 0 0 20 0;
>>>> > #P connect 16 0 3 0;
>>>> > #P fasten 3 0 10 0 147 218 146 218;
>>>> > #P fasten 15 0 10 0 628 218 146 218;
>>>> > #P connect 6 0 8 0;
>>>> > #P connect 7 0 6 0;
>>>> > #P connect 13 0 14 0;
>>>> > #P connect 11 0 13 0;
>>>> > #P connect 12 0 11 0;
>>>> > #P connect 5 0 0 0;
>>>> > #P connect 5 2 0 0;
>>>> > #P fasten 8 0 5 0 144 108 80 108;
>>>> > #P connect 14 0 1 0;
>>>> > #P window clipboard copycount 24;
>>>> >
>>>> >
>>>> >
>>>>
>>>>
>>
>
>


January 3, 2008 | 9:47 am

Hm, I must be missing something? What do you mean "store" a uyvy
"within" argb? UYVY has 3 planes to ARGBs 4? Youre going to loose
something somewhere.. unless you are being really tricky! (do share!)

Official UYVY codec? dont know, but any YUV codec should do, from my
understanding. Uncompressed 10 and 8 bit 4:2:2 YUV from Apple, MJpeg/
PhotoJpeg , DV, DVCPro, DVCProHD and so on

http://www.fourcc.org/yuv.php

On Jan 3, 2008, at 4:16 AM, John Dekron wrote:

> HI Vade,
>
> vade schrieb:
>> Hm. Unless I am mistaken, UYVY does absolutely nothing with the
>> alpha channel what so ever. That information is lost to the ether
>> unless you use the alphaglue shader and read in the alpha via other
>> means. UYVY has no alpha channel, and I dont think that
>> jit.qt.movie premulitplies alpha for you while doing the conversion?
> That is right. I was referring to the method where I store an uyvy
> movie in the argb animation codec. the alpha channel stores the u
> component.
>


January 3, 2008 | 1:25 pm

I posted the patch earlier in this thread:

http://www.cycling74.com/forums/index.php?t=msg&goto=121838&rid=0&S=d8d9e7b93c064870c613751e65b06833&srch=cc.uyvy2rgba.jxs+and+optimization#msg_121845

have a look at it. it explains pretty good what I am doing.

Photo Jpeg is a YUV Codec? and I was was convinced it stores RGB Data. I
will have to do my homework on that.

But if this is so: a Photojpeg file with dimensions x, y, decompressed
with jit.qt.movie @colormode uyvy decompresses way slower than a x/2, y
sized argb movie with jit.qt.movie @colormode argb. Why is this so.
(again, have a look at the patch to see what I mean).

John.

vade schrieb:
> Hm, I must be missing something? What do you mean "store" a uyvy
> "within" argb? UYVY has 3 planes to ARGBs 4? Youre going to loose
> something somewhere.. unless you are being really tricky! (do share!)
>
> Official UYVY codec? dont know, but any YUV codec should do, from my
> understanding. Uncompressed 10 and 8 bit 4:2:2 YUV from Apple,
> MJpeg/PhotoJpeg , DV, DVCPro, DVCProHD and so on
>
> http://www.fourcc.org/yuv.php
>
> On Jan 3, 2008, at 4:16 AM, John Dekron wrote:
>
>> HI Vade,
>>
>> vade schrieb:
>>> Hm. Unless I am mistaken, UYVY does absolutely nothing with the alpha
>>> channel what so ever. That information is lost to the ether unless
>>> you use the alphaglue shader and read in the alpha via other means.
>>> UYVY has no alpha channel, and I dont think that jit.qt.movie
>>> premulitplies alpha for you while doing the conversion?
>> That is right. I was referring to the method where I store an uyvy
>> movie in the argb animation codec. the alpha channel stores the u
>> component.
>>
>
>


January 3, 2008 | 4:40 pm

Im 90% sure. In fact, FOURCCs YUV to ARGB conversion guide states:

Avery Lee’s JFIF Clarification
"The equations you have from "Video Demystified" are the ones to use
most of the time on PCs, such as for MPEG-1 decoding and handing most
(all?) of the formats on the YUV FOURCC page. However, there is one
notable exception: JPEG. The JFIF file format that is commonly used
with JPEG defines a YCbCr color space that is slightly different than
usual, in that all components have the full [0,255] excursion rather
than [16,235] and [16,240]. The equations given by "Julien" are for
this variant and are very close to the ones in JFIF:

R = Y + 1.402 (Cr-128)

G = Y – 0.34414 (Cb-128) – 0.71414 (Cr-128)

B = Y + 1.772 (Cb-128)

It should be noted that some Motion JPEG codecs don’t take this
consideration into account and improperly exchange YCbCr values
between the YUY2 and UYVY formats and their internal JPEG planes
without performing the necessary scaling. This results in contrast
shifts when using YUV formats."

Im not sure if id take that as gospel, (or, um, you know what I mean),
I did a quick google search and while I did not see any giant blinking
times square neon sign stating mjpeg/photojpeg video codecs are YUV, I
saw lots of inferences to that point.

I’ll check your patch out in a few. Thanks.

these sorts of discussions are great.

On Jan 3, 2008, at 8:25 AM, John Dekron wrote:

> Photo Jpeg is a YUV Codec? and I was was convinced it stores RGB
> Data. I will have to do my homework on that.


January 6, 2008 | 8:58 pm

hey vade, jkc,

Sorry kind of been preoccupied with other stuff. In reference to the errors I was having with the cc.uyvy2rgba.jxs shader on my computer. My gpu has had a couple of new drivers made so I thought to revisit this issue. Still problems with the cc.uyvy2rgba.jxs shader but ya the cc.uyvy2rgba.lite.jxs program works.


January 7, 2008 | 5:18 pm

On Jan 6, 2008, at 12:58 PM, Tyler Nitsch wrote:

> Sorry kind of been preoccupied with other stuff. In reference to
> the errors I was having with the cc.uyvy2rgba.jxs shader on my
> computer. My gpu has had a couple of new drivers made so I thought
> to revisit this issue. Still problems with the cc.uyvy2rgba.jxs
> shader but ya the cc.uyvy2rgba.lite.jxs program works.

Thanks for the update. Sounds like it might be related to chroma
smoothing assumptions then. You might try sending the message "param
chromasmooth 0". Let us know if that works and we can try to figure
out what the issue might be.

Thanks,
Joshua


January 7, 2008 | 8:13 pm

Might be be the code in the shader as well? I looked at the
chromasmoothing portion of the code and noticed

yuv_val.zy =
mix(yvyu_left.yw,yuv_val.zy,vec2(coord_frak.x>=0.&&chromasmooth>0.5));

might the && operation cause issues on older cards? Im asking, as I
simply dont know :)

Also, speaking of, how on earth does one reasonably derive supported
options based on hardware/driver configurations?

Ive attempted to determine this occasionally and frankly it seems
harder to find this out than just try different codepaths.

On Jan 7, 2008, at 12:18 PM, Joshua Kit Clayton wrote:

>
> On Jan 6, 2008, at 12:58 PM, Tyler Nitsch wrote:
>
>> Sorry kind of been preoccupied with other stuff. In reference to
>> the errors I was having with the cc.uyvy2rgba.jxs shader on my
>> computer. My gpu has had a couple of new drivers made so I thought
>> to revisit this issue. Still problems with the cc.uyvy2rgba.jxs
>> shader but ya the cc.uyvy2rgba.lite.jxs program works.
>
> Thanks for the update. Sounds like it might be related to chroma
> smoothing assumptions then. You might try sending the message "param
> chromasmooth 0". Let us know if that works and we can try to figure
> out what the issue might be.
>
> Thanks,


January 7, 2008 | 11:51 pm

hey Joshua looks like the chromasmooth argument is the problem. Probably something to do with the line of code vade posted since when the value is greater than 0.5 the distortion occurs. The graphics card is ATI’s latest(Radeon HD 3870)… I wonder what they changed. Anywho I hope this helps. Sorry for not digging into parameter values initially… will do with future problems with shader programs.


January 8, 2008 | 12:09 am

On Jan 7, 2008, at 3:51 PM, Tyler Nitsch wrote:

>
> hey Joshua looks like the chromasmooth argument is the problem.
> Probably something to do with the line of code vade posted since
> when the value is greater than 0.5 the distortion occurs. The
> graphics card is ATI’s latest(Radeon HD 3870)… I wonder what they
> changed. Anywho I hope this helps. Sorry for not digging into
> parameter values initially… will do with future problems with
> shader programs.

Might just be some FP round off errors or some difference with the
way the texture coordinates are being generated (center to center vs.
corner to corner etc.). I’d try something like changing
"coord_frak.x>=0." to "coord_frak.x>0." or "coord_frak.x>=0.5".

I don’t have time at the moment to fiddle with this, but if you have
a chance and discover something that works for your card and others,
please let us know.

Thanks,
Joshua


Viewing 25 posts - 1 through 25 (of 25 total)