Cycling 74 » Topic: slab compositing and colormodes http://cycling74.com/forums/topic/slab-compositing-and-colormodes/feed Thu, 20 Jun 2013 05:05:08 +0000 http://bbpress.org/?v=2.2.4 http://cycling74.com/forums/topic/slab-compositing-and-colormodes/#post-52756 <![CDATA[slab compositing and colormodes]]> http://cycling74.com/forums/topic/slab-compositing-and-colormodes/#post-52756 Thu, 14 Oct 2010 21:41:43 +0000 mbiederman Hi, hoping someone can help out here. I am having some issue with slab compositing. The patch below illustrates my issue – some composite .jxs work fine (such as average and normal and others) while others like additive, subtractive and others, don’t. On my machine it ends up looking like a colormode issue, but I cant seem to get it right. It will work with both sources as ARGB, but not with both sources as UYVY.

I’ve distilled a much larger patch down to what you see below. Im running a MacBookPro i7 / OS 10.6.4 / NVIDIA330M 512

– Pasted Max Patch, click to expand. –
]]> http://cycling74.com/forums/topic/slab-compositing-and-colormodes/#post-189605 <![CDATA[Re: slab compositing and colormodes]]> http://cycling74.com/forums/topic/slab-compositing-and-colormodes/#post-189605 Fri, 15 Oct 2010 07:43:27 +0000 Yoann
– Pasted Max Patch, click to expand. –

]]> http://cycling74.com/forums/topic/slab-compositing-and-colormodes/#post-189606 <![CDATA[Re: slab compositing and colormodes]]> http://cycling74.com/forums/topic/slab-compositing-and-colormodes/#post-189606 Fri, 15 Oct 2010 15:18:09 +0000 mbiederman well, yes that works OK, but I loose half the resolution (vertical) oversampling by setting the dim@ 1440 480 does the trick, but it works the cpu too hard… – I’ve reworded my issue and posted it here:

http://cycling74.com/forums/topic.php?id=28971

I need to maintain full quality, yet Id like to use the uyvy colormode and Im not sure why the slab doesn’t seem to work correctly….the jit.colormode works, but it is pretty heavy too, i was trying to use the GPU version of course…

-matthew

]]> http://cycling74.com/forums/topic/slab-compositing-and-colormodes/#post-189607 <![CDATA[Re: slab compositing and colormodes]]> http://cycling74.com/forums/topic/slab-compositing-and-colormodes/#post-189607 Fri, 15 Oct 2010 20:26:02 +0000 Joshua Kit Clayton I think there is some misunderstanding about colormodes here. colormode UYVY primarily exists as a way to get data quickly to the GPU, where the colorspace conversion and upsampling to full resolution RGBA can be handled by the graphics card. Once on the GPU, *everything* is processed as RGBA. There is no concept of UYVY on the GPU or in any shader except for ones custom programmed to work with that sort of data *masquerading* as RGBA.

The cc.uyvy2rgba.jxs and cc.rgba2uyvy.jxs shaders are provided as ways to perform the conversion of UYVY data to RGBA and back for the transfer to and from the GPU.

The quick answer here, is that in all likelihood, you want to avoid using UYVY at any point beyond an initial move to the GPU. And at that point all processing on the GPU should be in RGBA.

OS X supports a UYVY texture type, which undergoes a shader based conversion to RGBA as it is loaded into the GPU, but it doesn’t do chroma smoothing or other color corrections, and Windows does not provide any. This is why cc.uyvy2rgba.jxs exists, and it should *always* be used with @dimscale 2 1 like the example jjit.gl.slab-uyvy-gpu.maxpat example, to convert from half chroma horizontal resolution to full horizontal resolution.

As for cc.rgba2uyvy.jxs, it is an example shader to be used for transfer back to the GPU in the half data UYVY format and should *always* used with the @dimscale 0.5 1 , to convert from full horizontal resolution to half chroma horizontal resolution. Chances are you will never need to use this.

I believe that Yoann’s post above is what you want to do. However, where Yoann’s post was incorrect, was in using cc.uyvy2rgba.jxs *without* @dimscale 2. 1. cc.uyvy2rgba.jxs is tricky (UYVY data masquerading as RGBA data), and requires this manual upsampling for the shader to work properly, otherwise there is still half horizontal resolution output. This is probably what you noticed.

Alternately, if you are on OS X, you can just use jit.gl.texture @colormode uyvy or jit.gl.slab @colormode uyvy for the very first time the data moves from CPU to GPU (and *nowhere else down the slab processing chain*), and you do not need to use cc.uyvy2rgba.jxs. However, for the chroma smoothing and some nice color correction along the way, I think cc.uyvy2rgba.jxs and cc.uyvy2rgba.exp.jxs provide better results than Apple’s default UYVY texture -> RGBA conversion. Play around with them in jit.gl.slab-uyvy-gpu.maxpat

I hope this helps. I guess we need to have a more detailed article on this subject.

-Joshua

]]> http://cycling74.com/forums/topic/slab-compositing-and-colormodes/#post-189608 <![CDATA[Re: slab compositing and colormodes]]> http://cycling74.com/forums/topic/slab-compositing-and-colormodes/#post-189608 Mon, 18 Oct 2010 19:11:37 +0000 mbiederman Huuuge thanks for the in depth explanation of this Joshua. Clears allot up for me, I was driving my self more than a little crazy!

]]> http://cycling74.com/forums/topic/slab-compositing-and-colormodes/#post-189609 <![CDATA[Re: slab compositing and colormodes]]> http://cycling74.com/forums/topic/slab-compositing-and-colormodes/#post-189609 Mon, 18 Oct 2010 19:45:42 +0000 JATom also a huge thanks to Joshua.

i have still one question to the last option using uyvy on mac osx.
if the matrix data is coming from an jit.qt.movie object.
do i have to set the @colormode uyvy at the jit.qt.movie object
to speed up or not ?

best,
tom

]]> http://cycling74.com/forums/topic/slab-compositing-and-colormodes/#post-189610 <![CDATA[Re: slab compositing and colormodes]]> http://cycling74.com/forums/topic/slab-compositing-and-colormodes/#post-189610 Mon, 18 Oct 2010 20:27:13 +0000 Joshua Kit Clayton Sure thing. Glad it helps. We’ll look at making the docs and or an article about this in the future.

@tom: Yes. jit.qt.movie must use @colormode uyvy in order to output UYVY data.

-Joshua

]]> http://cycling74.com/forums/topic/slab-compositing-and-colormodes/#post-189611 <![CDATA[Re: slab compositing and colormodes]]> http://cycling74.com/forums/topic/slab-compositing-and-colormodes/#post-189611 Wed, 20 Oct 2010 19:45:01 +0000 f_ilippo Merci Joshua
For this enlighting explanation
I was sending matrices from 3 webcams via jit.net.send in uyvy because I thought matrices were twice smaller than in RGBA ….
best.
Filippo

]]> http://cycling74.com/forums/topic/slab-compositing-and-colormodes/#post-189612 <![CDATA[Re: slab compositing and colormodes]]> http://cycling74.com/forums/topic/slab-compositing-and-colormodes/#post-189612 Fri, 22 Oct 2010 18:54:39 +0000 Joshua Kit Clayton Fillipo, what you describe is a perfectly appropriate reason to use UYVY as output from jit.qt.grab. It is 2x smaller when in RAM. Once on the GPU, things are expanded and processed as RGBA.

]]> http://cycling74.com/forums/topic/slab-compositing-and-colormodes/#post-189613 <![CDATA[Re: slab compositing and colormodes]]> http://cycling74.com/forums/topic/slab-compositing-and-colormodes/#post-189613 Wed, 02 Feb 2011 14:10:29 +0000 f_ilippo Hi joshua,
I’ve recorded movies (chunks of 15 seconds) using [jit.qt.record 160 240 @colormode uyvy] with the images received via jit.net.send as explained in my last post. Now if I want to record the movies back and recover the dim 320 240 and fullquality I must upload to GPU (with cc.uyvy2argb.jxs) dim 2 1 and use jit.gl.asyncread. Is there another way to do it?
thanx filippo

]]> http://cycling74.com/forums/topic/slab-compositing-and-colormodes/#post-189614 <![CDATA[Re: slab compositing and colormodes]]> http://cycling74.com/forums/topic/slab-compositing-and-colormodes/#post-189614 Wed, 02 Feb 2011 19:49:08 +0000 Rob Ramirez you can convert back without GPU using jit.uyvy2argb

]]> http://cycling74.com/forums/topic/slab-compositing-and-colormodes/#post-189615 <![CDATA[Re: slab compositing and colormodes]]> http://cycling74.com/forums/topic/slab-compositing-and-colormodes/#post-189615 Wed, 02 Feb 2011 21:02:12 +0000 f_ilippo Hi robert,
the fact is matrices were still in uyvy, recording movies with size 160 240 and attriute colormode set on uyvy compress a second time the images: the result is a movie with colormode argb and size 160 240, I’ve lost half matrices. I’m so stupid !
I don’t think there is a way to recover. As stated by joshua and generalized, once a matrix is in uyvy do not set it a second time in the chain processing ( GPU or CPU )
Have a good night
Filippo

]]> http://cycling74.com/forums/topic/slab-compositing-and-colormodes/#post-189616 <![CDATA[Re: slab compositing and colormodes]]> http://cycling74.com/forums/topic/slab-compositing-and-colormodes/#post-189616 Sat, 05 Feb 2011 11:11:08 +0000 DiGiTaLFX Well I had always wondered what the difference between using cc.uyvy2arbg.jxs and colormode uyvy was! Thanks very much for explaining Joshua :)

If anyone is interested, I made a small patch to see for myself the difference between these two modes. I was really surprised when that horrible red pixelation that you see on so many videos pretty much just disappeared! Finding Nemo is a great test video as you constantly see this around the clown fish against the blue of the sea.

DiGiTaLFX

– Pasted Max Patch, click to expand. –
]]> http://cycling74.com/forums/topic/slab-compositing-and-colormodes/#post-189617 <![CDATA[Re: slab compositing and colormodes]]> http://cycling74.com/forums/topic/slab-compositing-and-colormodes/#post-189617 Tue, 29 May 2012 17:02:36 +0000 Valentin Hi ,

I just came across this post, and I would love to get more detailed info on the usage with jit.qt.movie.

If I set @colormode uyvy when reading the QT movie in, can I pretty much ignore any worry about uyvy vs rgba if I process entirely on the GPU from there? If I understand, once the data gets to the GPU, it will be converted and processed as RGBA from there (unless I explicitly wanted to do an RGBA to UYVY conversion for some reason).

I am currently mapping the qt movie onto textures on .obj’s (jit.gl.model) etc. or onto videoplanes then rendering out to jit.window.

I just want to be sure I’m getting this right. Is there anywhere else in that chain that I need to explicitly set a colormode attribute or convert, or is it converted from the jit.qt.movie uyvy automatically by the GPU once it takes over the processing? And is there anywhere in that chain that I need to set dim attributes if the source data is from that QT movie?

Thanks! It is all a bit confusing. I am starting to understand, but there are a few pieces of missing info =)

Valentin

]]> http://cycling74.com/forums/topic/slab-compositing-and-colormodes/#post-189618 <![CDATA[Re: slab compositing and colormodes]]> http://cycling74.com/forums/topic/slab-compositing-and-colormodes/#post-189618 Tue, 29 May 2012 17:25:00 +0000 Rob Ramirez the common usage of colormode uyvy is to read from the disk to the gpu using uyvy.
once on the gpu, all texutres should be rgba.
for the users, all you need to do is set @colormode uyvy on the qt.movie, and send directly to a cc.uyvy2rgba.jxs @dimscale 2 1 slab.

after that initial conversion, you do not need to worry about the colormode.

– Pasted Max Patch, click to expand. –
]]> http://cycling74.com/forums/topic/slab-compositing-and-colormodes/#post-189619 <![CDATA[Re: slab compositing and colormodes]]> http://cycling74.com/forums/topic/slab-compositing-and-colormodes/#post-189619 Wed, 30 May 2012 00:29:22 +0000 Valentin I think that answers the question perfectly. Thanks!

V

]]> http://cycling74.com/forums/topic/slab-compositing-and-colormodes/#post-189620 <![CDATA[Re: slab compositing and colormodes]]> http://cycling74.com/forums/topic/slab-compositing-and-colormodes/#post-189620 Mon, 04 Jun 2012 14:16:41 +0000 lee wang hello,

i am a bit confused.

on my mac i do not need the cc.uyvy2rgba.jxs shader with @dimscale 2 1.
i simply can do this:

– Pasted Max Patch, click to expand. –

or this:

– Pasted Max Patch, click to expand. –

so which is the most efficient method for loading moviedata to the gpu on a mac ? do we need the cc.uyvy2rgba.jxs or not ?

best,
l

]]> http://cycling74.com/forums/topic/slab-compositing-and-colormodes/#post-189621 <![CDATA[Re: slab compositing and colormodes]]> http://cycling74.com/forums/topic/slab-compositing-and-colormodes/#post-189621 Mon, 04 Jun 2012 19:01:30 +0000 Rob Ramirez hi lee. quoting from Joshua’s post above:

“Alternately, if you are on OS X, you can just use jit.gl.texture @colormode uyvy or jit.gl.slab @colormode uyvy for the very first time the data moves from CPU to GPU (and *nowhere else down the slab processing chain*), and you do not need to use cc.uyvy2rgba.jxs. However, for the chroma smoothing and some nice color correction along the way, I think cc.uyvy2rgba.jxs and cc.uyvy2rgba.exp.jxs provide better results than Apple’s default UYVY texture -> RGBA conversion. Play around with them in jit.gl.slab-uyvy-gpu.maxpat ”

]]>