Shifting YUV channels by sub-pixel amounts? (GH2 camera HDMI chroma fix)
I'm capturing the HDMI out from a Panasonic GH2 camera, and trying to fix a flaw in the signal. The (yuv) channels are slightly shifted so saturated objects have a nasty interlaced fringe around them.
How others have solved it:
An avisynth script:
http://www.dvxuser.com/V6/showthread.php?237584-HDMI-Capture-Problem-SOLVED-AviSynth-RULES!
An After Effects plugin:
http://sites.google.com/site/hsgh2chromafix/
The basic idea:
Start with full rez (not subsampled) chroma
Shift each YUV channel by a half-pixel in certain directions
Convert to subsampled chroma format
I'm trying to port it to MAX but I'm coming up short.
Jitter cpu objects like jit.rota won't do subpixel positioning. It seems (from the forums) that there's no way to unpack channels on the GPU without either shader programming or MAX6/gen.
Here's a MAX5 patch that works, but it's extremely sloooow. Any ideas?
Here's a 1920x1080 uncompressed still image to load into the patch above:
http://zachpoff.com/guests/guestbox/GH2-chroma-test-rota1.pct.zip
you really wanna do this GPU accelerated, ie. by a shader program (or jit.gl.gen). processing full HD resolution on the CPU like this will always tax it very heavily.
btw, i'm interested in this. i've got a hacked GH2 myself.
I'll investigate gen via max6 and update this thread with any news. Meanwhile, some notes to the group:
Using the Panasonic GH2 with MAX via HDMI:
The camera will output a full resolution (1080i) live HDMI signal during idling or recording, and you can turn off all the graphic overlays. (Very rare in "HDSLR" type cameras.)
The camera must be recording if you want a good HDMI signal. When idling, the HDMI signal is contrasty with crushed blacks and extra fields, and it doesn't reflect the framerate of the current recording format. So if the cam is set to 24p, the HDMI signal is 60i and ugly until you press record. Then it improves radically. Some graphics appear momentarily when you hit record, but they disappear on their own.
In all NTSC modes (60i, "HBR" 30p, 24p) the camera adds random redundant fields to the HDMI output stream. The avisynth script above can remove these, along with the separate chroma fix. (I haven't looked into extra-field removal in MAX.) Internally, the sensor/recorder run at a 59.94 timebase (like every other product in NTSC-land) but the GH2 adds HDMI fields to make the output stream exactly 60i. (Nobody knows why.)
Blackmagic capture drivers on the Mac don't allow MAX to see 60i streams, so BM cards cannot capture from an NTSC GH2. Blackmagic's Mac capture application CAN see and record the GH2 stream, and supposedly it works on MAX in Windows, though I haven't tried.
The Matrox MXO2 Mini captures NTSC GH2 streams in MAX. Looks great, costs more, adds unnecessary external box, and has much longer capture latency than Blackmagic card. Grrr.
The solution for me is PAL. In PAL progressive mode (called "HBR" on the GH2) the camera will NOT add random fields to the HDMI signal during recording. It's a progressive 25p frame, transmitted over a 50i HDMI stream. When you capture it in MAX (via either Blackmagic or Matrox) you get 25 frames per second with no interlacing artifacts at all. It still has the chroma shift problem, but it's the best option I've found for live streaming from a
To enable PAL mode on NTSC cameras you need to hack the camera. (http://www.gh1-hack.info/wiki/PToolSoftware) This amazing firmware hack tool allows many exciting modifications, but in this case I care about the "NTSCPAL" option. After the hack, the NTSC GH2 has a new "VIDEO OUT" item in the setup menu. Set it to PAL, toggle the power, reformat the SD card (required), and you're done.
Whew. That was a long post, but hopefully my trials (and errors) will help somebody.
great info, tanx!
After much frustration and forum trolling, I implemented the GH2 chroma fix using jit.gl.pix. It works fine (at about 20fps instead of 2s/frame!)
Questions for gen gurus:
I couldn't figure out how to translate the input side of cc.uyvy2rgba.jxs into a gen patch, so my code requires an upstream conversion to AYUV. Does anybody want to implement UYVY to RGB conversion in gen patcher?
There's a slight saturation increase in the YUV to RGB conversion. I don't understand enough about the color science to figure it out.
The MAX patch is here. (The gen patch is attached as a file.)
too bad i don't have an hdmi grabber for toying with this...
what's the framerate like when you send the jit.gl.pix output to a jit.gl.videoplane for GPU rendering, cutting out all the superfluous CPU operations?
what's your machines specs? just to get an idea of what kind of performance to expect from something like this.
In my real world setup without the demo-patch CPU objects, I get great performance on an 8 core Mac Pro with a Radeon 5770 card: About 60 fps for live capture and 45 fps for 1080/25p ProRes test movies. On a first-gen Intel Mac Mini (Intel GMA graphics) I get nothing but a black screen. (If only Jitter would give some feedback when GPUs can't handle certain instructions!)
Remember that the grab/movie pipeline is rgb (not uyvy) and I'm using a jit.argb2ayuv cpu object before the gpu. Not the most efficient, but I'm too nooby to figure out uyvy->yuv conversion in my first gen patch.
I just added some basic v-channel blur which helps smooth even more artifacts without much impact on sharpness. It's on by default, but can be bypassed with "param v_blur 0". New patcher attached...
I fixed some unconnected stuff inside the gen patcher, added an "enable" option, and made defaults for the channel offsets. Now you can just drop this object in your patch and it will fix the image from the GH2 automatically.
(new genjit file and an updated max patch below)
NOTE: Throughout this process I've noticed differences in chroma and luminance, caused by converting between color modes in different places in the patch. I made a new thread here: https://cycling74.com/forums/bug-color-shift-when-translating-between-color-modes-jit-qt-movie
I made a shader "cc.uyvy2yuv.jxs" that outputs full-rez yuv to the GPU. (Just a hacked version of cc.uyvy2rgba.jxs)
Now you can capture in uyvy (for optimized speed), use cc.uyvy2yuv.jxs to convert to full-rez yuv, then fix the channel offsets in jit.gl.pix. The output is then rgb, ready for normal GPU processing.
BUT... The shader produces a green screen when you first load the patch. "Touch" the slab object box (add and remove a letter) and it fixes it. Dunno why this happens.
Whew. done for the night.
(New shader and updated max patch below. Get the genjit file from the previous post.)
great work man! i'm sure this will come in handy at some point.
FYI: When I inserted the shader/gen patcher in a real-world GPU pipeline instead of my demo patch, there was no green screen problem.
Maybe it was a gl context issue since my demo patch has 2 jit.windows and a jit.pwindow? Who cares. It works.