Playback of very high resolution files
Hey, just an update that might be of interest to the community. I'm working on a video installation that plays back several channels of very high resolution video (2x 3072x768, 1x 2048x768), and just had a very good experience with the Apple ProRes codec.
At first we tried using H.264, but it was a non-starter. Played fine in Quicktime Player, but no-go in Max/Jitter. About 6fps.
Switched to PhotoJPEG, and got much better results - about 24fps.
Accidentally rendered our output file using the Apple ProRes intermediate codec, played it through Max/Jitter - and got a shock: 60fps! (and with better color and detail)
Looking at CPU utilization suggests that the ProRes codec is much better at using multiple cores than PhotoJPEG, which is a rather old codec.
So anyone looking to do high resolution multi-channel playback in Max/Jitter, do look into the Apple ProRes codec. So far results have been stellar.
Yeah, Pro Res is good (but can use a lot of CPU), but if you want smaller files and less CPU usage, Apple Intermediate is a good candidate, its also very multi core aware, and gives you some decent control over encoding params, whereas Pro Res has some assumptions about files being formal video standards (ie fps/resolution).
Thanks for the tip, Apple Intermediate does indeed perform very well.
Still doing some side-by-side tests between AIC and ProRes...
What I'm finding is that while AIC files are smaller (probably due to 4:2:0 subsampling vs ProRes 4:2:2), ProRes does seem to perform a bit better, maybe because it can use hardware acceleration.
Also ProRes is cross-platform: Apple offers a free ProRes decoder for Windows.
NOt quite sure yet what we'll do - AIC does give us a bit more headroom on disk bandwith.
OK, we did a lot of tests, and it looks like ProRes is the way to go, for this application.
The files are about 50% larger, but still within bounds. Playback seems smoother: we keep getting higher framerates with ProRes than with AIC, I suspect either hardware acceleration or better multi-core support is in play here.
Color rendition and detail in ProRes seems better: subjective impressions from several observers were that contrast and sharpness were better than the AIC version.
The cross-platform availability is a nice plus.
Thanks for the feedback. I'll look into Pro Res for high resolution playback. Certainly sounds fun. Thats a lot of pixels!
Out of curiosity, where did you read Pro Res can use hardware acceleration (outside of FCP that is, since some systems like Aja and Decklink cards can encode direct to Pro Res and let the CPU skip encoding/decoding costs.. however usually when one says hardware accelerated these days one means GPU..)
Just curious, as id love to know!
As for performance deltas between systems, it may very well just be that AIC is optimized for standard video resolutions and bottlenecks on that many pixels. I mean, thats a ton of decoding work @ 60fps, even for lightly encoded files.
I also know that Pro Res is asymmetrical wrt to encoding vs decoding costs, meaning that its easier cpu wise to decode Pro Res than it is to encode, at least according to Apples whitepaper.
Cool stuff. Out of curiosity, what kind of machine are you running this on, and what throughput are you seeing drive wise (and are you using an array?)
Thanks.
I'm very curious as well to know what you're using and to see your patch because I've never been able to run that much resolution at 60fps. I use a top-of the-line MacPro and there's no way I can get a framerate at 60fps, even using GPU optimized patches.
Also, I did tests here with PhotoJPEG, H264, AIC and ProRes and although PhotoJPEG was slower, there was not that much difference between the other codecs.
I've attached a test patch that we're using.
This plays back a 3500 x 437 file by splitting it into two 1750 x 437 streams, each then going to a TripleHead2Go unit.
Note that everything goes through a jit.scissors operation, which is computationally expensive.
We are using an 8 core Mac Pro (clocked at 2.8Ghz) with 8GB of RAM, and three hard drives.
The reported fps from this patch using different codecs is as follows:
H.264 - ~7fps (this surprised me)
PhotoJPEG - ~24fps
AIC - ~40fps
ProRes - ~60 fps
The production patch does not use jit.scissors, it actually plays back three separate files in sync (2 @ 3072 x 768, 1 @ 2048 x 768). Each file is on a different drive (though having them all on one drive works too). We get pretty much the same frame rates as the test patch. (I cannot publish the production patch, unfortunately.)
Playing back the full resolution file (8192 x 768, or 6144 x 768 at one point) through the test patch was very slow, probably due to the jit.scissors operation.
Thanks a lot. I'll check that for sure. I understand you cannot publish the prod patch, no problems.
One last question: what video card are you using?
The video cards in system are:
1x ATI 3870 (512MB DDR3)
1x nVidia 8800 (512MB DDR3)
1x ATI of unknown type (256MB DDR3)
I wanted to go all ATI cards, but we got the nVidia card with the system. Doesn't seem to cause any problems.
We have a single TripleHead2Go connected to each of the 512MB cards, and the remaining card drives the control monitor and the 2048x768 TripleHead (in dual head mode).
The 256MB ATI card does not seem to able to drive a 3072x768 triple head, probably because the maximum texture is limited to 2048, or something like that. Seems OK with 2400 x 600, and with 2048 x 768.
Here's what I've gleaned:
Apparently only one of the AJA boards will do hardware acceleration for ProRes (and almost certainly only for standard resolutions).
I don't think the GPU is being used. CPU utilization while decoding seems quite low, which seems consistent with it being an asymmetric codec. (We were able to playback video through the patch while background rendering from After Effects on 4 cores.)
Apparently, ProRes supports some kind of real-time scaling, where it will automatically deliver a downscaled stream if the application doesn't need more. It seems this was built to work with Final Cut's RT preview engine. I am not sure if it is coming into play under Max.
I suspect that it is just a more modern and carefully coded codec, with more efficient use of multiple cores.
Thank you very much . Good tips about codecs
@gpvillamil use jit.submatrix instead of scissors .