VIDDLL package public beta for jit.record and jit.vcr
Hey everyone, I’ve just pushed an update to the viddll package that contains beta functionality for writing movie files with jit.record and jit.vcr. Any and all testing and feedback is appreciated during this beta period.
To start beta testing, simply install the latest version of the VIDDLL package (v 1.0.4) from the Package Manager. Access the Package Manager by selecting “Show Package Manager” in the File menu or by clicking on the cube icon on the top of the left toolbar.
Functionality of jit.record and jit.vcr should be similar to avf and qt engine. There is a new feature: cache_size (similar to the viddll jit.movie). This feature allows users to create a separate thread for writing frames, and controls the size (in gigabytes) of the memory cache used to store incoming frames.
The attached patch is provided as documentation.
Windows users wishing to test out jit.vcr functionality will also have to install a beta version of the jit.vcr external.
The installation process for installing jit.vcr on Windows is as follows:
- quit Max
- download the externals here:
32 bit - https://dl.dropboxusercontent.com/u/83824454/jit.vcr.mxe.zip
64 bit - https://dl.dropboxusercontent.com/u/83824454/jit.vcr.mxe64.zip
- unzip
- for 32 bit Max, drag the jit.vcr.mxe file into C:\Program Files (x86)\Cycling '74\Max 7\resources\externals\jitter, replacing the existing file
- for 64 bit Max, drag the jit.vcr.mxe64 file into C:\Program Files\Cycling '74\Max 7\resources\externals\jitter, replacing the existing file
You can now start beta testing the viddll 1.0.4 package with jit.vcr.
Feel free to follow up here with any feedback, questions, comments.
Thanks for testing!
Hi Rob
It's awesome !!
"jit.vcr" & "jit.record" ( h264)(max32) work like a charm on surface pro 4. win10
Don't miss to select "viddll" engine in the preferences
Thanks so much !
Mathieu
Thanks Rob!
So far the new jit.record works like a champ.
Even better, jit.vcr works in Windows 8.1 64, Ableton 9.6.1, M4L 64, h264 with sound.
The only odd thing is that I need to enable DSP (by opening jit.vcr Help). ;) But this I have to do also with qt and in 32bit version. Could somebody change this or help me to avoid this nuisance. And thanks for viddll. First time in 5 years as Max user I'm seriously thinking of doing something with jitter!
HAP seems work be working with [jit.record] ! thanks so much for this.
So i see the "cancel" message isn't exactly made for cancelling the recording, a feature that has been asked about and suggested, but perhaps still can be used as such? If the cache_size is large enough, we can essentially store the whole recording in the cache (is this stored in RAM?) and just "cancel" the recording and erase the cache. A file will be written, but maybe it's an empty file? With my quick tests, it doesn't seem to work this way.
Another question, does it store the frames in the cache as uncompressed/raw images and does the encoding when you "stop", or does it encode each frame before storage? This would be helpful to know so we can set the cache size dependent on the codec we use.
The GIF export is also a very cool option but seems to write huge files. Anyway to reduce the quality / increase the lossy-ness or compression on these (or other codec?)
Anyways, thanks for this. Recording to HAP will really come in handy.
thanks much for the feedback everyone.
So i see the "cancel" message isn’t exactly made for cancelling the recording, a feature that has been asked about and suggested, but perhaps still can be used as such?
Not entirely sure what you're referring to here.
If the cache_size is large enough, we can essentially store the whole recording in the cache (is this stored in RAM?)
Yup
and just "cancel" the recording and erase the cache. A file will be written, but maybe it’s an empty file?
When cancel is received, it will dump the cache and attempt to close out the file, thereby making it playable.
does it store the frames in the cache as uncompressed/raw images
Yup frames are stored in the cache uncompressed
The GIF export is also a very cool option but seems to write huge files. Anyway to reduce the quality / increase the lossy-ness or compression on these (or other codec?)
easiest way to decrease file size with gifs, is decrease framerate and dimensions.
if there's enough interest in gif export, i'd consider the possibility of implementing the optimizations described here: http://blog.pkh.me/p/21-high-quality-gif-with-ffmpeg.html
Hi Stonerich.
The only odd thing is that I need to enable DSP (by opening jit.vcr Help). ;)
are you referring to using jit.vcr in MFL?
Hey Rob,
I guess the cancel concern wasn't as widespread as I might have suggested, but here's a thread related to it: https://cycling74.com/forums/jit-qt-record-how-to-not-save-a-recorded-movie-if-duration-is-less-then-a-minute/
It'd be an interesting feature but not too important.
-------
and GIF export doesn't really interest me personally (definitely a cool option though), but a general control of codec quality is definitely desired. I know for HAP, there's diminishing returns for any quality over 75% and produces much larger files. Do you just have it's quality set on max/100% currently?
----
after looking at the async-framedump subpatch, it's more clear what's going on with the cache and how to use it properly.
every codec has different options exposed, so there's no easy answer here. and for add-on codecs, like Hap, your only source of info is the source code (welcome to the world of ffmpeg).
the Hap codec allows you to change the codec format via a dictionary option, and I've provided an undocumented (and therefore subject to change) feature that exposes setting these options. you are on your own in discovering these options and verifying their efficacy.
here's a patch that shows how to set codec options via a max dict object:
Hey Rob,
Would it be possible to modify jit.record so that we can specify arbitrary file extension? It used to be possible but now, jit.record throws an error.
since the viddll engine supports multiple file formats, it infers the file format based on the file extension, so no this isn't currently possible.
why would you need to do this?
Unrelated to filenames, I sometimes get a bunch of error cannot write frame: Invalid argument while recording, the file gets recorded anyway and it seems to play fine, but maybe there are some frames missing.
I use to record lighting data as video files and use the extension .dmx so there is no confusion as to which files can be played as video or lighting.
Is there any extra download needed to record as h264? I get the error: cannot open video codec Unknown error occurred.
Cool, thanks for the info and deeper function features. I'll look more into it. Just confirmed that Hap Alpha works. Awesome.
Ok, for the h264 error, its because it doesn't accept my matrix dimension of 512 x 1. No problem as huffyuv works.
Allright, so the jit.record error cannot write frame: Invalid argument happens when I try to record a non-conventional dimension (512 x 1) with the realtime 1 flag.
Yes, jit.vcr in M4L.
Ok, for some reason you can't enable Audio from the switch down right in M4L (but some help patchers strangely allows this, like jit.vcr and BeapConverter and probably some others too). But from the Options menu I found Audio Status, which allows me more elegantly (than open a help file) to enable audio. So all is well. But not to be too easy, the Audio Status window must stay open all the time for DSP to be enabled. Strange world of M4L!
hi stonerich.
i'm unable to reproduce your problem, and a little unsure of the steps. if you'd like us to take a look at this, please provide a live project and/or device file, and some exact steps to reproduce.
thanks much.
Hi Rob,
Do you think support for non-standard matrix dimensions can be improved?
sorry hugobox, forgot to reply. this is codec specific, so it seems if h264 is having problems with those dims, then you will have to use a codec that doesn't exhibit those problems. it sounds like the huffyuv codec was working as expected?
At least with huffyuv (512 x 1 matrix @realtime 1) there is a file output that contains the right data, but there are a lot of error messages (error cannot write frame: Invalid argument). So I think it works for most of the frames but certain frames might be left out of the recording? This error message happens only while using realtime 1.
please provide a patch that exhibits this issue and i'll see if anything can be done.
Ok figured it out, was sending matrix at over 150 FPS yet recording at 50 FPS. Thanks Rob!
Hi Rob,
I think this peculiarity is due to the fact that M4L is using Ableton's sound system/engine, and it's not a problem with Viddll. As I mentioned I had the peculiarity occurring with qt. And the quick solution is to open the Audio Status window (which shows that the Driver is Live abd the status is on!). But of course it's a bit funny to have to open the Audio Status window.
Setup: Windows 8.1 (64bit), Ableton 9.6.1 (64bit), Max for Live 7.2.3 (64bit), Viddll 1.04 beta
Here's the patch:
hi stonerich.
thanks for the patch. if i copy your device patch into a new Max Audio Effect device, and send the "write" message to jit.vcr from Live (rather than opening the device patch in Max) it works as expected. apologies for the inconvenience, but i suspect this is just a peculiarity of MFL (which it sounds like you've found a workaround for).
Thanks Rob,
and confirmed that the patch works, when unopened. Nothing to do with Viddll. Just an inconvenience of M4L that I need to live with. Sorry to have bothered you.
Is it me or does the recordings are always at 600FPS instead of 30? Testing with your provided patch.
Hi,
Is there a method to adjust the bitrate with "jit.record" (H264 codec or mpeg4) ?
i tried to find this option via "print_options" without success ...
mathieu
well, turns out you can!
print_options will only print codec specific options (otherwise there would be a massive dump of options). generic options can also be set, and you can find some info on these here, by searching for "AVCodecContext AVOptions" : http://ubwg.net/b/full-list-of-ffmpeg-flags-and-options
although i imagine the efficacy of these options is still codec specific.
patch below demonstrates how to specify target bitrate (option "b"):
excellent. most excellent. thank you for your illuminating this topic even more. the AVOptions are massive, like you said, but there's some in there that will come in handy.
interestingly enough, h264 bitrate seems to bottom out at 336 Kbps. Trying to set anything lower will just output 336. i'd never record at this bitrate, but i suppose someone somewhere would want to save really degraded video files.
Thanks !
That's great !
Thanks a lot M. Ramirez for sharing this solution and documentation.
I made a video from 3D sources playing from jit.movie going through some jitter nodes and recorded using jit.record (on win7 x64):
Here is a link :
https://www.youtube.com/watch?v=y4cHE0ZjuoM
There is some thought and questions that came during and after the process :
- "Framedump" saved my life, because I wanted to work in 1080p with heavy processes. That's a usefull function.
- I installed the HAP codec library, but while the encoder seems available to use from the jit.record encoders auto-built list (using the right outlet), I've not been able to record sucessfully. Is there any way to limit the list to only the functionnal encoders? Because, I had more that a hundred ones, which many were not able to record.
- I don't know if it's jit.record related, but is there any way to record the alpha channel to something like PNG sequence or Huffyuv.
- More generaly, can someone direct me to some infos on how to separate alpha channel from RGB, when playing a HAP or PNG sequence, to avoid playing a black mat to further compositing uses...
-One thing I never understood was that the duration of ouput file may vary depending of the encoder I selected... Huffyuv was always twice the time and jpeg sequence were just the exact duration of the video source. (using framedump on jit.movie) .
Cheers guys,
Is there any way to limit the list to only the functionnal encoders?
no.
I don’t know if it’s jit.record related, but is there any way to record the alpha channel to something like PNG sequence or Huffyuv.
yes both those codecs will record alpha channel. currently PNG sequence is not supported on windows. "animation" codec will also record alpha channel.
More generaly, can someone direct me to some infos on how to separate alpha channel from RGB, when playing a HAP or PNG sequence, to avoid playing a black mat to further compositing uses…
not really sure what you're asking, but if this isn't related to viddll, maybe better to start a new thread.
One thing I never understood was that the duration of ouput file may vary depending of the encoder I selected… Huffyuv was always twice the time and jpeg sequence were just the exact duration of the video source. (using framedump on jit.movie) .
can you provide a patch that reproduces this and i'll take a look?
Hello,
Is it possible to set the level of codec h264? It seems that it has been compiled with a level 4.0 but to make the videos readable with iOs we need a level of 3.1.
Best regards.
hi NESIFAO, how are you determining the level of the file recording?
Hi Rob,
It seems that Jit.record is still not recording what the real-time render is showing.
I have similar or probably the same problems as when I was emailing you about that problem (few months ago). This time I didn't use any materials and lights and visual feedback/smearing because I assumed the problem was there. What you can see on pictures below was created with few jit.gl.mesh objects, layers set to 0, 1 and 2, blend enabled on all of them, all white with different opacities. However the opacity is not the same any more when recorded...or just the blending works differently, I don't know. I was recording in UHD resolution, capturing with camera (last time I tried with node) and hi-res on render was set to 0.
So as you can see on picture called ''real-time'' the fine point mesh is much more prominent. The image looks grainy while the lines dominate on the ''it-record'' one. The effect is not very obvious because I needed to drastically compress the images in order to be able to attach them here. But I guess it is obvious that the lines are much more visible in 'it-record'' one.
Another thing is that on the ''real-time'' picture you can see a lot of very white areas which emerge from layers accumulating the luminosity/whiteness. This effect seem to be almost gone in 'it-record'. It is darker, not so dynamic in luminosity.
I REALY REALY WISH that this could be fixed somehow. I can not stress enough how much this is killing me.
PS2: recorded with prores4444
Your luminosity and visibility problems could be because of bugs in Max's handling of YUV data. Your prores444 *is* YUV data because almost all modern codecs use YUV (or the related Y'CbCr) to encode video data. This is used in h.264, h.265, prores444, jpeg, etc.
PLEASE if you want this fixed, they need to hear from Max users like yourself. CALL or EMAIL the Max support line to report the problem, reference the information below, and ask for fixes. https://cycling74.com/contact-support/
Max has bugs in the YUV related code (jit.window, jit.pwindow, jit.movie, jit.uyvy2argb, YUV example shaders, ... many places) where they do not correctly handle the encoding and decoding of YUV data. This results in
low contrast images
brights are washed out or merged together -and- darks are muddy or lost in blackness
colors are not correct; shifted in hue
There is no easy fix you can do in a patch. The fix needs to be in the C code of jit.movie, jit.record, jit.pwindow, etc. because the video data has already been corrupted and lost before you get access to it in the patch. I've reported it to the core Max team for two years with repro cases and mathematical proof. I also volunteered my help. Unfortunately, they have so far chosen to not fix the bugs or accept help.
The reason for the bugs is incorrect math/code due to a gap in knowledge on how to handle conversions to/from YUV (technically Y'CbCr). The same bugs are in Max 7, Max 6, and probably Max 5. Some of the fixes to objects like jit.uyvy2argb will take two minutes because only 3 multiply-add formulas in C code need to change. Other objects like jit.record and jit.movie will need more time to coordinate a holistic fix.
If you stay *ALWAYS* in rgba, then you should be ok.
However, you will likely encounter the bugs if you ever touch YUV (YÇbCr) by playing, converting, encoding, or recording a video. This is even when you playing back a movie made by another application because the bug can surface when the encoded YUV file (e.g. h.264) is decoded and converted to RGB by jit.movie.
hi T, hi Dale.
T, since you stated using prores4444, I assume you are using the default AVF engine for jit.record, so this issue is unrelated to the topic of the forum post, the VIDDLL beta of jit.record.
Dale, i appreciate your dedication to your cause, but this also is unrelated to the topic, and I highly doubt is the cause of T's rendering issues. the AVF jit.record handles all rgb to uyvy conversions using internal apple libraries, so entirely out of our hands. as mentioned to you previously, i will be examining the viddll library to see if there's a way to facilitate the full color range.
T, my best guess is your issues are the result of recording from a high resolution display, and could possibly be remedied by doubling the resolution of your recording (since this is basically what the hi res display is doing). try capturing your render with a jit.gl.node @adapt 0 @dim set to twice the display width x twice the display height, making sure you use that same resolution for the jit.record dims.
T, i remembered we had a previous email exchange about this issue, where the double-size suggestion was made. so i assume you are already doing that. therefore i investigated further and found out that there are in fact some color profile settings to the avf jit.record that should force the full color range when recording.
my apologies to Dale, for dismissing this possibility. I can't promise that this fixes the issues, because in my testing i was unable to notice a difference, (and Dale's test patches exhibit no differences with this change), but maybe this is the cause of your frustrations with jit.record. if this does in fact fix the issue, we will both owe Dale a big thanks for his persistence on this.
T, i'll send you an email with this version of the avf.engine to test with and we'll go from there.
Hi Rob,
When recording with @realtime 1 (with the example patch), the output file reports a different FPS that what was set.
Example:
a recording in h264 @30fps results in a file @22fps
a recording in huffyuv @30fps results in a file @600fps
I'm on windows 10, with Max 7.2.4 64bit.
the fps setting is pretty useless with realtime recording. the fps will be whatever fps you send frames in.
Ok but I don't understand why it would be codec dependant? (h264 at 22fps but huffyuv at 600fps in the jit.movie example patch) And I'm pretty sure the example patch doesn't output matrices at 600fps, does it? Thanks for your time!
Hi Rob,
Glad to see jit.vcr finally work in Windows 10 x64 like a charm! I'm using Max Forl Live 7.2.5 and h264 codec in jit.vcr.
Is there a way to specify audio bit rate for AAC LC codec?
Currently it defaults to 64kbps, I'd like to set 384kbps.
My videos written by jit.vcr have these properties:
Video: MPEG4 Video (H264) 1920x1080 29.99fps 22323kbps [V: DataHandler [eng] (h264 constrained baseline L5.1, yuv420p, 1920x1080, 22323 kb/s)];
Audio: AAC 48000Hz stereo 64kbps [A: DataHandler [eng] (aac lc, 48000 Hz, stereo, 64 kb/s)]
hi AVLG, i've noted your feature request and i'll see what can be done to allow for adjusting the audio bit rate.
Hi Rob,
In a similar vein as AVLG above, I also require high-quality audio in my recordings and thus need to be able to specify the bitrate (otherwise I have to replace the audio track in post-processing, which is tedious). Ideally I want it to be 320k VBR.
To that end, I've tried using a similar structure to the example you posted here on July 7 for jit.record, but in jit.vcr. Specifically, I am putting the message "set aq 320k" into a dict and then sending that to the left inlet of jit.vcr. I've tried a few variations on this, but none worked. Is there something I'm missing here or is it not possible yet to send ffmpeg options via jit.vcr?
Thank you for your help!
this remains an open feature request, and i've noted your additions.
Hey Rob,
thanks for this great jt.record implementation!
my problem which I had also with plain jt.record object: the recorded clips look like played in "high speed" no matter which codec I use
& no matter how long I record. if I record like 20 seconds or 2 minutes - my VLC player plays it always in 7-10seconds,
that's why the accelerated "high speed" look.
recording with plain jt.vcr gives me normal speed but just around 7fps no matter which fps I set.
---> activating "adapt" & "realtime" eliminates this "plays 1min clip in 10 seconds"-problem, but it constrains the resolution to 640x480,
even though the jt.record objects remains 1920 1080....
---->now after deactivating adapt & realtime, still 640x480 resolution - that smells like a max-bug.
Damn so frustrating.
Would be cool if you have any idea out of this dilemma - or an explanation?
Thanks!
(OSX, macbook pro mid 2012, MAX 7.3)
PS: probably important - what I try is processing HD-Video material through some vizzie effects & recording the output.
please provide a patch demonstrating what you are describing
I sent nearly the same help-request to the support & got this answer
"Perhaps you might wish to have a look at the realtime message to jit.record, for starters...."
Not sure what to make of it as I described in my message that I tried "realtime".
Or I am to noob to understand the lingo or the sarcasm. ;-)
Nevertheless, here is the code Rob & thanks for looking into it.
the adapt attribute tells the recorder to use the dimensions of the matrix input for its output file, ignoring the dimension arguments. if you've enabled adapt and the recorded file dims are 640x480, that means your input matrix dims are 640x480.
if you enable adapt, the dim attribute sets itself to the input dims, overriding previous values. disabling adapt will still use these dim values until set again by user input.
realtime tells the recorder make the recorded file duration the same as the amount of time recording is enabled, ignoring the fps attribute. if you've enabled realtime and the output file is playing at 7 FPS, that means Max is processing and recording frames at 7 FPS.
non-realtime uses the FPS value to determine the duration of the output file. if you've set the fps to 30, and are processing and recording frames at 7 fps, the file will play back much faster than realtime.
i would make some recorder tests using very simple patches, until you understand better what each of these options do.
thanks rob! you helped me already.
through your explanation & further testing I found this out:
- realtime is good for my purpose
- CPU overload results in low FPS:
it is not the crazy vizzie fx, just playing my HD-clips in the vizzie player overloads the CPU & results in very low framerate.
playing the stock low-res test clips results in nice & smooth recorded clips.
- just playing my HD-clips in the very basic "drag & drop" max7 player: whole max freezes.
so finally the codec (?) of my clips is the source of the problem - or do I need a video power machine to play HD-clips in max7?
Maybe there is a tutorial how to compress video for max7 usage?
I post the codec infos of my clips, which don't work, as screenshot attachment - maybe sb. got an idea?
sorry, for hijacking the thread again.
it has nothing to do with the grate recording patch from Rob.
I found my real core problem:
the second I pipe out video from qt.player (which is running smooth alone) to just one vizzie effect - the whole max7 overloads/stutters/freezes
I will continue to search for solutions in other threads. ;-)
minor expansion of the patch i just posted on the other thread that demonstrates how to readback to matrix for recording using jit.gl.asyncread and jit.record:
alternatively, you can send to syphon and use Syphon Recorder, if on the Mac:
hey guys, just wanted to point out that with the 7.3.2 update the default audio bitrate for jit.vcr has been increased, and should sound much better. i've also added a way to specify audio specific ffmpeg options via a dictionary. check out the jit.vcr help patch, in the VIDDLL specific tab for more info.
Rob, thanks a lot! It works:
Video: MPEG4 Video (H264) 1920x1080 30fps 34632kbps [V: DataHandler [eng] (h264 constrained baseline L5.1, yuv420p, 1920x1080, 34632 kb/s)]
Audio: AAC 48000Hz stereo 275kbps [A: DataHandler [eng] (aac lc, 48000 Hz, stereo, 275 kb/s)]
Windows 10 x64.