jit.movie video playback - small latency and rate
I use "jit.movie @output_texture 1 @vol 0 @autostart 1 @loop 1" to loop a very small video in Live. I change the rate of the video to match the bpm on the live session. But the video runs slower compared to if I drag it directly in Ableton (where in it in sync, and with better definition and color). When I trigger the playback with the Live "Play" button, there is always a small delay before it start playing again, and that delay varies every time.
What am I doing wrong? Is it coming from the rate modification?
You could try to sync movie to plugsync~
you need movie framecount and length in beats (in live that is quarter notes)
then scale raw ticks output to frames,
and run movie using frame_true $1, bang message.
in example below, tiny movie has 254 frames - captured 2 bars of
4/4 111 bpm loop in max
in most cases these kinds of issues are fixed by 2 things.
ensure you are processing textures. The first step of that is enabling output_texture on your jit.movie, as you've already done. But you must ensure you aren't converting back to matrix somewhere downstream. If you're certain that's the case then that is not the issue here (if you're not certain, then you'll have to post your patch so we can help you determine that).
Use a realtime performant video codec. Hap is the most widely used, but any prores variant should also work. h264 is the most common video codec, and also the worst for realtime manipulations.
If you'd rather not change codecs, then I'd do as suggested above, and control your frame output explicitly (via frame_true) rather than use the internal jit.movie playback control. In this case you want to make sure autostart is 0, and never send a start or rate message.
Hello Rob,
Very interesting insights.
I am just curious why we should avoid to send a rate message.
Does this ruin tight sync forever? Or do you mean that we have to trigger every frame with the frame_true message? So not by sending: frame_true, start
Petcode
I think rate will implicitly trigger a start playback, but I might be wrong. In any case if you aren't using internal playback (via start message), then rate has no effect and therefore should not be used.
Again, this is all if you want to control frame output explicitly and synch to some other timer than the internal one.
I've posted examples of this before, but IMO the best way to synch playback to something, assuming the 2 best practices above (texture output and proper realtime codec), is to trigger rate changes based on proximity to some target position value, at some regular cadence (e.g. every second or so).
And if wildly out of synch (again determined by some position difference with some target position), then use a seeking message (time, pos, frame_true, whatever makes the most sense for your use case)
here's an example that uses this two-pronged approach, ie rate corrections if within some threshold, otherwise seek corrections. This is one approach that might not work in all situations, and the parameters will likely need to be adjusted. If you know the sync server values should be close to the sync client, then this could probably be simplified, but I was just trying to sync two random built-in media sources with somewhat similar lengths. Seems to work well even with rate changes and other adjustments to the sync server (the playlist~ in this case).
Hey guys, sorry for the late response. Thank you so much for all your help, I really appreciate it!
I tried the solutions suggested here but always come up with problems and I don't have the knowledge to solve all those micro-obstacles to make it run smoothly. And here I thought it would only take me a few hours to finish this fun little project 😅
I'll keep working on it, and report back in a while (if the task hasn't crushed my spirit by then)
curious to hear more about the friction you're encountering, if you're willing to share as you go.
I tried Source Audio solution, but I get a new frame only on the beat, then nothing, then it moves a little, all in a seemingly random order. The video plays when I press play, so that part I understand, but the reset to "pos 0" doesn't work in my new version.
I tried with a simple 23 frames/2 beats mp4 or 292 frames/ 20 beats mp4, but and get the same glitchy results.
Also I do not understand the % bit: in the helper patch it says it divides the left by the right entry but I don't understand the results (and sometimes the result doesn't update when I change the entry values, but that's probably another story)
I am very sorry if it all seems overly obvious and simplistic to you guys 😅. And I greatly appreciate the help.
plugsync~ is not working in max, only in live device.
in max try transport
and as Rob mentined, you don't need to use any other message than
frame_true $1, bang.
example I posted was fixed 8 beats.
for unknown length you also need to set scale object.
your next mistake is int modulo
you see the difference here :
Oh I see, now it works properly. I was doing all the calculations to go from bpm to adjustable frame rate but your use of the scale function is so much simpler and elegant. Thank you again to you and Rob, this forum is awesome 🤩