Parmerud

After battling for some time with my own resources, searching the forum etc, I have decided too see if there is someone out there that can help me out of my predicament.

I am making a patch for a theatre company who wants to work with various kinds of live video on stage. One thing in particular is that they want to record short videos with image and sound on the fly and then replay them later. Fine, no problem I thought until I realized that making live grabbed sound and video is maybe not that simple. Here is what I do:

I use a jit.matrixset as my video buffer and a normal audio buffer for the sound. However, because the actual framerate of the video is fluctuating (sometimes a lot) playing back video and sound doesn't stay synced. Since the amount of recorded time is preset (so the matrix set id prepared) I have tried some desperate tricks like measuring the "real" time between first and last recorded frame and then using this time to measure the difference between video time and audio time so I can time stretch the audio to fit the video. That improved things slightly but is not consistently trustworthy especially over longer clips.

I have also tried various ways of ensuring a steady frame rate but that proves to be almost impossible as well...Aaaagh!

Hi,
After battling for some time with my own resources, searching the forum etc, I have decided too see if there is someone out there that can help me out of my predicament.
I am making a patch for a theatre company who wants to work with various kinds of live video on stage. One thing in particular is that they want to record short videos with image and sound on the fly and then replay them later. Fine, no problem I thought until I realized that making live grabbed sound and video is maybe not that simple. Here is what I do:
I use a jit.matrixset as my video buffer and a normal audio buffer for the sound. However, because the actual framerate of the video is fluctuating (sometimes a lot) playing back video and sound doesn't stay synced. Since the amount of recorded time is preset (so the matrix set id prepared) I have tried some desperate tricks like measuring the "real" time between first and last recorded frame and then using this time to measure the difference between video time and audio time so I can time stretch the audio to fit the video. That improved things slightly but is not consistently trustworthy especially over longer clips.
I have also tried various ways of ensuring a steady frame rate but that proves to be almost impossible as well...Aaaagh!
Ideas, help, solutions anyone?


syncing-of-live-grabbed-video-and-sound

jit.record in realtime mode (@realtime 1) should keep things in sync.

meaning if you start a recording with jit.record, and at the same time start a recording with sfrecord~, they should stay in sync when played back and started at the exact same time.

jit.record in realtime mode (@realtime 1) should keep things in sync. 

meaning if you start a recording with jit.record, and at the same time start a recording with sfrecord~,  they should stay in sync when played back and started at the exact same time.


I am just looking at that solution. However its a bit cumbersome since they also want to play with very fast access sometimes so I guess I will have to make a smart scheme to immediately read back the file to a it.matrixset to be able to play around with it in ways jit.movie would find less funky like playing very fast or jumping around at excessive speeds...

Yes thanks Rob,
I am just looking at that solution. However its a bit cumbersome since they also want to play with very fast access sometimes so I guess I will have to make a smart scheme to immediately read back the file to a it.matrixset to be able to play around with it in ways jit.movie would find less funky like playing very fast or jumping around at excessive speeds...

I just encountered the jit.vcr that I have never tried. It works fine and is extra attractive since the sound is embedded in the film. Happily loading up the jit.movie~ however was a little disappointment. I don't know if my Max version of it is corrupt since even the help file didn't work. There are no outlets from jit.movie~... Is it me/my Max or is this a known issue?

I don´t feel like using spigot~ since it takes some time to read the sound part of a movie...

will need lots more information to help track down the jit.movie~ issues. to start with OS and max version.

Ok, tried it on my stationary machine and it works. Could it be a hardware issue with my laptop? It´s a 2012 Macbook pro with i7....

In any case I could move the project to my old Mac Pro and finish there.

Ok, tried it on my stationary machine and it works. Could it be a hardware issue with my laptop? It´s a 2012 Macbook pro with i7....
In any case I could move the project to my old Mac Pro and finish there.


One thing I wonder about is why I get a constantly decreasing frame rate over time. It seems to be consistent no matter what machine (not just my own ones) that I am using. Memory leakage? Over a time span of maybe 10-15 minutes frame rate goes down by approx 50 %.

Again just me being a lousy Maxer or is this something others have been experiencing?

One thing I wonder about is why I get a constantly decreasing frame rate over time. It seems to be consistent no matter what machine (not just my own ones) that I am using. Memory leakage? Over a time span of maybe 10-15 minutes frame rate goes down by approx 50 %.
Again just me being a lousy Maxer or is this something others have been experiencing?


Seems like I am lost in space here. Actually the fram rate fluctuates up and down a lot over time... OSX activities maybe (backing up, cleaning stuff in the background, calling grandma...). Never mind, I guess it´s just another of those things you learn to live with in the glorious world of computers.

Seems like I am lost in space here. Actually the fram rate fluctuates up and down a lot over time... OSX activities maybe (backing up, cleaning stuff in the background, calling grandma...). Never mind, I guess it´s just another of those things you learn to live with in the glorious world of computers.
Thanks for your attention anyway!


Syncing of live grabbed video and sound