lists@lowfrequency.or

So that's the question - I'd really like to make my own version of jit.qt.movie, but I'm having trouble conceptualizing where the movie data becomes max-land matrix data. Or possibly even jit.gl.texture data. I don't really know if any of this is possible, but a movie-direct-to-texture object would be very useful, as would a stripped-down jit.qt.movie.

I know you guys are probably super-busy with Max 5 and all that, but I'd really appreciate any advice on this subject. Perhaps I should just do this all using Cocoa / Objective C / QT?

So that's the question - I'd really like to make my own version of jit.qt.movie, but I'm having trouble conceptualizing where the movie data becomes max-land matrix data.  Or possibly even jit.gl.texture data.  I don't really know if any of this is possible, but a movie-direct-to-texture object would be very useful, as would a stripped-down jit.qt.movie.

I know you guys are probably super-busy with Max 5 and all that, but I'd really appreciate any advice on this subject.  Perhaps I should just do this all using Cocoa / Objective C / QT?

could-i-write-my-own-jit_qt_movie_matrix_to_image

Nuts, didn't read properly before I posted. I don't mean "jit_qt_movie_matrix_to_image" because that exports a movie to an image - I meant something more along the lines of taking a quicktime movie file, reading it, decompressing it, possibly changing it, and transferring it to jitter for further processing. Forgive the hasty initial post, please.

Nuts, didn't read properly before I posted.  I don't mean "jit_qt_movie_matrix_to_image" because that exports a movie to an image - I meant something more along the lines of taking a quicktime movie file, reading it, decompressing it, possibly changing it, and transferring it to jitter for further processing.  Forgive the hasty initial post, please.


> So that's the question - I'd really like to make my own version of 

> jit.qt.movie, but I'm having trouble conceptualizing where the 

> movie data becomes max-land matrix data. Or possibly even 

> jit.gl.texture data. I don't really know if any of this is 

> possible, but a movie-direct-to-texture object would be very 

> useful, as would a stripped-down jit.qt.movie.

> I know you guys are probably super-busy with Max 5 and all that, 

> but I'd really appreciate any advice on this subject. Perhaps I 

> should just do this all using Cocoa / Objective C / QT?

It is possible (movie-direct-to-texture), but it is a _lot of work. 

Writing your own movie-to-matrix object probably wouldn't give you 

much of a win over what is currently available from jit.qt.movie. If 

you want a stripped down version, couldn't you just use the [movie] 

On 07-10-17, at 0635, evan.raskob wrote:
>
> So that's the question - I'd really like to make my own version of  
> jit.qt.movie, but I'm having trouble conceptualizing where the  
> movie data becomes max-land matrix data.  Or possibly even  
> jit.gl.texture data.  I don't really know if any of this is  
> possible, but a movie-direct-to-texture object would be very  
> useful, as would a stripped-down jit.qt.movie.
>
> I know you guys are probably super-busy with Max 5 and all that,  
> but I'd really appreciate any advice on this subject.  Perhaps I  
> should just do this all using Cocoa / Objective C / QT?
>
It is possible (movie-direct-to-texture), but it is a _lot of work.  
Writing your own movie-to-matrix object probably wouldn't give you  
much of a win over what is currently available from jit.qt.movie. If  
you want a stripped down version, couldn't you just use the [movie]  
Max object?

Could you please elaborate more? What I'm thinking has to do with multi-threading, something which I don't think jit.qt.movie does (because multi-threading is mainly a 10.4 thing), but I'd love to be proved wrong about that. The theory is that this might get around frame skipping issues that jit.qt.movie has right now. But again, if I'm wrong, I'd really like to know.

Could you please elaborate more? What I'm thinking has to do with multi-threading, something which I don't think jit.qt.movie does (because multi-threading is mainly a 10.4 thing), but I'd love to be proved wrong about that.  The theory is that this might get around frame skipping issues that jit.qt.movie has right now.  But again, if I'm wrong, I'd really like to know.

> Could you please elaborate more? What I'm thinking has to do with 

> multi-threading, something which I don't think jit.qt.movie does 

> (because multi-threading is mainly a 10.4 thing), but I'd love to 

> be proved wrong about that. The theory is that this might get 

> around frame skipping issues that jit.qt.movie has right now. But 

> again, if I'm wrong, I'd really like to know.

As far as I know QuickTime is not yet multithreaded for movie 

playback, so threading won't help too much to prevent skipping. You 

can thread some operations so that loading a 2nd movie (for instance) 

doesn't affect an already playing one, but as for eliminating skips 

completely I haven't yet found a solution.

Consider looking at the CoreVideo stuff, specifically the 

LiveVideoMixer sample code from Apple, which details how to set up a 

CVDisplayLink to drive movie playback from the video card.

On 07-10-17, at 0954, evan.raskob wrote:
>
> Could you please elaborate more? What I'm thinking has to do with  
> multi-threading, something which I don't think jit.qt.movie does  
> (because multi-threading is mainly a 10.4 thing), but I'd love to  
> be proved wrong about that.  The theory is that this might get  
> around frame skipping issues that jit.qt.movie has right now.  But  
> again, if I'm wrong, I'd really like to know.
>
As far as I know QuickTime is not yet multithreaded for movie  
playback, so threading won't help too much to prevent skipping. You  
can thread some operations so that loading a 2nd movie (for instance)  
doesn't affect an already playing one, but as for eliminating skips  
completely I haven't yet found a solution.

Consider looking at the CoreVideo stuff, specifically the  
LiveVideoMixer sample code from Apple, which details how to set up a  
CVDisplayLink to drive movie playback from the video card.

http://developer.apple.com/technotes/tn/tn2125.html

"ThreadsImportMovie demonstrates importing and displaying QuickTime Movies on separate threads."

but how to hook this up to max? i'm having trouble figuring out how to go about this.

and this opens the possibility of opening up multiple movies on different threads, i think?

yes, especially: http://developer.apple.com/technotes/tn/tn2125.html

but how to hook this up to max? i'm having trouble figuring out how to go about this.  

and this opens the possibility of opening up multiple movies on different threads, i think? 

> "ThreadsImportMovie demonstrates importing and displaying QuickTime 

> but how to hook this up to max? i'm having trouble figuring out how 

> and this opens the possibility of opening up multiple movies on 

http://developer.apple.com/technotes/tn/tn2125.html#TNTAG3

"The Movie Toolbox provides the functionality that allows you to 

import, play, render, create, edit, and store time-based data. Movies 

cannot be played on background threads, therefore calling Movie 

Toolbox playback APIs such as StartMovie or SetMovieRate should only 

Opening movies on different threads is possible (although not for all 

codecs) but this will not address your frame dropping issue. Movie 

playback must happen on the main (UI) thread.

This is not a straightforward project you are undertaking. I would 

recommend that you really familiarize yourself with both jit.gl.* 

sample code and the Apple sample code before you embark. If you have 

troubles understanding any of the sample code you'll have a lot of 

trouble. If you _do understand all of the sample code, then you 

should be able to figure out how to hook it all up. Good luck =)

On 07-10-17, at 1409, evan.raskob wrote:
>
> yes, especially: http://developer.apple.com/technotes/tn/tn2125.html
>
> "ThreadsImportMovie demonstrates importing and displaying QuickTime  
> Movies on separate threads."
>
> but how to hook this up to max? i'm having trouble figuring out how  
> to go about this.
>
> and this opens the possibility of opening up multiple movies on  
> different threads, i think?
>
http://developer.apple.com/technotes/tn/tn2125.html#TNTAG3

"The Movie Toolbox provides the functionality that allows you to  
import, play, render, create, edit, and store time-based data. Movies  
cannot be played on background threads, therefore calling Movie  
Toolbox playback APIs such as StartMovie or SetMovieRate should only  
be done from the main thread."

Opening movies on different threads is possible (although not for all  
codecs) but this will not address your frame dropping issue. Movie  
playback must happen on the main (UI) thread.

This is not a straightforward project you are undertaking. I would  
recommend that you really familiarize yourself with both jit.gl.*  
sample code and the Apple sample code before you embark. If you have  
troubles understanding any of the sample code you'll have a lot of  
trouble. If you _do understand all of the sample code, then you  
should be able to figure out how to hook it all up. Good luck =)

On Oct 17, 2007, at 10:26 PM, Ritchie Argue wrote:

> On 07-10-17, at 1409, evan.raskob wrote:

>> "ThreadsImportMovie demonstrates importing and displaying 

>> QuickTime Movies on separate threads."

>> but how to hook this up to max? i'm having trouble figuring out 

>> and this opens the possibility of opening up multiple movies on 

> "The Movie Toolbox provides the functionality that allows you to 

> import, play, render, create, edit, and store time-based data. 

> Movies cannot be played on background threads, therefore calling 

> Movie Toolbox playback APIs such as StartMovie or SetMovieRate 

> should only be done from the main thread."

> Opening movies on different threads is possible (although not for 

> all codecs) but this will not address your frame dropping issue. 

> Movie playback must happen on the main (UI) thread.

I see. That makes sense. Still, I can't be sure where the frame 

dropping occurs - is it in the jit.window at display time, or are 

frames lost in the pipeline on the way there? Can I do some caching 

at some point to get around this? I searched a bit harder and found 

this page that seems to spell out a possible method for caching 

https://cycling74.com/twiki/bin/view/ProductDocumentation/

But the more I look at this, the more I think its the scheduler 

itself that is the limitation. You can see similar effects with 

metro skipping beats (there was a thread on this awhile back). I'd 

need to go with the apple examples and forego max/jitter in order to 

> This is not a straightforward project you are undertaking. I would 

> recommend that you really familiarize yourself with both jit.gl.* 

> sample code and the Apple sample code before you embark. If you 

> have troubles understanding any of the sample code you'll have a 

> lot of trouble. If you _do understand all of the sample code, then 

> you should be able to figure out how to hook it all up. Good luck =)

Yes, certainly not straightforward... but now that I found the online 

jitter API documentation (I didn't realize it was separate from the 

max universal binary docs) I have a lot to work with. Still, it 

would be interesting to see some snippets of what jit.qt.movie 

> On 07-10-17, at 1409, evan.raskob wrote:
>>
>> yes, especially: http://developer.apple.com/technotes/tn/tn2125.html
>>
>> "ThreadsImportMovie demonstrates importing and displaying  
>> QuickTime Movies on separate threads."
>>
>> but how to hook this up to max? i'm having trouble figuring out  
>> how to go about this.
>>
>> and this opens the possibility of opening up multiple movies on  
>> different threads, i think?
>>
> http://developer.apple.com/technotes/tn/tn2125.html#TNTAG3
>
> "The Movie Toolbox provides the functionality that allows you to  
> import, play, render, create, edit, and store time-based data.  
> Movies cannot be played on background threads, therefore calling  
> Movie Toolbox playback APIs such as StartMovie or SetMovieRate  
> should only be done from the main thread."

> Opening movies on different threads is possible (although not for  
> all codecs) but this will not address your frame dropping issue.  
> Movie playback must happen on the main (UI) thread.

I see.  That makes sense.  Still, I can't be sure where the frame  
dropping occurs - is it in the jit.window at display time, or are  
frames lost in the pipeline on the way there? Can I do some caching  
at some point to get around this?  I searched a bit harder and found  
this page that seems to spell out a possible method for caching  
ahead: https://cycling74.com/twiki/bin/view/ProductDocumentation/ 
JitterSdkUsingQTObjects

But the more I look at this, the more I think its the scheduler  
itself that is the limitation.  You can see similar effects with  
metro skipping beats (there was a thread on this awhile back).  I'd  
need to go with the apple examples and forego max/jitter in order to  
get truly seamless playback, it seems.

> This is not a straightforward project you are undertaking. I would  
> recommend that you really familiarize yourself with both jit.gl.*  
> sample code and the Apple sample code before you embark. If you  
> have troubles understanding any of the sample code you'll have a  
> lot of trouble. If you _do understand all of the sample code, then  
> you should be able to figure out how to hook it all up. Good luck =)
>
> r.

Yes, certainly not straightforward... but now that I found the online  
jitter API documentation (I didn't realize it was separate from the  
max universal binary docs) I have a lot to work with.  Still, it  
would be interesting to see some snippets of what jit.qt.movie  
actually does under the hood.

On 07-10-17, at 1625, evan.raskob [lists] wrote:

> I see. That makes sense. Still, I can't be sure where the frame 

> dropping occurs - is it in the jit.window at display time, or are 

> frames lost in the pipeline on the way there? Can I do some caching 

> at some point to get around this? I searched a bit harder and 

> found this page that seems to spell out a possible method for 

> ProductDocumentation/JitterSdkUsingQTObjects

You could try the loadram argument to jit.qt.movie to see if that 

helps any. That's about as much as you can hope for in terms of 

caching QT, as it does a lot of caching and preloading behind the 

scenes already. Look at the various threads regarding prerolling 

movies, what that does, and how easy it is to accidentally invalidate 

> But the more I look at this, the more I think its the scheduler 

> itself that is the limitation. You can see similar effects with 

> metro skipping beats (there was a thread on this awhile back). I'd 

> need to go with the apple examples and forego max/jitter in order 

> to get truly seamless playback, it seems.

I would recommend rendering a really obvious test clip that 

demonstrates the smoothness you are looking for, and play it back in 

some of the sample Apple QT projects. I've found even outside of Max, 

it's frustratingly difficult to never drop frames. This is just a 

limitation of the multitasking nature of consumer OSs these days. If 

you need guaranteed playback, something like realtime linux might be 

a better place to start (though good luck finding a decent media 

On 07-10-17, at 1625, evan.raskob [lists] wrote:
>
> I see.  That makes sense.  Still, I can't be sure where the frame  
> dropping occurs - is it in the jit.window at display time, or are  
> frames lost in the pipeline on the way there? Can I do some caching  
> at some point to get around this?  I searched a bit harder and  
> found this page that seems to spell out a possible method for  
> caching ahead: https://cycling74.com/twiki/bin/view/ 
> ProductDocumentation/JitterSdkUsingQTObjects
>
You could try the loadram argument to jit.qt.movie to see if that  
helps any. That's about as much as you can hope for in terms of  
caching QT, as it does a lot of caching and preloading behind the  
scenes already. Look at the various threads regarding prerolling  
movies, what that does, and how easy it is to accidentally invalidate  
the effects of prerolling.

> But the more I look at this, the more I think its the scheduler  
> itself that is the limitation.  You can see similar effects with  
> metro skipping beats (there was a thread on this awhile back).  I'd  
> need to go with the apple examples and forego max/jitter in order  
> to get truly seamless playback, it seems.
>
I would recommend rendering a really obvious test clip that  
demonstrates the smoothness you are looking for, and play it back in  
some of the sample Apple QT projects. I've found even outside of Max,  
it's frustratingly difficult to never drop frames. This is just a  
limitation of the multitasking nature of consumer OSs these days. If  
you need guaranteed playback, something like realtime linux might be  
a better place to start (though good luck finding a decent media  
framework there).

On Oct 18, 2007, at 12:32 AM, Ritchie Argue wrote:

> On 07-10-17, at 1625, evan.raskob [lists] wrote:

>> I see. That makes sense. Still, I can't be sure where the frame 

>> dropping occurs - is it in the jit.window at display time, or are 

>> frames lost in the pipeline on the way there? Can I do some 

>> caching at some point to get around this? I searched a bit harder 

>> and found this page that seems to spell out a possible method for 

>> ProductDocumentation/JitterSdkUsingQTObjects

> You could try the loadram argument to jit.qt.movie to see if that 

> helps any. That's about as much as you can hope for in terms of 

> caching QT, as it does a lot of caching and preloading behind the 

> scenes already. Look at the various threads regarding prerolling 

> movies, what that does, and how easy it is to accidentally 

Yes, tried a few variations (and realized just what you're saying 

about how easy it is to invalidate those prerolling effects!). Still 

>> But the more I look at this, the more I think its the scheduler 

>> itself that is the limitation. You can see similar effects with 

>> metro skipping beats (there was a thread on this awhile back). 

>> I'd need to go with the apple examples and forego max/jitter in 

>> order to get truly seamless playback, it seems.

> I would recommend rendering a really obvious test clip that 

> demonstrates the smoothness you are looking for, and play it back 

> in some of the sample Apple QT projects. I've found even outside of 

> Max, it's frustratingly difficult to never drop frames. This is 

> just a limitation of the multitasking nature of consumer OSs these 

> days. If you need guaranteed playback, something like realtime 

> linux might be a better place to start (though good luck finding a 

I have a very smooth test clip in full PAL DVD resolution/frame rate, 

compressed in photojpeg, played back off a FW800 RAID drive on an 8- 

core Mac pro tower with 16GB RAM, and I see occasional skips in Max/ 

Jitter but NOT in the various apple QT example applications (all the 

ones where they show you basic video effects) and quicktime player. 

http://developer.apple.com/samplecode/QuickTime/

idxMovieBasics-date.html - QTCarbonCoreImage101 is an interesting one 

I completely agree that its frustratingly difficult to never drop 

frames, but now I'm trying to overcome that frustration :) There 

must be some way. Even if it's difficult or near-impossible I'd 

still like to know what the possibilities are!

> On 07-10-17, at 1625, evan.raskob [lists] wrote:
>>
>> I see.  That makes sense.  Still, I can't be sure where the frame  
>> dropping occurs - is it in the jit.window at display time, or are  
>> frames lost in the pipeline on the way there? Can I do some  
>> caching at some point to get around this?  I searched a bit harder  
>> and found this page that seems to spell out a possible method for  
>> caching ahead: https://cycling74.com/twiki/bin/view/ 
>> ProductDocumentation/JitterSdkUsingQTObjects
>>
> You could try the loadram argument to jit.qt.movie to see if that  
> helps any. That's about as much as you can hope for in terms of  
> caching QT, as it does a lot of caching and preloading behind the  
> scenes already. Look at the various threads regarding prerolling  
> movies, what that does, and how easy it is to accidentally  
> invalidate the effects of prerolling.

Yes, tried a few variations (and realized just what you're saying  
about how easy it is to invalidate those prerolling effects!).  Still  
skippage.

>> But the more I look at this, the more I think its the scheduler  
>> itself that is the limitation.  You can see similar effects with  
>> metro skipping beats (there was a thread on this awhile back).   
>> I'd need to go with the apple examples and forego max/jitter in  
>> order to get truly seamless playback, it seems.
>>
> I would recommend rendering a really obvious test clip that  
> demonstrates the smoothness you are looking for, and play it back  
> in some of the sample Apple QT projects. I've found even outside of  
> Max, it's frustratingly difficult to never drop frames. This is  
> just a limitation of the multitasking nature of consumer OSs these  
> days. If you need guaranteed playback, something like realtime  
> linux might be a better place to start (though good luck finding a  
> decent media framework there).

I have a very smooth test clip in full PAL DVD resolution/frame rate,  
compressed in photojpeg, played back off a FW800 RAID drive on an 8- 
core Mac pro tower with 16GB RAM, and I see occasional skips in Max/ 
Jitter but NOT in the various apple QT example applications (all the  
ones where they show you basic video effects) and quicktime player.  
(Examples from http://developer.apple.com/samplecode/QuickTime/ 
idxMovieBasics-date.html - QTCarbonCoreImage101 is an interesting one  
that uses some overlaid effects)

I completely agree that its frustratingly difficult to never drop  
frames, but now I'm trying to overcome that frustration :)  There  
must be some way.  Even if it's difficult or near-impossible I'd  
still like to know what the possibilities are!

my suggestion (which might be what you're already thinking) is to use corevideo to drive the video (and not use any of jitters-qt objects). let the external have its own timing mechanism and just render the video as an opengl texture, allowing it to use the opengl context of gl.render. i can send you the basics of retrieving the aglcontext if you need.

the livevideomixer seems like a great place to start, and shouldn't take much work to test out the theory.

if you haven't already, just build that example and see if it actually does handle the video playback better than jit-qt.

my suggestion (which might be what you're already thinking) is to use corevideo to drive the video (and not use any of jitters-qt objects).  let the external have its own timing mechanism and just render the video as an opengl texture, allowing it to use the opengl context of gl.render. i can send you the basics of retrieving the aglcontext if you need.

the livevideomixer seems like a great place to start, and shouldn't take much work to test out the theory.
if you haven't already, just build that example and see if it actually does handle the video playback better than jit-qt.
-rob


could I write my own jit_qt_movie_matrix_to_image?