Forums > Jitter

Quartz Composer Screen Capture into jit.matrix

November 13, 2008 | 2:36 pm

Hi, I’m looking for a cpu efficent way of streaming video from an external video preview window into a jit.matrix (for native processing). So far the best solution seems to be using vade’s GPU accelerated screen capture plugin for Quartz Composer accessed through DIPS ( http://dips.dacreation.com/ ) or Remy Muller’s quartzplayer ( http://sourceforge.net/projects/quartzplayer/ ).

I compiled a working external of quartzplayer from the SVN ( http://quartzplayer.svn.sourceforge.net/viewvc/quartzplayer/trunk.tar.gz?view=tar ), but I can’t figure out how to tweak the code to become compatible with Max5.

Anyway, using Max4.6 I trying to find out how to perform a readback from VRAM to RAM for native processing in a jit,matrix. I’d be very interested in hearing about any possible solutions for achieving this, maybe by doing something similar to using glreadpixels with jit.gl.sketch or (in Max5) jit.gl.asyncread.

Alternatively a port of Vade’s QC plugin would circumvent all what’s mentioned above. Or, maybe someone would be interested in tweaking the code of quartzplayer for Max5 compatibility and "glreadpixels"/readback-functionality?

Thanks!


November 13, 2008 | 4:11 pm

i am also intensely interested in this.

On Nov 13, 2008, at 9:36 AM, Stig Holte wrote:

>
> Hi, I’m looking for a cpu efficent way of streaming video from an
> external video preview window into a jit.matrix (for native
> processing). So far the best solution seems to be using vade’s GPU
> accelerated screen capture plugin for Quartz Composer accessed
> through DIPS ( http://dips.dacreation.com/ ) or Remy Muller’s
> quartzplayer ( http://sourceforge.net/projects/quartzplayer/ ).
>
> I compiled a working external of quartzplayer from the SVN ( http://quartzplayer.svn.sourceforge.net/viewvc/quartzplayer/trunk.tar.gz?view=tar
> ), but I can’t figure out how to tweak the code to become
> compatible with Max5.
>
> Anyway, using Max4.6 I trying to find out how to perform a readback
> from VRAM to RAM for native processing in a jit,matrix. I’d be very
> interested in hearing about any possible solutions for achieving
> this, maybe by doing something similar to using glreadpixels with
> jit.gl.sketch or (in Max5) jit.gl.asyncread.
>
> Alternatively a port of Vade’s QC plugin would circumvent all what’s
> mentioned above. Or, maybe someone would be interested in tweaking
> the code of quartzplayer for Max5 compatibility and "glreadpixels"/
> readback-functionality?
>
> Thanks!


November 14, 2008 | 11:18 pm

quartzplayer? Is it compatible with jitter?


November 15, 2008 | 1:44 pm

Quote: Kyred wrote on Fri, 14 November 2008 16:18
—————————————————-
> quartzplayer? Is it compatible with jitter?
it does not seem so, the quartzplayer is rendering in a memoryspace which I’m unable to access from Jitter.

For anyone who’d like to look into it then download the attached zip – containing quartzplayer.mxo with source (compiled from the SVN) and a patch tested with MaxMSP 4.6.3.


November 15, 2008 | 1:57 pm

Nice, thank you.


November 18, 2008 | 10:06 am

Quote: Kyred wrote on Sat, 15 November 2008 06:57
—————————————————-
> Nice, thank you.
—————————————————-
thanks for the interest, and to the developers at c74: any plans for tighter intergration of Quartz in Max5 in form of a separate QC host external – or does playing Quartz Compositions through jit.qt.movie suffice?

Is vade’s screen capture plugin one of few cases where a QC host external would be useful, or are there many other occurrences of incompatibilty when using quartz compositions in jit.qt.movie?


November 18, 2008 | 9:38 pm

Well, something like an accelerated jit.desktop would allow captureing/integration from more that QC, IE, anything you can see on the desktop.

However, it looks like the DIPS solution is quite nice, as it allows one to play the file.

jit.qt.movie, does not to my knowledge use an accelerated ‘gworld’ or whatnot, which means QC comps loaded by jit.qt.movie are not GPU accelerated, which means it punts to the software renderer and ends up being slow. I *think* that is right, but dont swear on it.

Both would be very useful I think.


November 19, 2008 | 10:14 am

Quote: vade wrote on Tue, 18 November 2008 14:38
—————————————————-
> Well, something like an accelerated jit.desktop would
> allow captureing/integration from more that QC, IE,
> anything you can see on the desktop.
> jit.qt.movie, does not to my knowledge use an
> accelerated ‘gworld’ or whatnot, which means QC comps
> loaded by jit.qt.movie are not GPU accelerated, which
> means it punts to the software renderer and ends up being
> slow. I *think* that is right, but dont swear on it.
—————————————————-
Hi, thanks for clarifying! So far a potential "jit.gpu_accelerated.desktop" sounds like the best solution when it comes to latency and efficency.

A question to the developers at c74: have you considered developing (/porting) a GPU accelereated version of jit.desktop? It would really open up Jitter in terms of connecting with other video applications.

This connectivity be expanded even further by virtual video drivers. This concept is already implemented in OS X: the "Digital Cinema Desktop" (distributed with Final Cut) is accessible as a virtual video driver for desktop preview. If Apple (or someone else) just could make a virtual input driver with a corresponding output driver, then "videoflower" would become reality. Is this technically feasible?


November 19, 2008 | 7:32 pm

I’ve had this on my to-do list for about three months now. Maybe I’ll
get it done over Christmas.

bt

On Nov 19, 2008, at 2:14 AM, Stig wrote:

>
> Quote: vade wrote on Tue, 18 November 2008 14:38
> —————————————————-
>> Well, something like an accelerated jit.desktop would
>> allow captureing/integration from more that QC, IE,
>> anything you can see on the desktop.
>> jit.qt.movie, does not to my knowledge use an
>> accelerated ‘gworld’ or whatnot, which means QC comps
>> loaded by jit.qt.movie are not GPU accelerated, which
>> means it punts to the software renderer and ends up being
>> slow. I *think* that is right, but dont swear on it.
> —————————————————-
> Hi, thanks for clarifying! So far a potential
> "jit.gpu_accelerated.desktop" sounds like the best solution when it
> comes to latency and efficency.
>
> A question to the developers at c74: have you considered developing
> (/porting) a GPU accelereated version of jit.desktop? It would
> really open up Jitter in terms of connecting with other video
> applications.
>
> This connectivity be expanded even further by virtual video drivers.
> This concept is already implemented in OS X: the "Digital Cinema
> Desktop" (distributed with Final Cut) is accessible as a virtual
> video driver for desktop preview. If Apple (or someone else) just
> could make a virtual input driver with a corresponding output
> driver, then "videoflower" would become reality. Is this technically
> feasible?

barry threw
Media Art and Technology

San Francisco, CA
Work: 857-544-3967
Email: bthrew (at) gmail (dot) com
Web: http://www.barrythrew.com


November 19, 2008 | 7:47 pm

> This connectivity be expanded even further by virtual video drivers. This concept is already implemented in OS X: the "Digital Cinema Desktop" (distributed with Final Cut) is accessible as a virtual video driver for desktop preview. If Apple (or someone else) just could make a virtual input driver with a corresponding output driver, then "videoflower" would become reality. Is this technically feasible?
—————————————————-

I believe that ‘Cinema Desktop’ just uh, plays the final cut output to a full screen window. I dont think there is any magic behind it. Where did you find this information?


November 19, 2008 | 8:29 pm

There are examples of this in the developer tools. Have at it
grasshopper.
This has also been on the todo list for awhile, but delves more into
the realm of the unknown for me.

bt

On Nov 19, 2008, at 2:14 AM, Stig wrote:

> This connectivity be expanded even further by virtual video drivers.
> This concept is already implemented in OS X: the "Digital Cinema
> Desktop" (distributed with Final Cut) is accessible as a virtual
> video driver for desktop preview. If Apple (or someone else) just
> could make a virtual input driver with a corresponding output
> driver, then "videoflower" would become reality. Is this technically
> feasible?

barry threw
Media Art and Technology

San Francisco, CA
Work: 857-544-3967
Email: bthrew (at) gmail (dot) com
Web: http://www.barrythrew.com


November 20, 2008 | 1:08 pm

Quote: vade wrote on Wed, 19 November 2008 12:47
> —————————————————-
> I believe that ‘Cinema Desktop’ just uh, plays the final cut output to a full screen window. I dont think there is any magic behind it. Where did you find this information?
—————————————————-
There are hints to this in the "FireWire Device Interface Guide" at
http://developer.apple.com/documentation/HardwareDrivers/FireWire-date.html

There are some source code examples as well for creating virtual video input and output devices at http://developer.apple.com/ such as SoftVDigX and SoftVideoOutputComponent.

Searching for Virtual FireWire Device at http://lists.apple.com/ also provide some answers.

These findings stems from searching for ways to transfer video from external apps into Jitter. I have yet to find a working solution for my needs, nor do I have the necessary programming skills or experience to develop this myself, so I humbly present my findings in hope that some of the learned users of this forum might want to delve into it.

Quote: barry threw wrote on Wed, 19 November 2008 12:32
—————————————————-
> I’ve had this on my to-do list for about three months now. Maybe I’ll
> get it done over Christmas.
—————————————————-
And kudos to Barry for taking the task of porting vade’s screen capture.

-stig


November 26, 2008 | 3:07 pm

hi,
is it possible to get the glcontext name out of the quartzplayer object ? shouldn


January 29, 2009 | 5:51 pm

I can’t launch "quartzplayer.help"

Max win tells:
• error: quartzplayer: unable to load object bundle executable
2009-01-29 18:48:38.804 MaxMSP[433] CFLog (21): dyld returns 2 when trying to load /Users/me/Desktop/quartzplayer_screen_capture/quartzplayer.mxo/Contents/MacOS/maximum

I’m on PB G4, OSX 4 Tiger
Max 4.6.3 jitter 1.6.3

Where am I wrong?


Viewing 14 posts - 1 through 14 (of 14 total)