Quartz Composer Screen Capture into jit.matrix

Nov 13, 2008 at 2:36pm

Quartz Composer Screen Capture into jit.matrix

Hi, I’m looking for a cpu efficent way of streaming video from an external video preview window into a jit.matrix (for native processing). So far the best solution seems to be using vade’s GPU accelerated screen capture plugin for Quartz Composer accessed through DIPS ( http://dips.dacreation.com/ ) or Remy Muller’s quartzplayer ( http://sourceforge.net/projects/quartzplayer/ ).

I compiled a working external of quartzplayer from the SVN ( http://quartzplayer.svn.sourceforge.net/viewvc/quartzplayer/trunk.tar.gz?view=tar ), but I can’t figure out how to tweak the code to become compatible with Max5.

Anyway, using Max4.6 I trying to find out how to perform a readback from VRAM to RAM for native processing in a jit,matrix. I’d be very interested in hearing about any possible solutions for achieving this, maybe by doing something similar to using glreadpixels with jit.gl.sketch or (in Max5) jit.gl.asyncread.

Alternatively a port of Vade’s QC plugin would circumvent all what’s mentioned above. Or, maybe someone would be interested in tweaking the code of quartzplayer for Max5 compatibility and “glreadpixels”/readback-functionality?

Thanks!

#40827
Nov 13, 2008 at 4:11pm

i am also intensely interested in this.

On Nov 13, 2008, at 9:36 AM, Stig Holte wrote:

>
> Hi, I’m looking for a cpu efficent way of streaming video from an
> external video preview window into a jit.matrix (for native
> processing). So far the best solution seems to be using vade’s GPU
> accelerated screen capture plugin for Quartz Composer accessed
> through DIPS ( http://dips.dacreation.com/ ) or Remy Muller’s
> quartzplayer ( http://sourceforge.net/projects/quartzplayer/ ).
>
> I compiled a working external of quartzplayer from the SVN ( http://quartzplayer.svn.sourceforge.net/viewvc/quartzplayer/trunk.tar.gz?view=tar
> ), but I can’t figure out how to tweak the code to become
> compatible with Max5.
>
> Anyway, using Max4.6 I trying to find out how to perform a readback
> from VRAM to RAM for native processing in a jit,matrix. I’d be very
> interested in hearing about any possible solutions for achieving
> this, maybe by doing something similar to using glreadpixels with
> jit.gl.sketch or (in Max5) jit.gl.asyncread.
>
> Alternatively a port of Vade’s QC plugin would circumvent all what’s
> mentioned above. Or, maybe someone would be interested in tweaking
> the code of quartzplayer for Max5 compatibility and “glreadpixels”/
> readback-functionality?
>
> Thanks!

#144864
Nov 14, 2008 at 11:18pm

quartzplayer? Is it compatible with jitter?

#144865
Nov 15, 2008 at 1:44pm

Quote: Kyred wrote on Fri, 14 November 2008 16:18
—————————————————-
> quartzplayer? Is it compatible with jitter?
it does not seem so, the quartzplayer is rendering in a memoryspace which I’m unable to access from Jitter.

For anyone who’d like to look into it then download the attached zip – containing quartzplayer.mxo with source (compiled from the SVN) and a patch tested with MaxMSP 4.6.3.

#144866
Nov 15, 2008 at 1:57pm

Nice, thank you.

#144867
Nov 18, 2008 at 10:06am

Quote: Kyred wrote on Sat, 15 November 2008 06:57
—————————————————-
> Nice, thank you.
—————————————————-
thanks for the interest, and to the developers at c74: any plans for tighter intergration of Quartz in Max5 in form of a separate QC host external – or does playing Quartz Compositions through jit.qt.movie suffice?

Is vade’s screen capture plugin one of few cases where a QC host external would be useful, or are there many other occurrences of incompatibilty when using quartz compositions in jit.qt.movie?

#144868
Nov 18, 2008 at 9:38pm

Well, something like an accelerated jit.desktop would allow captureing/integration from more that QC, IE, anything you can see on the desktop.

However, it looks like the DIPS solution is quite nice, as it allows one to play the file.

jit.qt.movie, does not to my knowledge use an accelerated ‘gworld’ or whatnot, which means QC comps loaded by jit.qt.movie are not GPU accelerated, which means it punts to the software renderer and ends up being slow. I *think* that is right, but dont swear on it.

Both would be very useful I think.

#144869
Nov 19, 2008 at 10:14am

Quote: vade wrote on Tue, 18 November 2008 14:38
—————————————————-
> Well, something like an accelerated jit.desktop would
> allow captureing/integration from more that QC, IE,
> anything you can see on the desktop.
> jit.qt.movie, does not to my knowledge use an
> accelerated ‘gworld’ or whatnot, which means QC comps
> loaded by jit.qt.movie are not GPU accelerated, which
> means it punts to the software renderer and ends up being
> slow. I *think* that is right, but dont swear on it.
—————————————————-
Hi, thanks for clarifying! So far a potential “jit.gpu_accelerated.desktop” sounds like the best solution when it comes to latency and efficency.

A question to the developers at c74: have you considered developing (/porting) a GPU accelereated version of jit.desktop? It would really open up Jitter in terms of connecting with other video applications.

This connectivity be expanded even further by virtual video drivers. This concept is already implemented in OS X: the “Digital Cinema Desktop” (distributed with Final Cut) is accessible as a virtual video driver for desktop preview. If Apple (or someone else) just could make a virtual input driver with a corresponding output driver, then “videoflower” would become reality. Is this technically feasible?

#144870
Nov 19, 2008 at 7:32pm

I’ve had this on my to-do list for about three months now. Maybe I’ll
get it done over Christmas.

bt

On Nov 19, 2008, at 2:14 AM, Stig wrote:

>
> Quote: vade wrote on Tue, 18 November 2008 14:38
> —————————————————-
>> Well, something like an accelerated jit.desktop would
>> allow captureing/integration from more that QC, IE,
>> anything you can see on the desktop.
>> jit.qt.movie, does not to my knowledge use an
>> accelerated ‘gworld’ or whatnot, which means QC comps
>> loaded by jit.qt.movie are not GPU accelerated, which
>> means it punts to the software renderer and ends up being
>> slow. I *think* that is right, but dont swear on it.
> —————————————————-
> Hi, thanks for clarifying! So far a potential
> “jit.gpu_accelerated.desktop” sounds like the best solution when it
> comes to latency and efficency.
>
> A question to the developers at c74: have you considered developing
> (/porting) a GPU accelereated version of jit.desktop? It would
> really open up Jitter in terms of connecting with other video
> applications.
>
> This connectivity be expanded even further by virtual video drivers.
> This concept is already implemented in OS X: the “Digital Cinema
> Desktop” (distributed with Final Cut) is accessible as a virtual
> video driver for desktop preview. If Apple (or someone else) just
> could make a virtual input driver with a corresponding output
> driver, then “videoflower” would become reality. Is this technically
> feasible?

barry threw
Media Art and Technology

San Francisco, CA
Work: 857-544-3967
Email: bthrew (at) gmail (dot) com
Web: http://www.barrythrew.com

#144871
Nov 19, 2008 at 7:47pm

> This connectivity be expanded even further by virtual video drivers. This concept is already implemented in OS X: the “Digital Cinema Desktop” (distributed with Final Cut) is accessible as a virtual video driver for desktop preview. If Apple (or someone else) just could make a virtual input driver with a corresponding output driver, then “videoflower” would become reality. Is this technically feasible?
—————————————————-

I believe that ‘Cinema Desktop’ just uh, plays the final cut output to a full screen window. I dont think there is any magic behind it. Where did you find this information?

#144872
Nov 19, 2008 at 8:29pm

There are examples of this in the developer tools. Have at it
grasshopper.
This has also been on the todo list for awhile, but delves more into
the realm of the unknown for me.

bt

On Nov 19, 2008, at 2:14 AM, Stig wrote:

> This connectivity be expanded even further by virtual video drivers.
> This concept is already implemented in OS X: the “Digital Cinema
> Desktop” (distributed with Final Cut) is accessible as a virtual
> video driver for desktop preview. If Apple (or someone else) just
> could make a virtual input driver with a corresponding output
> driver, then “videoflower” would become reality. Is this technically
> feasible?

barry threw
Media Art and Technology

San Francisco, CA
Work: 857-544-3967
Email: bthrew (at) gmail (dot) com
Web: http://www.barrythrew.com

#144873
Nov 20, 2008 at 1:08pm

Quote: vade wrote on Wed, 19 November 2008 12:47
> —————————————————-
> I believe that ‘Cinema Desktop’ just uh, plays the final cut output to a full screen window. I dont think there is any magic behind it. Where did you find this information?
—————————————————-
There are hints to this in the “FireWire Device Interface Guide” at
http://developer.apple.com/documentation/HardwareDrivers/FireWire-date.html

There are some source code examples as well for creating virtual video input and output devices at http://developer.apple.com/ such as SoftVDigX and SoftVideoOutputComponent.

Searching for Virtual FireWire Device at http://lists.apple.com/ also provide some answers.

These findings stems from searching for ways to transfer video from external apps into Jitter. I have yet to find a working solution for my needs, nor do I have the necessary programming skills or experience to develop this myself, so I humbly present my findings in hope that some of the learned users of this forum might want to delve into it.

Quote: barry threw wrote on Wed, 19 November 2008 12:32
—————————————————-
> I’ve had this on my to-do list for about three months now. Maybe I’ll
> get it done over Christmas.
—————————————————-
And kudos to Barry for taking the task of porting vade’s screen capture.

-stig

#144874
Nov 26, 2008 at 3:07pm

hi,
is it possible to get the glcontext name out of the quartzplayer object ? shouldn

#144875
Jan 29, 2009 at 5:51pm

I can’t launch “quartzplayer.help”

Max win tells:
• error: quartzplayer: unable to load object bundle executable
2009-01-29 18:48:38.804 MaxMSP[433] CFLog (21): dyld returns 2 when trying to load /Users/me/Desktop/quartzplayer_screen_capture/quartzplayer.mxo/Contents/MacOS/maximum

I’m on PB G4, OSX 4 Tiger
Max 4.6.3 jitter 1.6.3

Where am I wrong?

#144876

You must be logged in to reply to this topic.