Forums > Jitter

jit.net.send/recv limitations

August 31, 2006 | 4:05 pm

Hi,

i’m developing a video-mixer patch that takes in up to three quicktime movies and two cameras. The final output is a long triple-screen (16:3 ratio) matrix that is sent to 3 projectors.

since this is a quite intensive patch i created two subpatches that send data from the first to the second patch through the jet.net.send/recv objects.

the first patch reads in the single-screen movies/cameras and creates two decks (each a triple-screen matrix – 16:3). So in numbers this is:
- sources: reading 5x (800×600) matrix
- processing: glueing them together to 2x (2400×600) matrix [i select for each deck three of the five sources to be glueed in an order i decide using the router and jit.glue objects]
- output: sending out 2x (2400×600) 4 plane char matrix through jit.net.send/recv.

if i want to be able to send this over UTP with 25 fps, i would need a 10Gbit LAN interface on both machines (pc + laptop running WinXp). Gigabit LAN should get me up to 10 fps.
(more numbers: 10fps = little less than 1 Gbit/s – 15fps = 1,38 Gbit/s – 25 fps = 2,30 Gbit/s – this if my calculations are correct offcourse)

I was wondering if, apart from redesigning my patch to sending just one deck over lan, if there is a hardware configuration which could allow me to send data from my laptop to the pc so that at the desired speed. Or am I just really wrong trying to accomplish this?

Maybe i should notice that my laptop is approximately running at 75 to 80 pct of ist CPU usage, so adding a lot more functionality to my first patch is not really what i’m looking for,…au contraire…

thanks for all advice
d


August 31, 2006 | 9:35 pm

> i’m developing a video-mixer patch that takes in up to three
> quicktime movies and two cameras.

> Or am I just really wrong trying to accomplish this?

i just love to give out hi-res multi-projector advice.. so here goes

A) it is most definitely unrealistic to send those huge matrices over
a network
B) you probably need to redesign your mixer architecture, but the
final result is achievable.
C) with a proper design you should be able to do it on a single
desktop machine

I am assuming by your numbers that each projector resolution is
800×600 SVGA, there is no actual rule that says that your movies must
be at the native projector resolution, unless you are steadfast on
having a 1to1 correlation in pixel resolution. the scalars on either
the gpu or the hardware scalars built into the projectors themselves
can be quite good at scaling…

The perception of resolution is in fact very subjective, primarily a
factor of audience distance to the screen:

The larger the screen, the more likely the audience is to be viewing
at a distance, the farther you are from screen, the resolution
matters much less, because ours eyes ‘mix’ the pixles for us, and are
unable to perceive fine detail anyway. therefore the less likely it
becomes that you actually need 1to1 pixels to achieve an excellent
result.

sit in the front row of any movie theater, and then move to the back
row and you’ll see exactly what i mean.

just a small side example, 4 years ago i did an 8 channel piece on a
single SXGA projector in a long ‘strip’ as you suggest, each channel
was only 160×120, but the end result was a hyper long 1280×120
resolution image with all those channels playing… the perception
for the audience was as exhilarating as if they watched an omnimax
panoramic display — my point being, that you are on the right track,
but you probably do not need to push the outer limits of resolution/
processing power to achieve it.

i would suggest using 400×300 source/process resolution, (because
scalars do the best job when the source is exactly 1/4 (1/2 depending
on you look at it) the resolution — indeed, your cameras are most
likely 640×480 (perhaps not if you’ve gone for a machine vision
solution) , so you would already be scaling up those two sources up
before mixing.

if you accept this law of scale and perception, than you could easily
perform everything on a single machine with two graphics cards.
source and mixing all happens at 400×300 (or 640×480 if you like),
then allow the scalars to do the hard work for you.

if you are not already, you will achieve much more efficient results
with mixing using jit.gl.slab, and also your sources in uyvy mode…
it might not be out of the question to use 800×600 given this
powerful combination

> The final output is a long triple-screen (16:3 ratio) matrix that
> is sent to 3 projectors.
> – processing: glueing them together to 2x (2400×600) matrix [i
> select for each deck three of the five sources to be glueed in an
> order i decide using the router and jit.glue objects]
> – output: sending out 2x (2400×600) 4 plane char matrix through
> jit.net.send/recv.

this part is slightly confusing– not sure why you are ‘glueing’ the
three streams together for mixing, AFAIK you still need to send
800×600 stream to three vga/dvi outputs — are you trying to achieve
a ‘seamless’ image? are you using ‘edge blending’ to achieve this?
otherwise you WILL have seams :)

how are you mixing the 5 channels into 3 (that is, 5x 800 =4000, not
2400) if i am missing something here let me know.

//

there is one limitation i should mention, you have 3 projectors, so
you you need 3 outputs, and as far as i can see, you want one long
matrix to ‘span’ across 3 desktops. unfortunately opengl cards do
very poorly at spanning images ACROSS CARDS. they do perfectly fine
spanning across 2 outputs of the same card though. so you may want
to consider redesigning the system to use 2 SXGA projectors instead
of 3 SVGA. 2x SXGA = 2560×1024 — you do not need to utilize the
full horizontal resolution, just leave your desktop black and display
a 2400×600 window. now you only need one dedicated graphics card for
display, and greatly simplify your architecture in terms of ‘gluing’

lastly, if you want to explore hardware solutions, check out http://
folsomresearch.com/ — the screenpro and/or blendpro might be of
interest to you, and can be rented from most video/tradeshow
production houses. this is typically how its done in the ‘pro’ world
- not to say that its not achievable in jitter (it is, as several
here including myself will attest)

by the way, your aspect ratio is 12×3 not 16×3 ; )

good luck!

–DeKam

>
>
> since this is a quite intensive patch i created two subpatches that
> send data from the first to the second patch through the
> jet.net.send/recv objects.
>
> the first patch reads in the single-screen movies/cameras and
> creates two decks (each a triple-screen matrix – 16:3). So in
> numbers this is:
> – sources: reading 5x (800×600) matrix
>
>
> if i want to be able to send this over UTP with 25 fps, i would
> need a 10Gbit LAN interface on both machines (pc + laptop running
> WinXp). Gigabit LAN should get me up to 10 fps.
> (more numbers: 10fps = little less than 1 Gbit/s – 15fps = 1,38
> Gbit/s – 25 fps = 2,30 Gbit/s – this if my calculations are correct
> offcourse)
>
> I was wondering if, apart from redesigning my patch to sending just
> one deck over lan, if there is a hardware configuration which could
> allow me to send data from my laptop to the pc so that at the
> desired speed.
>
> Maybe i should notice that my laptop is approximately running at 75
> to 80 pct of ist CPU usage, so adding a lot more functionality to
> my first patch is not really what i’m looking for,…au contraire…
>
> thanks for all advice
> d
>
>
>


September 6, 2006 | 12:01 am

Hi DeKam,
First off, sorry for the late reply and thanks for the input!
… I tried replying earlier but just now i noticed that the message bounced since i tried attaching an image using my mail client… o well, here it is anyway

(my comments to your post are between /* and */…)

> i’m developing a video-mixer patch that takes in up to three
> quicktime movies and two cameras.

> Or am I just really wrong trying to accomplish this?

i just love to give out hi-res multi-projector advice.. so here goes

A) it is most definitely unrealistic to send those huge matrices over
a network
B) you probably need to redesign your mixer architecture, but the
final result is achievable.
C) with a proper design you should be able to do it on a single
desktop machine

I am assuming by your numbers that each projector resolution is
800×600 SVGA, there is no actual rule that says that your movies must
be at the native projector resolution, unless you are steadfast on
having a 1to1 correlation in pixel resolution. the scalars on either
the gpu or the hardware scalars built into the projectors themselves
can be quite good at scaling…

The perception of resolution is in fact very subjective, primarily a
factor of audience distance to the screen:

The larger the screen, the more likely the audience is to be viewing
at a distance, the farther you are from screen, the resolution
matters much less, because ours eyes ‘mix’ the pixles for us, and are
unable to perceive fine detail anyway. therefore the less likely it
becomes that you actually need 1to1 pixels to achieve an excellent
result.

sit in the front row of any movie theater, and then move to the back
row and you’ll see exactly what i mean.

just a small side example, 4 years ago i did an 8 channel piece on a
single SXGA projector in a long ‘strip’ as you suggest, each channel
was only 160×120, but the end result was a hyper long 1280×120
resolution image with all those channels playing… the perception
for the audience was as exhilarating as if they watched an omnimax
panoramic display — my point being, that you are on the right track,
but you probably do not need to push the outer limits of resolution/
processing power to achieve it.

i would suggest using 400×300 source/process resolution, (because
scalars do the best job when the source is exactly 1/4 (1/2 depending
on you look at it) the resolution — indeed, your cameras are most
likely 640×480 (perhaps not if you’ve gone for a machine vision
solution) , so you would already be scaling up those two sources up
before mixing.

/*
This is indeed a good argument to start working with smaller matrices; allthough, when i first tested the patch on the screens (which are three screens -3m x 4m- standing next to each other in an open-cube-like configuration), i was using 640 x 480 matrices which resulted in quite pixelated images. Anyhow, i will investigate this issue further and try to work with smaller matrices, which will definitely result in more breathing space for me and my pc and laptop.
*/

if you accept this law of scale and perception, than you could easily
perform everything on a single machine with two graphics cards.
source and mixing all happens at 400×300 (or 640×480 if you like),
then allow the scalars to do the hard work for you.

if you are not already, you will achieve much more efficient results
with mixing using jit.gl.slab, and also your sources in uyvy mode…
it might not be out of the question to use 800×600 given this
powerful combination

/*
As mentioned before, for now I am only using processing power. It is definitely not an option that the final mixer will work completely on the CPU, so i will dig deeper in the gl objects to get better performance. the reason why i’m starting out based on the CPU is because I thought it might be a good ‘lower border’ for performance. So if get reasonable results using only the CPU i can go a step further and make an even better patch using the GPU. Considering your arguments, and my instinct, i think i’d better start using the GPU straight away since, i’ll need its performance anyhow.
*/

> The final output is a long triple-screen (16:3 ratio) matrix that
> is sent to 3 projectors.
> – processing: glueing them together to 2x (2400×600) matrix [i
> select for each deck three of the five sources to be glueed in an
> order i decide using the router and jit.glue objects]
> – output: sending out 2x (2400×600) 4 plane char matrix through
> jit.net.send/recv.

this part is slightly confusing– not sure why you are ‘glueing’ the
three streams together for mixing, AFAIK you still need to send
800×600 stream to three vga/dvi outputs — are you trying to achieve
a ‘seamless’ image? are you using ‘edge blending’ to achieve this?
otherwise you WILL have seams :)

how are you mixing the 5 channels into 3 (that is, 5x 800 =4000, not
2400) if i am missing something here let me know.

//

/*
actually, what i’m doing is, I have 5 inputs of which i only use 3 for each deck. to select which sources to use, i use a the max router object, more precise as ‘ router 5 3 ‘. afterwards i glue these three output matrices (from the router) together with jit.glue.
*/

there is one limitation i should mention, you have 3 projectors, so
you you need 3 outputs, and as far as i can see, you want one long
matrix to ‘span’ across 3 desktops. unfortunately opengl cards do
very poorly at spanning images ACROSS CARDS. they do perfectly fine
spanning across 2 outputs of the same card though. so you may want
to consider redesigning the system to use 2 SXGA projectors instead
of 3 SVGA. 2x SXGA = 2560×1024 — you do not need to utilize the
full horizontal resolution, just leave your desktop black and display
a 2400×600 window. now you only need one dedicated graphics card for
display, and greatly simplify your architecture in terms of ‘gluing’

/*
You should know that i am using a Matrox Triplehead2Go ‘screensplitter’ This little box shows up to my pc as being a 12×3 (thanks for the correction :) screen, and has three outputs; for three screens or projectors.
I attached a little basic sketch of the hardware setup which should make things a bit more clear.
*/

lastly, if you want to explore hardware solutions, check out http://
folsomresearch.com/ — the screenpro and/or blendpro might be of
interest to you, and can be rented from most video/tradeshow
production houses. this is typically how its done in the ‘pro’ world
– not to say that its not achievable in jitter (it is, as several
here including myself will attest)

/*
nice hardware products, but i’m afraid they’re an itsypitsy out of mine and the project’s price range ;)
*/

by the way, your aspect ratio is 12×3 not 16×3 ; )

/*
indeedio
*/

good luck!

–DeKam

/*
Thanks
*/

>
>
> since this is a quite intensive patch i created two subpatches that
> send data from the first to the second patch through the
> jet.net.send /recv objects.
>
> the first patch reads in the single-screen movies/cameras and
> creates two decks (each a triple-screen matrix – 16:3). So in
> numbers this is:
> – sources: reading 5x (800×600) matrix
>
>
> if i want to be able to send this over UTP with 25 fps, i would
> need a 10Gbit LAN interface on both machines (pc + laptop running
> WinXp). Gigabit LAN should get me up to 10 fps.
> (more numbers: 10fps = little less than 1 Gbit/s – 15fps = 1,38
> Gbit/s – 25 fps = 2,30 Gbit/s – this if my calculations are correct
> offcourse)
>
> I was wondering if, apart from redesigning my patch to sending just
> one deck over lan, if there is a hardware configuration which could
> allow me to send data from my laptop to the pc so that at the
> desired speed.
>
> Maybe i should notice that my laptop is approximately running at 75
> to 80 pct of ist CPU usage, so adding a lot more functionality to
> my first patch is not really what i’m looking for,…au contraire…
>
> thanks for all advice
> d
>
> _______________________________________________
> jitter mailing list
> jitter@cycling74.com
> http://www.cycling74.com/mailman/listinfo/jitter
>
>


September 6, 2006 | 12:27 am

Just so you know,

i’ve been altering my patch to make use of the GPU. This results in much better performance… however it’s still not optimal.
I can fairly say that the bottleneck now, is no longer the jit.net.send/recv objects, so i’ll be moving my lamentation to another topic :)

cheers!
d


September 6, 2006 | 9:38 am

A smart little solution for a wide-screen 3-projector setup we use in some situations is to render to a 1080 x 288 window (on one 1152 x 864 resolution output). We send this to a video splitter (DC VS-814) and to 3 beamers that support zooming (e.g. the Sharp Notevision PG-A20X Projector). Each beamer zooms to another portion of the incoming signal.

So that makes 360 x 288 per screen.
> The perception of resolution is in fact very subjective
I agree. We’ve been performing with this resolution for fairly large crowds, with fairly large screens, both projections and led-screens and I never heard complaints about resolution. Of course there are better solutions for this but the priority is not very high.. the content is much more important.

Cheers,
Mattijs

http://www.samplemadness.nl/smadsteck


September 6, 2006 | 2:46 pm


> /*
> This is indeed a good argument to start working with smaller
> matrices; allthough, when i first tested the patch on the screens
> (which are three screens -3m x 4m- standing next to each other in
> an open-cube-like configuration), i was using 640 x 480 matrices
> which resulted in quite pixelated images. Anyhow, i will
> investigate this issue further and try to work with smaller
> matrices, which will definitely result in more breathing space for
> me and my pc and laptop.
> */

it boils down to your own subjective decision, as its is often
dependent on the content itself (for example ‘photographic’ content
shot with a camera may scale better than things generated digitally)

> Considering your arguments, and my instinct, i think i’d better
> start using the GPU straight away since, i’ll need its performance
> anyhow.
> */

good!

> You should know that i am using a Matrox Triplehead2Go
> ‘screensplitter’ This little box shows up to my pc as being a 12×3
> (thanks for the correction :) screen, and has three outputs; for
> three screens or projectors.
> I attached a little basic sketch of the hardware setup which should
> make things a bit more clear.
> */

ok – personally i haven’t used this, but i’ve heard good things from
friends on the list :)

> i’m experiencing low framerates when i’m loading in three large
> quicktime movies simultaneously. Each of the files is about 1.7GB
> and i paste em all three together using the jit.glue object. This
> is all rendered on the GPU.
> I’m pretty sure that it’s my Harddisk (7200 rpm) that cannot keep
> up,… so i was wondering if there’s a way to work around this issue.

you are correct intuitively, the movie datarate is a large bottleneck
to overcome. — we have discussed this issue on the list previously,
if you search on other threads i’ve posted before – essentially, the
codec you compress the movie in will have the big effect, and
naturally you should also have an adequate disk subsystem capable of
handling the load.

The best work around is simply to add more hardrives(!) — a low end
RAID 0, such as eSATA, internal SATA, or even a firewire 800 ‘dual
disk’ raid will greatly improve things. the main thing is to have
multiple disks reading the movies in. I’ve had good luck with a 3
drive internal SATA raid, which can be had for as little as $500.

For codecs, the most favored codec generally is PhotoJPEG at 70-75%
quality – it offers a great balance of quality, datarate and cpu
load. (keep in mind that different codecs require different CPU
loads to decompress) Others use ‘Animation’ codec, which yields 2:1
and is very cheap on the cpu, but the datarate is still quite high.
Most shy away from modern ‘h264′, ‘mp4′, ‘pixlet’ and the like
because they are expensive on the cpu. MPEG2 generally sucks for
quality, but works for some. If you invest in a wicked wicked disk
subsystem, you might opt for the ‘Component’ codec which is
temporally and spatially uncompressed, great quality, zero CPU,
native for jitter’s uyvy mode, but also very high datarate (thus you
need STRONG disks for this)

I would buy more hardrives before you spend any time on ‘buffering’
schemes in Max. at the very least you can read ‘one movie per disk’
without a raid, even on cheap FW400 external disks, and have better
luck than trying to read all 3 from one 7200 disk.

–deKam


September 9, 2006 | 8:07 pm

Thanks for the tip!

using more disks is indeed the best solution… once again i was searching answers too far.

A nice disk subsystem is on my list!


Viewing 7 posts - 1 through 7 (of 7 total)