d17e

i'm developing a video-mixer patch that takes in up to three quicktime movies and two cameras. The final output is a long triple-screen (16:3 ratio) matrix that is sent to 3 projectors.

since this is a quite intensive patch i created two subpatches that send data from the first to the second patch through the jet.net.send/recv objects.

the first patch reads in the single-screen movies/cameras and creates two decks (each a triple-screen matrix - 16:3). So in numbers this is:

- processing: glueing them together to 2x (2400x600) matrix [i select for each deck three of the five sources to be glueed in an order i decide using the router and jit.glue objects]

- output: sending out 2x (2400x600) 4 plane char matrix through jit.net.send/recv.

if i want to be able to send this over UTP with 25 fps, i would need a 10Gbit LAN interface on both machines (pc + laptop running WinXp). Gigabit LAN should get me up to 10 fps. 

(more numbers: 10fps = little less than 1 Gbit/s - 15fps = 1,38 Gbit/s - 25 fps = 2,30 Gbit/s - this if my calculations are correct offcourse)

I was wondering if, apart from redesigning my patch to sending just one deck over lan, if there is a hardware configuration which could allow me to send data from my laptop to the pc so that at the desired speed. Or am I just really wrong trying to accomplish this?

Maybe i should notice that my laptop is approximately running at 75 to 80 pct of ist CPU usage, so adding a lot more functionality to my first patch is not really what i'm looking for,...au contraire...

the first patch reads in the single-screen movies/cameras and creates two decks (each a triple-screen matrix - 16:3). So in numbers this is:
- sources: reading 5x (800x600) matrix
- processing: glueing them together to 2x (2400x600) matrix [i select for each deck three of the five sources to be glueed in an order i decide using the router and jit.glue objects]
- output: sending out 2x (2400x600) 4 plane char matrix through jit.net.send/recv.

if i want to be able to send this over UTP with 25 fps, i would need a 10Gbit LAN interface on both machines (pc + laptop running WinXp). Gigabit LAN should get me up to 10 fps. 
(more numbers: 10fps = little less than 1 Gbit/s - 15fps = 1,38 Gbit/s - 25 fps = 2,30 Gbit/s - this if my calculations are correct offcourse)

I was wondering if, apart from redesigning my patch to sending just one deck over lan, if there is a hardware configuration which could allow me to send data from my laptop to the pc so that at the desired speed. Or am I just really wrong trying to accomplish this? 

jit-net-sendrecv-limitations

> i'm developing a video-mixer patch that takes in up to three 

> Or am I just really wrong trying to accomplish this?

i just love to give out hi-res multi-projector advice.. so here goes

A) it is most definitely unrealistic to send those huge matrices over 

B) you probably need to redesign your mixer architecture, but the 

C) with a proper design you should be able to do it on a single 

I am assuming by your numbers that each projector resolution is 

800x600 SVGA, there is no actual rule that says that your movies must 

be at the native projector resolution, unless you are steadfast on 

having a 1to1 correlation in pixel resolution. the scalars on either 

the gpu or the hardware scalars built into the projectors themselves 

The perception of resolution is in fact very subjective, primarily a 

factor of audience distance to the screen:

The larger the screen, the more likely the audience is to be viewing 

at a distance, the farther you are from screen, the resolution 

matters much less, because ours eyes 'mix' the pixles for us, and are 

unable to perceive fine detail anyway. therefore the less likely it 

becomes that you actually need 1to1 pixels to achieve an excellent 

sit in the front row of any movie theater, and then move to the back 

just a small side example, 4 years ago i did an 8 channel piece on a 

single SXGA projector in a long 'strip' as you suggest, each channel 

was only 160x120, but the end result was a hyper long 1280x120 

resolution image with all those channels playing... the perception 

for the audience was as exhilarating as if they watched an omnimax 

panoramic display -- my point being, that you are on the right track, 

but you probably do not need to push the outer limits of resolution/ 

i would suggest using 400x300 source/process resolution, (because 

scalars do the best job when the source is exactly 1/4 (1/2 depending 

on you look at it) the resolution -- indeed, your cameras are most 

likely 640x480 (perhaps not if you've gone for a machine vision 

solution) , so you would already be scaling up those two sources up 

if you accept this law of scale and perception, than you could easily 

perform everything on a single machine with two graphics cards. 

source and mixing all happens at 400x300 (or 640x480 if you like), 

then allow the scalars to do the hard work for you.

if you are not already, you will achieve much more efficient results 

with mixing using jit.gl.slab, and also your sources in uyvy mode... 

it might not be out of the question to use 800x600 given this 

> The final output is a long triple-screen (16:3 ratio) matrix that 

> - processing: glueing them together to 2x (2400x600) matrix [i 

> select for each deck three of the five sources to be glueed in an 

> order i decide using the router and jit.glue objects]

> - output: sending out 2x (2400x600) 4 plane char matrix through 

this part is slightly confusing-- not sure why you are 'glueing' the 

three streams together for mixing, AFAIK you still need to send 

800x600 stream to three vga/dvi outputs -- are you trying to achieve 

a 'seamless' image? are you using 'edge blending' to achieve this? 

how are you mixing the 5 channels into 3 (that is, 5x 800 =4000, not 

2400) if i am missing something here let me know.

there is one limitation i should mention, you have 3 projectors, so 

you you need 3 outputs, and as far as i can see, you want one long 

matrix to 'span' across 3 desktops. unfortunately opengl cards do 

very poorly at spanning images ACROSS CARDS. they do perfectly fine 

spanning across 2 outputs of the same card though. so you may want 

to consider redesigning the system to use 2 SXGA projectors instead 

of 3 SVGA. 2x SXGA = 2560x1024 -- you do not need to utilize the 

full horizontal resolution, just leave your desktop black and display 

a 2400x600 window. now you only need one dedicated graphics card for 

display, and greatly simplify your architecture in terms of 'gluing'

lastly, if you want to explore hardware solutions, check out 

folsomresearch.com/ -- the screenpro and/or blendpro might be of 

interest to you, and can be rented from most video/tradeshow 

production houses. this is typically how its done in the 'pro' world 

- not to say that its not achievable in jitter (it is, as several 

by the way, your aspect ratio is 12x3 not 16x3 ; )

> since this is a quite intensive patch i created two subpatches that 

> send data from the first to the second patch through the 

> the first patch reads in the single-screen movies/cameras and 

> creates two decks (each a triple-screen matrix - 16:3). So in 

> if i want to be able to send this over UTP with 25 fps, i would 

> need a 10Gbit LAN interface on both machines (pc + laptop running 

> WinXp). Gigabit LAN should get me up to 10 fps.

> (more numbers: 10fps = little less than 1 Gbit/s - 15fps = 1,38 

> Gbit/s - 25 fps = 2,30 Gbit/s - this if my calculations are correct 

> I was wondering if, apart from redesigning my patch to sending just 

> one deck over lan, if there is a hardware configuration which could 

> allow me to send data from my laptop to the pc so that at the 

> Maybe i should notice that my laptop is approximately running at 75 

> to 80 pct of ist CPU usage, so adding a lot more functionality to 

> my first patch is not really what i'm looking for,...au contraire...

> i'm developing a video-mixer patch that takes in up to three  
> quicktime movies and two cameras.
...
> Or am I just really wrong trying to accomplish this?

A) it is most definitely unrealistic to send those huge matrices over  
a network
B) you probably need to redesign your mixer architecture, but the  
final result is achievable.
C) with a proper design you should be able to do it on a single  
desktop machine

I am assuming by your numbers that each projector resolution is  
800x600 SVGA, there is no actual rule that says that your movies must  
be at the native projector resolution, unless you are steadfast on  
having a 1to1 correlation in pixel resolution.  the scalars on either  
the gpu or the hardware scalars built into the projectors themselves  
can be quite good at scaling...

The perception of resolution is in fact very subjective, primarily a  
factor of audience distance to the screen:

The larger the screen, the more likely the audience is to be viewing  
at a distance, the farther you are from screen, the resolution  
matters much less, because ours eyes 'mix' the pixles for us, and are  
unable to perceive fine detail anyway. therefore the less likely it  
becomes that you actually need 1to1 pixels to achieve an excellent  
result.

sit in the front row of any movie theater, and then move to the back  
row and you'll see exactly what i mean.

just  a small side example, 4 years ago i did an 8 channel piece on a  
single SXGA projector in a long 'strip' as you suggest, each channel  
was only 160x120, but the end result was a hyper long 1280x120  
resolution image with all those channels playing... the perception  
for the audience was as exhilarating as if they watched an omnimax  
panoramic display -- my point being, that you are on the right track,  
but you probably do not need to push the outer limits of resolution/ 
processing power to achieve it.

i would suggest using 400x300 source/process resolution, (because  
scalars do the best job when the source is exactly 1/4 (1/2 depending  
on you look at it) the resolution -- indeed, your cameras are most  
likely 640x480 (perhaps not if you've gone for a machine vision  
solution) , so you would already be scaling up those two sources up  
before mixing.

if you accept this law of scale and perception, than you could easily  
perform everything on a single machine with two graphics cards.  
source and mixing all happens at 400x300 (or 640x480 if you like),  
then allow the scalars to do the hard work for you.

if you are not already, you will achieve much more efficient results  
with mixing using jit.gl.slab, and also your sources in uyvy mode...  
it might not be out of the question to use 800x600 given this  
powerful combination

> The final output is a long triple-screen (16:3 ratio) matrix that  
> is sent to 3 projectors.
> - processing: glueing them together to 2x (2400x600) matrix [i  
> select for each deck three of the five sources to be glueed in an  
> order i decide using the router and jit.glue objects]
> - output: sending out 2x (2400x600) 4 plane char matrix through  
> jit.net.send/recv.

this part is slightly confusing--  not sure why you are 'glueing' the  
three streams together for mixing, AFAIK you still need to send  
800x600 stream to three vga/dvi outputs -- are you trying to achieve  
a 'seamless' image?  are you using  'edge blending' to achieve this?   
otherwise you WILL have seams :)

how are you mixing the 5 channels into 3 (that is, 5x 800 =4000, not  
2400)  if i am missing something here let me know.

there is one limitation i should mention, you have 3 projectors, so  
you you need 3 outputs, and as far as i can see, you want one long  
matrix to 'span' across 3 desktops. unfortunately opengl cards do  
very poorly at spanning images ACROSS CARDS.  they do perfectly fine  
spanning across 2 outputs of the same card though.  so you may want  
to consider redesigning the system to use 2 SXGA projectors instead  
of 3 SVGA.  2x SXGA = 2560x1024 -- you do not need to utilize the  
full horizontal resolution, just leave your desktop black and display  
a 2400x600 window.  now you only need one dedicated graphics card for  
display, and greatly simplify your architecture in terms of 'gluing'

lastly, if you want to explore hardware solutions, check out http:// 
folsomresearch.com/ -- the screenpro and/or blendpro might be of  
interest to you, and can be rented from most video/tradeshow  
production houses.  this is typically how its done in the 'pro' world  
- not to say that its not achievable in jitter (it is, as several  
here including myself will attest)

by the way, your aspect ratio is 12x3 not 16x3  ; )

>
>
> since this is a quite intensive patch i created two subpatches that  
> send data from the first to the second patch through the  
> jet.net.send/recv objects.
>
> the first patch reads in the single-screen movies/cameras and  
> creates two decks (each a triple-screen matrix - 16:3). So in  
> numbers this is:
> - sources: reading 5x (800x600) matrix
>
>
> if i want to be able to send this over UTP with 25 fps, i would  
> need a 10Gbit LAN interface on both machines (pc + laptop running  
> WinXp). Gigabit LAN should get me up to 10 fps.
> (more numbers: 10fps = little less than 1 Gbit/s - 15fps = 1,38  
> Gbit/s - 25 fps = 2,30 Gbit/s - this if my calculations are correct  
> offcourse)
>
> I was wondering if, apart from redesigning my patch to sending just  
> one deck over lan, if there is a hardware configuration which could  
> allow me to send data from my laptop to the pc so that at the  
> desired speed.
>
> Maybe i should notice that my laptop is approximately running at 75  
> to 80 pct of ist CPU usage, so adding a lot more functionality to  
> my first patch is not really what i'm looking for,...au contraire...
>
> thanks for all advice
> d
>
>
>


First off, sorry for the late reply and thanks for the input!

... I tried replying earlier but just now i noticed that the message bounced since i tried attaching an image using my mail client... o well, here it is anyway

(my comments to your post are between /* and */...)

> i'm developing a video-mixer patch that takes in up to three

A) it is most definitely unrealistic to send those huge matrices over

B) you probably need to redesign your mixer architecture, but the

C) with a proper design you should be able to do it on a single

I am assuming by your numbers that each projector resolution is

800x600 SVGA, there is no actual rule that says that your movies must

be at the native projector resolution, unless you are steadfast on

having a 1to1 correlation in pixel resolution. the scalars on either

the gpu or the hardware scalars built into the projectors themselves

The perception of resolution is in fact very subjective, primarily a

The larger the screen, the more likely the audience is to be viewing

at a distance, the farther you are from screen, the resolution

matters much less, because ours eyes 'mix' the pixles for us, and are

unable to perceive fine detail anyway. therefore the less likely it

becomes that you actually need 1to1 pixels to achieve an excellent

sit in the front row of any movie theater, and then move to the back

just a small side example, 4 years ago i did an 8 channel piece on a

single SXGA projector in a long 'strip' as you suggest, each channel

was only 160x120, but the end result was a hyper long 1280x120

resolution image with all those channels playing... the perception

for the audience was as exhilarating as if they watched an omnimax

panoramic display -- my point being, that you are on the right track,

but you probably do not need to push the outer limits of resolution/

i would suggest using 400x300 source/process resolution, (because

scalars do the best job when the source is exactly 1/4 (1/2 depending

on you look at it) the resolution -- indeed, your cameras are most

likely 640x480 (perhaps not if you've gone for a machine vision

solution) , so you would already be scaling up those two sources up

This is indeed a good argument to start working with smaller matrices; allthough, when i first tested the patch on the screens (which are three screens -3m x 4m- standing next to each other in an open-cube-like configuration), i was using 640 x 480 matrices which resulted in quite pixelated images. Anyhow, i will investigate this issue further and try to work with smaller matrices, which will definitely result in more breathing space for me and my pc and laptop.

if you accept this law of scale and perception, than you could easily

perform everything on a single machine with two graphics cards.

source and mixing all happens at 400x300 (or 640x480 if you like),

if you are not already, you will achieve much more efficient results

with mixing using jit.gl.slab, and also your sources in uyvy mode...

it might not be out of the question to use 800x600 given this

As mentioned before, for now I am only using processing power. It is definitely not an option that the final mixer will work completely on the CPU, so i will dig deeper in the gl objects to get better performance. the reason why i'm starting out based on the CPU is because I thought it might be a good 'lower border' for performance. So if get reasonable results using only the CPU i can go a step further and make an even better patch using the GPU. Considering your arguments, and my instinct, i think i'd better start using the GPU straight away since, i'll need its performance anyhow.

> The final output is a long triple-screen (16:3 ratio) matrix that

> - processing: glueing them together to 2x (2400x600) matrix [i

> select for each deck three of the five sources to be glueed in an

> - output: sending out 2x (2400x600) 4 plane char matrix through

this part is slightly confusing-- not sure why you are 'glueing' the

three streams together for mixing, AFAIK you still need to send

800x600 stream to three vga/dvi outputs -- are you trying to achieve

a 'seamless' image? are you using 'edge blending' to achieve this?

how are you mixing the 5 channels into 3 (that is, 5x 800 =4000, not

actually, what i'm doing is, I have 5 inputs of which i only use 3 for each deck. to select which sources to use, i use a the max router object, more precise as ' router 5 3 '. afterwards i glue these three output matrices (from the router) together with jit.glue.

there is one limitation i should mention, you have 3 projectors, so

you you need 3 outputs, and as far as i can see, you want one long

matrix to 'span' across 3 desktops. unfortunately opengl cards do

very poorly at spanning images ACROSS CARDS. they do perfectly fine

spanning across 2 outputs of the same card though. so you may want

to consider redesigning the system to use 2 SXGA projectors instead

of 3 SVGA. 2x SXGA = 2560x1024 -- you do not need to utilize the

full horizontal resolution, just leave your desktop black and display

a 2400x600 window. now you only need one dedicated graphics card for

You should know that i am using a Matrox Triplehead2Go 'screensplitter' This little box shows up to my pc as being a 12x3 (thanks for the correction :) screen, and has three outputs; for three screens or projectors.

I attached a little basic sketch of the hardware setup which should make things a bit more clear.

folsomresearch.com/ -- the screenpro and/or blendpro might be of

interest to you, and can be rented from most video/tradeshow

production houses. this is typically how its done in the 'pro' world

- not to say that its not achievable in jitter (it is, as several

nice hardware products, but i'm afraid they're an itsypitsy out of mine and the project's price range ;)

> since this is a quite intensive patch i created two subpatches that

> send data from the first to the second patch through the

> the first patch reads in the single-screen movies/cameras and

> creates two decks (each a triple-screen matrix - 16:3). So in

> if i want to be able to send this over UTP with 25 fps, i would

> need a 10Gbit LAN interface on both machines (pc + laptop running

> (more numbers: 10fps = little less than 1 Gbit/s - 15fps = 1,38

> Gbit/s - 25 fps = 2,30 Gbit/s - this if my calculations are correct

> I was wondering if, apart from redesigning my patch to sending just

> one deck over lan, if there is a hardware configuration which could

> allow me to send data from my laptop to the pc so that at the

> Maybe i should notice that my laptop is approximately running at 75

> to 80 pct of ist CPU usage, so adding a lot more functionality to

> _______________________________________________

https://cycling74.com/mailman/listinfo/jitter

Hi DeKam,
First off, sorry for the late reply and thanks for the input!
... I tried replying earlier but just now i noticed that the message bounced since i tried attaching an image using my mail client... o well, here it is anyway

    > i'm developing a video-mixer patch that takes in up to three
    > quicktime movies and two cameras.
    ...
    > Or am I just really wrong trying to accomplish this?

    i just love to give out hi-res multi-projector advice.. so here goes

    A) it is most definitely unrealistic to send those huge matrices over
    a network
    B) you probably need to redesign your mixer architecture, but the
    final result is achievable.
    C) with a proper design you should be able to do it on a single
    desktop machine

    I am assuming by your numbers that each projector resolution is
    800x600 SVGA, there is no actual rule that says that your movies must
    be at the native projector resolution, unless you are steadfast on
    having a 1to1 correlation in pixel resolution.  the scalars on either
    the gpu or the hardware scalars built into the projectors themselves
    can be quite good at scaling...

    The perception of resolution is in fact very subjective, primarily a
    factor of audience distance to the screen:

    The larger the screen, the more likely the audience is to be viewing
    at a distance, the farther you are from screen, the resolution
    matters much less, because ours eyes 'mix' the pixles for us, and are
    unable to perceive fine detail anyway. therefore the less likely it
    becomes that you actually need 1to1 pixels to achieve an excellent
    result.

    sit in the front row of any movie theater, and then move to the back
    row and you'll see exactly what i mean.

    just  a small side example, 4 years ago i did an 8 channel piece on a
    single SXGA projector in a long 'strip' as you suggest, each channel
    was only 160x120, but the end result was a hyper long 1280x120
    resolution image with all those channels playing... the perception
    for the audience was as exhilarating as if they watched an omnimax
    panoramic display -- my point being, that you are on the right track,
    but you probably do not need to push the outer limits of resolution/
    processing power to achieve it.

    i would suggest using 400x300 source/process resolution, (because
    scalars do the best job when the source is exactly 1/4 (1/2 depending
    on you look at it) the resolution -- indeed, your cameras are most
    likely 640x480 (perhaps not if you've gone for a machine vision
    solution) , so you would already be scaling up those two sources up
    before mixing. 

jit.net.send/recv limitations