Low latency video input – seemingly impossible!
I apologize for the long message but it includes some useful resources that attempt to clarify the problem. I’ve had this problem for a number of years and have yet to find a reasonable solution.
How do I get low latency video from an external camera into jitter (or any software for that matter), perform simple manipulations and then send it out to a projector?
I need the projected image to not apply a noticeable delay to the original source. If I am filming someone talking I want the sound coming from their lips and projected image to appear "live" without latency. I’m at the point where I’d like the image to be better than SD but will settle for anything.
A humble request in 2012 with such powerful computing power no?
I have a very fast macbook pro laptop with great graphics card and a thunderbolt connection and firewire 800. I also have a Mac Pro desktop.
I own a canopus advc55 firewire and have tried many many different cameras with it. I still can’t achieve low enough latency. I’ve found that analog cameras seem to work better than digital cameras with analog outputs but it’s still not good enough. I’ve tried the same camera setup in modul8 and isadora with similar latency results. I assume this means that it doesn’t have to do with jitter or my not so great jitter programming.
I read about the new Blackmagic intensity. Sounds great! but after reading this thread (http://cycling74.com/forums/topic.php?id=38378) it doesn’t seem like a solution, especially considering that it uses the only thunderbolt port and can’t daisy chain to allow a projector. Lame!
It was suggested that the DFG 1394-1e was a good option but it is now discontinued. The Matrox MXO2 Mini seems ok but again doesn’t solve the thunderbolt daisy chain problem.
Frieder Weiss developed the Eyecon system and created an amazing visual system for Chunky Move. He suggests the FALCON PCI Bus Framegrabber. Unfortunately it will not work with a laptop and I don’t even think it works on Mac.
Vade wrote this article 5 years ago discussing an older intensity card but says that it "requires some fairly in depth knowledge of Jitter," to make work.
So after all of this I still am left without a solution. It’s frustrating because it seems like a simple operation that many people would want to do.
Are there any options left aside from getting a PC with the Falcon grabber?
Your best bet is to get a PC running windows xp.
Get the aiptek 5900 dv cam
It works as a high quality webcam.
(Onboard software controls let you control sharpness gamma contrast ect.)
It can be output live via its software that comes with it.
I used the same setup and displayed footage real time that I could tweak and control on my projector.
Matrox does make a PCI-e card for the MX02 mini – I have one and this works well in my Mac Pro tower. I also have the Expresscard interface, but you need a 17" MacBook Pro to make that work. Have not tried the Thunderbolt option and I likely won’t.
People have also reported good luck with using the Intensity Pro PCI-e card from Blackmagic, which is a cheaper system.
any update on this? I’m also searching for a portable low latency video input solution for the current MacBook Pro with Thunderbolt and USB 3.0. But it doesn’t have an Expresscard slot anymore. How is the latency with the MX02 mini?
Beware of the Matrox MX02 Mini. I have one with the PCI interface and there is a very long latency (feels like 1/4 or 1/2 sec!). I also have the BlackMagic Intensity PCI card and there is hardly any latency (about 1 frame). I assume the thunderbolt version is similar but I’ve never tried it. (Bummer about the lack of thunderbolt passthrough though.)
FYI: There is no "in-depth knowledge of MAX" required beyond the OpenGL optimizations that should be used for all capture and rendering pipelines anyway. (Capture in uyvy with "@unique 1", do rgb conversion with a shader, only bang the OpenGL renderer when a new frame arrives, etc) See the opengl tutorials and search for vade’s "jitter movie playback optimizations" for ideas. I think he mentioned the "in-depth knowledge" because that blog post covered several programming systems and some of them are more optimized out-of-the-box.)
You will never get low latency with a camera connected to the Canopus box because it converts your cam’s analog SD signal to DV which then must be decoded on the CPU causing unavoidable latency. I have the DFG box for analog SD capture and it has virtually no latency.
Overall I’m leaning toward the Intensity for your application.
Thanks for the advice. I’ll check out the Blackmagic stuff. If anyone has experience with either the Intensity Shuttle or Extreme, please tell. Seems like the Shuttle USB 3.0 doesn’t work with Mac. For Mac you need the Thunderbolt version.
I’m needing low latency for live processing of staged video capture (theater project). I’m also needing to upgrade my macbook from several gens.
I’m eyeing the BlackMagic Extreme for capture from hdmi (directly from an HMC-40).
I am wondering if any one has had experience with the new retina macbook pro that has onboard hdmi out. Would that solve problems of Jitter not being able to consume and broadcast a signal through the thunderbolt port – since you could broadcast out the htmi port instead?
I ask, because otherwise i would stick with the non-retina macbook pro to keep my firewire800…
sorry for reopening this older thread.
I am starting to work on a project where I will need 3 live-inputs with no (or almost no) latency. I would like to go better then SD.
I have different choices and wonder what would be the best way to go. Right now I am using a MacPro (actually a Hackintosh) which I could reconfigure into a Windows if necessary.
A) getting 3 HD-SDI security cameras and use a Blackmagic DeckLink Duo + 1 Blackmagic Decklink Extreme. I have both cards "lying around" but never tested if they work all at once. I also don’t know about quality on security cameras
B) setup a windows system and get 3 PCI Gamer Capture Cards (something like this:
http://www.amazon.de/StarTech-com-Express-Video-Capture-Karte/dp/B007U5MGBE/ref=sr_1_11?ie=UTF8&qid=1432898744&sr=8-11&keywords=hdmi+capture+card) (sorry, it’s a german site, but you’ll get the idea). The card is able to capture component or HDMI. That way I would have the choice of getting eather 3 HD(V) Cameras with component-out or HDMI-out.
C) something else I haven’t yet thought of :) Maybe a multi-stream capture card??
What’s your opinion? Where are my best chances to get 3 live-inputs at all and at low/no latency (720p would be ok): SDI, HDMI or Component?
Good day to all,
I have had some good experience with some simple USB webcams, especially using Logitech c310 (http://www.logitech.com/en-gb/product/hd-webcam-c310), I have encountered very little latency overall (around 1 or 2 frames, tops).
Is there any way to control those web-cams in terms of exposure, focus etc.?
I usually use FW / USB3 PointGrey cameras for machine vision.
They have the lowest latency I’ve encountered, and you can programmatically control exposure, shutter etc (on some of them).
(bear in mind usually projectors can introduce a lot of latency too),
I also haven’t used them in max/jitter, but directly in C++ with PtGrey SDK on windows, and libdc on mac. I’d be curious to hear if there are max externals. Actually I’d also be curious to hear if it’s possible to have a cross-platform Max patch that uses the relevant external. I.e. a windows .mxe which uses PtGrey SDK, and an osx .mxo which uses the libdc external, and they respond to the same messages / attributes etc. So you can develop a max patch that uses the camera and can seamlessly work on windows and mac? (In my native C++ apps I use #ifdef to include and compile the relevant SDK for the platform, so the same application code can be compiled and run on both platforms).
@Mr.L : on Mac I use this app: https://itunes.apple.com/en/app/webcam-settings/id533696630?mt=12
@Memo Atken: Yeah, Point Grey are great…mine is the poor man’s solution!
Re: cross-platform patches: I think this is totally doable. The externals should just have the same name and be in Max’s file path.
the advantage would basically be (besides low latency) to save a capture device, right?
Have you used more then 1 camera at a time? (Would MAX be able to recognize 3?).
I still see disadvantage in not having manual control and no preview for the actor.
How about image quality?
@Memo Atken: there are also industrial cameras with USB2.0. Are they sufficient, or better go for 3.0. (quite expensive).
@LSKA: good idea about the app. I will try out one of those logistic. It’s worth the money… Have you used more then 1?
Yes, I used up to 2 cameras, but I think it’s only a limit of my setup (I use a Macbook Pro with 2 USB ports, so far, I have been able to have only one camera for USB port working, I guess it’s a bandwidth/power consumption issue. I think with a desktop and more USB ports, you can have more cameras attached.
I have used multiple Blackmagic cards in a single machine, and have had good luck with this using the Black Syphon application, which provides feeds into Jitter via Syphon. The program maintained by Vidvox and has been very reliable.
Anecdotally, SDI is lower latency than analog, HDMI, or USB. Less overhead in the protocol. Most people I know who are looking for lowest latency are using SDI cameras connected to dedicated digitizers, but you’ll have to test the specific models you’re targeting.
For multiple SDI cameras I’d probably buy the Decklink Quad
As Memo points out, many lower end projectors themselves have large framebuffers, which results in high latency.
Forums > Jitter