jit.ndi - Externals for sending video/audio over the network
Had this one in the 'should probably get this released soon' pile for quite a while but have now taken advantage of my current excess of free time to push it over the finish line. Above is a first version of my NDI package for Max, which allows you to send/receive jitter matrices and MSP signals over a network to/from any device which supports NDI.
If you don't know what NDI is, it's a protocol created by NewTek for video/audio over IP. It has very high performance and low-latency, and there's a lot of reasonably cost-effective hardware which supports it natively (e.g. https://www.bird-dog.tv/studio-overview/). NewTek also sell a camera app for iOS (https://www.newtek.com/software/ndi-camera/) which is a very cheap way to get a wireless camera into your patch.
Package is free for all and LGPL licensed. Have done moderate testing, but still could be a few fatal crashes in there so please be careful if using in anything important. The thing that's been tested least is the audio receiving side, if you're using this please let me know how it's working for you.
Amazing, Thank You!
David,
Thx for this.
Is the runtime for Windows only?
Link is to .exe. file?
B
No there’s a mac runtime as well, I’ll update the link in a bit. If you download the Mac version of NDI tools it will install the runtime.
Cool, got the Mac runtime link from console.
B
Hello David,
Avec mes plus sincères remerciements.
imp, ndi, posistage... you're the man!
H.
Thanks David!
Don't hate me, but I'm just wondering... do you have any plans in implementing GL texture input/output for your externals?
@LSKA So because it's network based, all NDI data has to go through the CPU. Therefore adding GL texture IO natively to the object wouldn't help much as essentially I'd have to implement a mechanism to convert the texture to/from a standard jitter matrix before it could be sent/received. Jitter already has this functionality via jit.gl.asyncread and jit.gl.texture objects, so there's not much point in me duplicating it. What I may do however is add an extra page to the help patchers showing how to do the texture conversion.
@David You're right, I didn't think of that.
This is great—looking forward to testing it out! I'm getting 'no such object' for jit.ndi.send~.... I've installed the runtime for Mac and the NDI tools. Do I need another step to convert the vcxproj files?
@antialias Looks like you've downloaded the source code rather than the compiled objects, this is the download you need.
Thanks David!
MASSIVE!🙇♂️
Superb! I used it with the NDI Syphon, great that this extra step isn't necessary anymore.
But unfortunately i don't get the external running, i still get the message in the Max window: "Unable to load NDI runtime library. Please download and/or reinstall NDI runtime from 'http://new.tk/NDIRedistV4Apple' and restart Max"
I have done this and installed also NewTekNDIHXDriverForMacOS, NewTekNDIToolsForMacOS and libNDI_for_Mac.pkg
Is there anything i forgot or can check to get it working?
Max 8.1.3
MacOS 10.14.6
Thanks in advance!
@Aartcore Have you restarted your system since installing? If not try that. Otherwise could you run find /usr/local/lib -name libndi.4.dylib in the terminal and post the result?
Yes tried also restarting.
terminal found the file: /usr/local/lib/libndi.4.dylib (file date is 26 march 2020)
NDI is also working on my computer, for instance the NewTek NDI Scan Converter sends the image to NDISyphon
btw. when i load the jit.ndi.receive~ or send object i get the message 6 times:
"Unable to load NDI runtime library. Please download and/or reinstall NDI runtime from 'http://new.tk/NDIRedistV4Apple' and restart Max"
may be this could be helpful in solving the problem?
@Aartcore I've checked my code for loading the library and it's exactly as detailed in the SDK example. Could you try completely removing all NDI installations and files from your system and reinstall just the runtime from http://new.tk/NDIRedistV4Apple? Check that all version of libndi are removed from /usr/local/libfirst.
Hi David, i tried it on another computer (without already NDI installed) and it worked without problems. Max 8.1.3 and an older MacOS HighSierra 10.13.6
I tried to uninstall NDI, but unfortunately there is no uninstaller, so i removed all the files from Library>Application Support>NewTek and the /usr/local/lib/libndi.4.dylib (the only NDI related file in that folder i could find)
After restarts and installing the drivers again it did't work...
Are there any other locations i have to remove and reinstall the NDI drivers? It's clear it is something on my computer, conflicting with drivers or so..
@Aartcore Not sure of any other relevant locations. Essentially there's two steps to the loading process, finding the library file and then loading the named function. Would be interested to find out which of those is failing.
Thanks! This is great, have been working on some building-encompassing AV interactive art, NDI is far from perfect but is certainly handy! Bravo!
hello david, is it possible to stream peer2peer (from max to max) with the NDI externals? like with netsend~ etc. ? thanks for any hints
@klaus Yes the jit.ndi.receive object can receive from the jit.ndi.send object. Should likely have better performance than netsend as well.
thanks for the quick reply.
sounds great, but i did not find out how to from the help files.
can i connect to an IP with jit.ndi.send~ ?
@klaus NDI doesn’t work with IPs addresses directly. Sources get advertised on the local network using host name and source name. Set those two attributes on the receive object to match the send object.
ah ok, thanks for clarifying this.
so with a VPN it should work out of the house...will test this very soon.
jit.NDI via VPN does not work out of the box. the VPN needs to be open for bonjour recognition (or such).
any insights in how to setup this?
@klaus NDI wasn't really designed to be used over the internet as the bandwidth requirement for an HD stream is pretty high (around 100 mbps). I know there are a couple of products that allow NDI routing over the web (for example), but the price is pretty high as they're quite pro-oriented.
if we send jitter matrices like the bball.mov in the helpfile, is it blown up to HD ?
@kalus Nope, the matrix is sent at the exact dimension it is when passed to the object.
Hi David, same issue as @aartcore when ever I instantiate the jit.ndi.receive~ I get this message Unable to load NDI runtime library. Please download and/or reinstall NDI runtime from 'http://new.tk/NDIRedistV4Apple' and restart Max
I've downloaded the NDI runtime and installed, restarted my machine, still no success.
@Aaron seems everyone is having this issue apart from me. I have a github issue up for it now, https://github.com/impsnldavid/jit.ndi/issues/6, check there for updates or if you have any useful information to add.
Thank you so much! I had stopped doing my regular search on this forum hoping that someone would be kind enough to come up with this.. util today. You made my day :) I will start testing.
@Aarron I had the same problem as everyone. Got it working like a charm as soon as I have updated to Catalina.
gonna try to use this to send my macbook's built-in camera to my windows machine.
Thanks to help from Rama and Todd Ingalls on the Max slack, the issue with loading the NDI runtime on Mojave seems resolved. Updated package below:
Another new release, new features this time. jit.ndi.receive~ can now control NDI enabled PTZ cameras. Additionally jit.ndi.send~ can optionally act as a 'virtual PTZ' camera. This means that it will advertise the NDI source as having PTZ capability, and any received PTZ control data will be output from the object. You can then hook this up to whatever you want in Max. In the help patcher I've shown a basic example of interacting with a jit.gl scene.
Thanks for the update! It's working fine now on my computer!
Hi David, super useful externals!
Quick question: I notice quite a long latency (maybe half a second) when using this with the NDI virtual camera -- is this something innate to NDI virtual camera or is it something we could tweak in the jit.ndi.send~ external?
Thanks!
Graham
@Graham Wakefield
Shouldn't be, the control values are output immediately upon being received. Do you get the same result when testing on the same machine (ie via loopback)?
Thanks David!
It *is* on the same machine... but, the system's wifi is on (it is part of a larger system in which other parts need internet access), so I guess it may be bouncing the video stream to the wifi router and back even for same-machine streams. I did notice in playing around that streaming between machines on the wifi network has the same latency as a single machine streaming to itself.
Could there be a way to configure the jit.ndi externals to use a specific network, such as loopback, without disabling wifi? More generally, if there are several network providers on a machine (e.g. wifi, wired, loopback), how can we tell it which network device to use?
Thanks!
@Graham Wakefield
NDI doesn't allow you to specify specific network adapters. It will actually try and use multiple network adapters (to increase bandwidth) as long as there is a route to the destination available. The only way to prevent NDI traffic on an adapter is to use the firewall specifically block access to certain ports.
However I'd be very surprised if traffic is being routed via your wireless router as NDI should automatically use the loopback adapter in this circumstance. Try disabling your WiFi adapter and see if the result is any different.
Hey, David. These objects are fantastic - thanks for your excellent work!
One small issue - the NDI Runtime does not install automatically when creating an instance of jit.ndi.send~, and the URL it's trying to pull the install from doesn't work even when entered manually: http://new.tk/NDIRedistV4Apple
Just a "not found" page.
I was able to install the macOS runtime provided with the obs-ndi package, and that did the trick for the Jitter NDI objects.
I find it very strange - there's no NDI Runtime provided by Newtek, and the top results when searching on Google are from the obs-ndi package. Is NDI Runtime something that has to be packaged and provided specifically by developers like you on a case-by-case basis or something?
@Eric Usually the runtime would be packaged by developers, however in this case I'd have to package jit.ndi in an installer which I'd prefer not to do as eventually I hope to have it distributed via the Max package manager. Looks like NewTek have changed their download links though annoyingly, I will correct them in the next version.
The easiest way to get the NDI runtime is to install the NDI Tools, which I would highly recommend anyway as this package has all the test applications and allows things like desktop capture and use of an NDI source as a virtual webcam for use with other software.
Hmm. I'm pretty sure I had the NDI Tools installed before trying jit.ndi, but it's hard to say at this point. The NDI Tools installer for macOS is weird because it breaks each utility into its own installer (not sure if the Windows installer does the same). I have the Virtual Input and Video Monitor installed, but did not bother with Access Manager, AdobeCC plugin, or Scan Converter as I didn't think they were relevant to me. Perhaps the runtime stuff I needed was in the Access Manager installer?
Regardless, thanks again for your incredible work, and I hope jit.ndi gets added to the Max package manager soon!
Thank you for creating this - unfortunately having a slight issue getting it to work. On a Mac using 10.15.5 Max 8.1.5
The first time I created the object Max opened a message about it being quarantined and that it would try and fix it. Unfortunately I did not copy the exact message and it only happened the first time so I can't be more specific.
The object now appears but has no inlets and throws up the error message in the max window - "Can't find NDI runtime library - 'dlopen(/usr/local/lib/libndi.4.dylib, 5): image not found'. Please download and/or reinstall NDI runtime from 'http://new.tk/NDIRedistV4Apple' and restart Max"
That link unfortunately no longer works. I have downloaded the NDI Tools and NDI Driver. I cannot find a NDI Runtime anywhere to download.
Any help will be most appreciated!!
Thanks,
Stefan
@Stefan If you've installed the NDI tools then you should have the runtime. Can you check that the dylib file specified actually exists at that path?
Its not at that path - nor does a find in terminal find the dylib file.
The NDI Tools work - or at least some subset of them do - I can use the NDI IOS Camera App and see my iphone output in the NDI Video Monitor.
@Stefan Try uninstalling the tools, and reinstall the latest version from the NewTek site. Also make sure you've got the most recent version of jit.ndi.
Hey, David.
One other thing that might be helpful for users of the jit.ndi objects:
OBS Studio does not always appear to play well with the alpha channel in NDI sources. The NDISyphon standalone app has an alpha channel enable/disable checkbox that helps to fix this, though of course that requires using Syphon instead (and going though an intermediary application). Would it be possible to add an alpha enable/disable to a future version of jit.ndi.send~?
@Eric If you send a 3-plane matrix in ARGB colour mode, then it will be sent as NDI RGBX frame type. A value for alpha does get written here but it's 255 for every pixel. It's then up to the receiver to examine the NDI frame type and decide what to do with this, the correct implementation would be to deserialize the NDI frame as a non-alpha texture but obviously software can vary.
You can also use UVYV colour mode, this doesn't support an alpha channel.
@David When you eventually put the jit.ndi external into the package manager, are you planning on publishing it with the NDI Runtime as a whole package? Just curious, we would love to integrate this into the Signal Culture app suite, but i'd like to avoid user confusion about which runtime to download.
And thanks for your work on this, pretty fantastic stuff!
@Jason Bundling it with the runtime would necessitate running an installer which as far as I'm aware is not possible for a package manager installation.
On Windows, the runtime is pretty clear cut. It's only on Mac that there's confusion as the official link to the Mac runtime which is embedded in the NewTek SDK is broken! Basically just link Mac users to the NDI Tools package and you'll get the runtime with that installation.
Hi @David,
I'm testing your jit.ndi externals, and I find that the quality of the video in Max is much lower than the quality I'm receiving in the NewTek NDI Video Monitor. When I enable "low bandwidth" in the NDI Video Monitor, the quality appears to be equal to that which I receive in Max.
I've played with the low_bandwidth message into jit.ndi.receive~, and it does not seem to change the quality of the video in Max. Is it possible that the jit.ndi object is currently only receiving video in low_bandwidth mode? Or could you suggest another direction for me to test?
Thanks for your hard work on this project.
[edit] for reference, I'm on OSX 10.13.6, and jit.ndi.receive~ is V0.2.0.
Just to update. I'm not super familiar with Jitter. Running the [jit.ndi.receive~] matrix output into [jit.matrixinfo] tells me that I'm receiving 640px x 360px video. But I am sending via OBS-ndi, as far as I can tell, 1920px x 1080px video.
It appears to me video in the NDI Video Monitor is being received at the correct resolution. Is there an option I'm missing on the [jit.ndi.receive~] side of things?
@Peter jit.ndi.receive~ creates the output matrix at the native resolution of the NDI stream. I've just done a quick test playing an HD media file from OBS, receiving in Jitter, and I got the same resolution as the source. What is your OBS configuration?
@David thank you for your help. Your description of jit.ndi.receive~ functionality is as I had thought. It's possible I've made an error in my OBS settings! However the OBS-ndi stream received by NDI Monitor looks good.
I am attaching here screenshots of the situation. The image in the stream, it is just a test feed from an IP camera. Thanks again for your insight to my situation.
@Peter What version of obs-ndi are you using?
@David I'm running the latest: OBS-ndi 4.9.1, with OBS 25.0.8.
I did a test using Newtek NDI Scan Converter to share my desktop over NDI. The stream appeared in Jitter with the same 640x360 resolution that I had received from OBS. So given that, and that my OBS stream appears fine in Newtek Video Monitor, I think it's possible to rule out a problem on the OBS side?
Bit of a head scratcher!
@Peter Do you get the same result when receiving an NDI stream sent from jit.ndi.send~?
@David -- good idea. I've tested sending 1080p video out of jit.ndi.send~ into jit.ndi.receive~, and the same problem persists. Note screenshots, where 1080p is received as 640x360.
On the other hand, the default "bball.mov", a 320x240 video, is correctly received by jit.ndi.receive~ as a 320x240 stream. Strange--it gives the impression that I have a setting somewhere that caps the resolution at 640x360.
@Peter Would you mind creating this as a Github issue so I can track and investigate?
https://github.com/impsnldavid/jit.ndi/issues
@David sure thing. The issue has been added:
I'll be receiving a 2010 Mac Pro with Yosemite in the next couple days, with which I intend run this patch. So I'll be testing on that system, and will update my bug report with the results.
Thanks again warmly for your help!
FWIW Max 8 minimum requirement on Mac is 10.11 - https://support.cycling74.com/hc/en-us/articles/360051184013-System-Requirements
For anyone else tracking this here, I can reproduce this problem, but it only affects Mac OS. The issue is that the NDI runtime is returning the incorrect dimensions.
@Rob thanks for the info! I will update that Mac to 10.11.
Just to update anyone tracking the issue in this thread. LSka responded to my Git issue, with this info:
Hi, I tried the test patch and I'm getting 1920x1080 stream in jit.ndi.receive~
I'm on Max 8.1.6 MacOS 10.14.6, NDI runtime downloaded from here: https://ndi.palakis.fr/runtime/ndi-runtime-4.5.1-macOS.pkg
After re-installing NDI from LSka's provided link, and rebooting, my 1080p stream is correctly received as 1080p video by [jit.ndi.receive~].
Thanks again everyone for working to resolve this issue.
I'm putting together a large remote orchestra project and I've gotten this to work with Max but I'm wondering if this is doable in Ableton, has anyone tried this yet? I'm only routing audio.
Great device!
There's no reason it wouldn't work in Ableton. However, I would suggest that NDI isn't the best option for sending audio over a network. Something like AVB or Dante is far more reliable and capable.
For my project I pretty much have to use ndi from skype as it's the only widely available conferencing software that allows me to pull everyones audio into separate channels for mixing. I'm trying to make a multi channel device in Max to do this. Bad idea? Or is there a better step to have in between?
I was trying to use OBS but I need to have that available for streaming.
@TREVOR have you heard of Jamulus https://jamulus.io/ ?
Following thread...
@LSKA I do know of Jamulus, it's a great program and I' musing Jacktrip and Sonobus for the more tech savvy members of the orchestra, but i'm trying to keep things as simple as possible for the others. This is working in Max for me right now but maybe I don't have the external placed correctly for M4L, it keeps crashing. Everything else is running great and I'm headed into phase 2, so to speak, of getting the project off the ground.
It's seem like NDI runtime is not available anymore.
Somebody have the installer ? :-/
this link still seems to work: https://ndi.palakis.fr/runtime/ndi-runtime-4.5.1-macOS.pkg
my assumption is that installing NDI tools installs the runtime...
newobj: jit.ndi.receive~: No such object
Catalina issue ? 10.15.7
I removed the Catalina packages security...
Hi all, I resolved the same issue with "Can't find NDI runtime library - 'dlopen(/usr/local/lib/libndi.4.dylib, 5): image not found'. Please download and/or reinstall NDI runtime from 'http://new.tk/NDIRedistV4Apple' and restart Max"
I have solved it this way:
1) Download the SDK from here: https://ndi.tv/sdk/
2) Install it
3) Look for it in your system Library: /Library/NDI SDK for Apple/lib/x64
4) Copy libndi.4.dylib to /usr/local/lib
5) Restart Max
hi there,
i got everything running with one issue:
jit.ndi.receive only works when running "NDI virtual input" .
the menu shows my cam (android) as well, but selecting it stays black.
not sure if this should work and there is this issue with libndi.4.dylib:
i never saw this file anywhere, even now following the above post from gpvillamil i got
/Library/NDI SDK for macOS/lib/macOS/libndi.dylib installed (with the missing "4")
i just changed the name in /usr/local/lib/libndi.dylib to libndi.4.dylib and the external was working then.
but maybe thats the reason why i cannot access the cam directly..
maybe someone can provide a direct link to the real libndi.4.dylib ?
best
klaus
I did not know why the patch started to slow down from 60 fps to 15 until I realized it was from jit.ndi.send~. The framerate of jit.ndi.send~ was set to 4:30. I set it to 8:60 and now the framerate of the patch is 30 fps.
Any idea why jit.ndi.send~ keeps the fps of the patch to half the framerate of what you set in jit.ndi.send~ @framerate ?
Thank you!
Hi, Just having a look at this. Am I correct in assuming that things are broken with NDI 5?
As the NDI runtime is loaded dynamically, everything should still work with NDI 5, although I haven't updated the SDK headers yet so any new features which require this won't yet be supported.
Thanks!
Anyone else experiencing fps getting lower as soon as one opens jit.ndi.receive sources? On a slower laptop, a fullHD stream is all it takes for the fps to drop from 60 to 14. On a more powerful laptop, it takes a couple of streams.
Do you think it's related to NDI in Max or NDI in general? I used NDI in OBS as well, but unfortunately I don't know how to check fps there so I can properly compare.
Please let me know what is your experience.
Thank you!
Would the max object jit.ndi.receive~ receive on the new M1 pro macbooks?
- max doesn't recognize the object in my patch. It worked just fine on my previous laptop.
Thank you
E.
I think the object has to be compiled for M1.
In the meanwhile, you should run Max with Rosetta.
New update, v0.3.0. Now uses the NDI 5 headers (more for keeping up to date, this provides no new features and the previous version will use the NDI 5 runtime if that's what you have installed).
Also now supports Apple Silicon, however I haven't tested this as I don't own any Apple Silicon hardware. If someone could give this a try and report back I would be grateful. Obviously will require a compatible install of the NDI runtime.
it works ! VERY nice ! Thanks for being so fast David and so kind to make this happen. (Using a macbook with the "M1 pro" apple silicon chip)
----- a question though:
Now I'm sending NDI (video and audio) between my Iphone (using ZIGSIMpro app) and macbook with Max. It's still a little bit laggy, and the image gets stuck sometimes, also in lower quality. Would it be because of the wifi-network I'm using, or do you have another solution to have it working more smooth?
I don’t know that app, but it would depend both on the available WiFi bandwidth/signal quality and also whether the app uses NDI or NDI-HX. Unfortunately good quality wireless video is really hard without specialist equipment.
thanks david, for your work and the update, it works here on my 13" M1.
i was using 7 android ndi-hx-cams together with some jit.ndi.receive~s in a production very recently.
i had to change from wireless to ethernet to make it work perfectly without any glitches.
(before that we found out, that switching from 5G back to 2.4G Wifi gave better results as well, but this maybe depends very much on the architecture of the room..)
Does it work for you with NDI Bridge? I mean I get the NDI streams from the other place using NDI Monitor or OBS, but in Max I just see the streams advertised, but if I select them, I get no image.
Any ideas? What's your experience?
running on M1 (native, without rosetta), opening the helpfile i get the error:
mubutrack.javascript: could not load due to incorrect architecture
jit.ndi.receive~: mubutrack.javascript: No such object
... took me a while to find out, that this is caused by the "comment-looking" JSUI
grabbing the Arguments via helpargs.js .
so nothing that affects jit.ndi's functionality.
@YGREQ
do you get any errors in the max-window?
@YGreq I have not done any testing with NDI bridge, but my understanding is that it is just effectively a software NDI receiver that translates streams to a lower-bandwidth format for internet transmission. It shouldn't impose any specific requirements on the sender.
@Klaus I saw this issue when briefly testing with Max 8.2.0, upgrading to 8.2.1 sorted it.
@Klaus I do not get any errors. Did you also test with NDI Bridge and works for you?
@David, I will create an issue. Maybe someone else can see the issue and test and confirm.
what is NDI Bridge ?
Hi Klaus!
NDI is for local use, meaning you can stream audio-video between devices on the same local network. NDI Bridge offers the possibility to connect multiple local networks and thus share devices from one local network to the other local network
You can find more info here: https://www.ndi.tv/tools/bridge/
So did anyone have the chance to test NDI Bridge with jit.ndi?
hi ygreq
ah ja, thats what i wanted from jit.ndi a year ago.
i can offer to test this remotely with you.
if you want lets connect via email.. klaus@klingt.org
oups,
ndi bridge currently only works on windows, right?
Thank you so much, Klaus. I have a collaborator I can test the set up with. This is how I found out that OBS and NDI Monitors work with NDI Bridge, but jit.ndi in Max does not.
I was curious if anyone else tested NDI Bridge with their own setup.
Also related to jit.ndi, any idea how I can send commands to my PTZ Cameras? I wanted to use a joystick to move the camera around.
Thank you!!!
If the NDI source supports PTZ over NDI, the PTZ attributes on jit.ndi.receive~ will be enabled. You can test this with the jit.ndi.send~ and jit.ndi.receive~ help patchers,
Thank you, David! I don't know how I missed the extra patches for PTZ devices.
I wanted to test that function, but it seems I have 2 problems regarding NDI in Max. I have 2 PTZ Cameras. I have no problems with them in NDI Monitor and in OBS as you can see from the images I attached. But in Max, only 1 out the 2 PTZ cameras are advertised (meaning I can see them if I press getsourcelist or getsourcelistmenu) and none of them output any video stream. :/
Considering the fact that these cameras are NDI HX cameras and also considering my earlier observation regarding NDI Bridge which changes the steams from NDI to NDI HX, and I didn't get any streams via NDI Bridge either, my guess is that this problem that I can find only on Max (and not in OBS or NDI Monitor) is related to this fact that maybe jit.ndi does not support NDI HX. It's a wild guess, but maybe you can look into it.
I will again test tomorrow the NDI Bridge and will try to use pure NDI (not NDI HX) and see if I still have problems with it.
Thank you so much!
