jit.ndi - Externals for sending video/audio over the network
I managed to get video data as well in one out of 3 laptops from the one PTZ camera that was advertising as well. I still can't see the other one even on this laptop. Weird that I don't have these problems in OBS and NDI Monitor. Any suggestion are still very much welcomed.
Anyway I tested the PTZ control functions in jit.ndi. It seems I can only use ptz_pantilt_speed, but ptz_pantilt and ptz_zoom are not working. Any idea why that is? Thank you!!
I created a demo here: https://youtu.be/3tsdetR8tfs
It's possible that the PTZ hardware device doesn't support directly setting the pan/tilt/zoom/focus from a discreet value, which case you have to use the 'speed' attributes. (@ptz_pantilt_speed, @ptz_zoom-speed, @ptz_focus_speed). These don't directly set their respective properties, they instead adjust the rate at which they should change. This is equivalent to a PTZ joystick or zoom/focus slider which returns to a centre position when you let go.
Great!! Makes sense now. I managed to control that specific camera. Thank you so much.
I still gotta figure out why the other camera cannot be seen in Max. :/
Any idea how @ptz_zoom_speed works? If I send something bigger than 0. like 0.3, I see it starts to zoom in. If I send 0. it stops zooming. But I don't know how to zoom out. If I send a negative value like. -0.1 or -0.3 or any other, it does not zoom out.
Looks like you found a bug! The minimum value for that attribute was specified incorrectly. Have fixed, will be in the next release in a couple of days.
Grrreat!!
Related to the other problem, is it possible that the fact that the PTZ camera's firmware is old, jit.ndi might not see that specific camera?
Ok. I believe I finally have the actual proof that jit.ndi.receive~ does not receive NDI HX sources. Or at least, it has issues with most of them.
Like I was mentioning before, my 2 NDI HX cameras have issues in Max, but not in OBS or NDI Monitor. Also NDI Bridge has issues in Max. I wrote about the problem here: https://cycling74.com/forums/errors-using-ptz-cameras-with-jit-ndi-receive~
But now, with the help of a friend, we noticed something else. NDI 5 comes with a few programs. One of them is Screen Capture which send NDI data of a captured screen. But there si also a program called Screen Capture HX that captures the screen and creates an NDI HX stream. We tested both options of capture and tried to play the stream on the same laptop using either Max with jit.ndi.receive~ or NDI Monitor.
While capturing a normal NDI stream worked on both NDI Monitor and Max, capturing an NDI HX stream did not work on Max while still working on NDI Monitor.
So the conclusion is that jit.ndi.receive~ definitely has some issues with NDI HX streams.
Here is a demo where we show how jit.ndi.receive~ reacts to both normal NDI and NDI HX: https://www.youtube.com/watch?v=BPB8_D7CZxk
May I ask that you replicate the test and see if you have the same issues?
Thank you so much!!
I have tested with the iOS NDI HX camera app, and with the Screen Capture HX (on Windows), and could not replicate this issue. I would note that as far as jit.ndi.receive~ is concerned, there is no difference between NDI and NDI-HX sources. They are both discovered and received via the NDI runtime library in exactly the same way. I'm not sure if it's even possible to determine from the runtime library whether a source is HX or not.
Hence, if you're not seeing HX sources in Max, then the problem is going to be NDI runtime library related. I'm not sure how the NDI tools applications link to the runtime, but jit.ndi is dynamically linked to the system-wide NDI install, meaning that it will use whichever version of the NDI runtime which is installed on the system. If the NDI tools applications link statically then they might end up using a different version of the runtime which would explain the different result. It's also worth checking that you've definitely installed the NDI HX driver with the tools, as this may also be something which the tools statically link but other applications need to access dynamically.
Thank you, David. I didn't want to respond before trying a few more things. I still didn't manage to fix this issue. It seems I have the same problem with another PTZ NDI HX camera from PTZ Optics.
And weirdly enough one of the 2 cameras does not work on all 4 laptops I tested and the second one for sure works on 2 of the 3 laptops I tested. I mean it gets advertised and I can also get the stream.
Anyway I will look further.
But I wanted to ask you. From my understanding jit.ndi.send~ sends normal a NDI stream. Is there a possibility to send an NDI HX stream instead?
Thank you!
AFAIK Newtek do not make HX encoding available in the public SDK.
Something that might be worth checking is that mDNS (and multicast traffic generally) is being received on all your laptops. This would affect discovery. You could also try running an NDI discovery server instance (part of the Windows SDK download) to check discovery using a non-multicast mode.
I managed to solve the problem. The cameras I had did not have a setting to change the name. Thus both cameras had the same name and this caused the issues. I talked to the developers of those cameras and they game a tool to change the name of each camera and thus the issue was solved.
Did you manage to fix the bug with the ptz_zoom and ptz_focus acting weird? If not, may I ask that you fix that? Thank you so much! And thank you for all your help!
Hi, David, and anyone else reading this :)
It seems the fix I mentioned in the last comment resolved one problem where jit.ndi would not see both cameras at the same time.
But I still have the following problem. On 2 out of 5 laptops I tested, jit.ndi does not see any image from streams that are NDI HX. I even tested with steams coming from the same laptop from NDI Screen Capture HX. I reinstalled Max and jit.ndi, I reinstalled NDI 5 SDK and Tools, I disabled the Firewall (just in case even though it does not see HX steams from the same device). This problem only affects Max and jit.ndi. I don't have this problem using OBS or NDI Monitor. Considering I don't have this problem on the other 3 laptops, I just don't know where the problem is coming from. I am thinking of installing Win 10 again and start from scratch. You mentioned mDNS, but would it affect an HX stream coming from the same laptop? Also wouldn't it affect normal NDI streams as well and also create the same problem on other apps? Any idea what else should I check?
And please, please fix the bug with ptz_zoom!! Thank you so much!
Updated release with the PTZ fixes above.
Thank you so much, David. Much appreciated!
Related to the previous problems I mentioned, I forgot to add that when getting HX streams from PTZ cameras, even though I see only black, I can move the aforementioned cameras.
That would suggest that the NDI runtime is failing to decode the H264 component of the stream, which could indicate an issue with the NDI HX driver.
Thank you, David. But wouldn't that mean I should have the same problem in OBS or NDI Monitor? Which I don't. I only have this problem in Max.
Like I was saying above, this problem only affects 2 out of 5 win machines I tested on and the problem is only related to jit.ndi and Max.
I reformulated the whole problem to have more sense. Hopefully.
I have a weird problem where I cannot see HX streams in one application out of 3 tested. -I have tested NDI on 5 Windows machines. I have the following problem on 2 out the 5 Windows machines.
-The problem affects only the Cycling 74 Max application, but not OBS nor NDI Monitor.
-I cannot see the HX steams (even HX streams created on the same machine - for example Screen Capture HX) on those 2 machines. I can see HX streams on the other 3 machines. Normal NDI streams just fine.
-On PTZ camera that output HX streams, even though I only see black (so no image), I can control those specific cameras
Any idea what could be wrong that it affects only HX sources, only on some Windows machines and only on one application, but not the other?
I reinstalled Windows on one of the devices where NDI HX is not streamed in Max. Still the same issue. Here are 2 videos where the problem is shown.
Video 1 with a laptop where everything runs smoothly. NDI HX is streamed in Max: https://youtu.be/FBB_vkniA4s
Video 2 with a laptop where NDI HX is not streamed in Max, not even the HX stream from the same laptop. OBS and NDI Monitor do get those streams.
https://youtu.be/RPnE0SQH69c
Sorry. No audio as I had some issues recording it on OBS.
I recently reinstalled Win 10 on one laptop where I have no issues and on another one where I do. I reinstalled afterwards Max, jit.ndi and NDI Tools. I have exactly the same issues as before. On the laptop where it worked before, it works now. On the laptop where I didn't get any HX NDI images in Max, I still don't get them.
I have to mention that on the laptop where I have no issues I only installed NDI Tools and not NDI DSK (which comes with NDI Runtime). And it worked. No issues whatsoever including in Max. So I did the same on the other laptop. Still no HX images in Max, but with normal NDI images in Max. And both normal and HX NDI in NDI Monitor.
This is weird as hell.
I also activated mDNS and still nothing.
Hi, David and everyone reading this.
I reinstalled Win 10 6 times in the last days. For science! :) I reinstalled Win 10 and the necessary apps (Max, jit.ndi and NDI Tools) on the 2 laptops again. And I recorded the screen just so you can see the weirdness.
I started with the laptop where NDI in Max worked before. And after Win 10 finished installing, I immediately installed Max , opened Max, then copied jit.ndi. I then installed NDI Tools. After NDI Tools installed, I opened NDI Monitor and tested to see if I could see the HX streams coming from the 2 cameras. Everything fine there. Then opened Max. It didn't work as I wasn't able to see external NDI sources. I restarted Max and all the sources became available. Including HX streams coming from the 2 cameras. I even tested with NDI Screen Capture HX coming from the same laptop. Everything worked as expected. No issues at all. Every software I used is last version. You can see the demo here: https://youtu.be/LUSjnTNOXZ8 . I recommend 2x speed playback.
I then reinstalled Win 10 on the laptop that had issues previously where Max couldn't see video from HX sources. I followed the same steps and used exactly the same software (including the same Win 10 usb stick): waited for Win 10 to finish installing, installing Max afterwards, copied jit.ndi, installed NDI Tools, opened NDI Monitor, opened Max, closed and reopened Max. And Max again has the same issue. It does not see HX streams from PTZ cameras as well as HX streams coming from NDI Screen Capture HX running on the same laptop. It does see normal NDI streams. And NDI Monitor does see normal as well as HX streams. You can see a demo here: https://youtu.be/YPMqQW41hDk
Any suggestions or opinions are truly welcome. I am considering writing Newtek and presenting this situation. But considering it is related to Max, I don't know if they can do anything about it. What do you think?
Thank you so much!
Updates as they happen. The laptop where I had no issues with HX streams in Max has a Nvidia GeForce GTX 1060 GPU. The laptop where I had issues with HX streams in Max has a GeForce GTX 2070 GPU. This is important because this is what I found.
On the laptop with a GeForce GTX 1060, like I was saying earlier, I reinstalled the necessary software for testing HX streaming in Max and everything was working. I forgot to mention that the GPU drivers were installed automatically by Windows. This happened before I had the time to make the tests. So the tests were done with GPU drivers installed automatically by Windows. After the tests I also installed GeForce Experience app and updated the GPU drivers via that app. (GeForce reported I had at this point v511.23 , but the Device Manager reported I had v30.0.15.1123 - I don't know why they report differently). Once I updated the GPU drivers, I realized that jit.ndi stopped streaming HX streams. Just like on the other laptop which has a GeForce RTX 2070 GPU. I went to the Device Manager and unrolled the drivers for the GPU card (and now the Device Manager reports I have v22.21.13.8205). I restarted the laptop and now I get HX streaming inside of Max once more. So definitely HX streams are affected by the newer GPU drivers.
On the other laptop that boasts a GeForce RTX 2070 GPU card, the drivers installed automatically by Windows are probably newer and this is why they don't work with HX streams in Max either. I will try to update the GPU drivers there as well, but my guess is that I will have to use older drivers in order to make it work. And I hope I can find older drivers.
As an extra note, my colleague told me that his laptop that had similar issues also has a RTX 2070 GPU card. Surprise, surprise!
As a conclusion, for sure the newer GPU drivers are creating this issue where I get black instead of HX streams in Max via jit.ndi.receive~. And I am asking what I should do in this situation other then using older drivers? Should I also talk to Nvidia or maybe Newtek. I have to bring again the fact that the only app affected by this GPU driver / NDI bug is jit.ndi. NDI Monitor and OBS work just fine. I need your advice! Thank you so much!
PS:
These are the drivers I used and logged after tests:
Laptop 1 Clevo P751 P751DM2-G
Configuration
CPU: Intel® Core™i7 Quad Core Processor i7-6700k (4.0GHz) 8MB Cache 91w
GPU: NVIDIA® GeForce GTX 1060 (N17E-G1) 6GB GDDR5 Video RAM
RAM: DDR4 SO-DIMM 2x16GB HyperX IMPACT 2133MHz SODIMM DDR4
Drivers tested:
Not OK v30.0.15.1123 (which is the last version for this laptop)
OK v22.21.13.8205 (This is the rolled back version after I tested the one above. I believe this is the version that Windows 10 automatically installs)
OK v21.21.13.7620 from 12/2/2016 (installed from the laptop manufacturer package of drivers)
Laptop 2 Dream Machine
CPU: AMD Ryzen™ 9 3900X 12 Core 3.8GHz Up to 4.6GHz Boost Clock 64MB L3 Cache
GPU: NVIDIA® GeForce® RTX 2070 - 8.0GB GDDR6 Video RAM
RAM: 2x32GB Corsair VENGEANCE 2666MHz SODIMM DDR4
Drivers tested:
Not OK v27.21.14.5720 from 11/2/2020 (installed automatically by Windows 10 and also available from the laptop manufacturer package of drivers)
Not OK v10.0.19041.868 from 6/21/2006 (rolled back drivers for generic graphics card)
Not OK v30.0.15.1123 from 1/10.2022
Not OK v27.21.14.6677 from 6/8/2021
Or maybe I can disable some specific feature in the NVidia Control Panel..
For anyone interested, I still have no image from NDI HX. I use jit.ndi to control the camera and I use another jit.ndi to get the video stream from the camera via another app called Zen NDI RTx: http://www.zenvideo.co.uk/ndi.htm
It's been a while since last update. Has anyone else tested NDI HX devices and let me know if you have live image coming from your NDI | HX devices into Max?
Thank you!!
Unfortunately there's nothing I can actually do in terms of an update to fix any issues with HX sources working. The NDI SDK has zero visibility into the inner workings of NDI HX as it's entirely controlled by the NDI runtime.
What I have discovered which may or may not be of use to you is that the NDI tools applications don't actually use the installed NDI runtime DLLs, they use a statically linked runtime as part of their binaries. This means that testing with the NDI tools is not actually equivalent to running an application which uses the system-wide installed runtime (which jit.ndi does). Additionally a lot of applications supporting NDI use their own local bundled copy of the runtime DLL rather than the system-wide one. The 'Zen NDI RTx' application you mentioned above does this for example.
The upshot of this is that testing with other NDI applications isn't necessarily testing the same configuration as jit.ndi. I would be interested to see if the problem you're experiencing still occurs when testing with an application which loads the NDI runtime in the exact same way as jit.ndi.
Thank you so much for the extra info.
Do you happen to know of an application that loads the NDI runtime in the exact same way as jit.ndi? :) I will happily test it.
PS: In the meantime I also am using OBS as an alternative for ZEN. I get the NDI | HX stream inside of OBS and then send that as a new High Bandwidth NDI stream that I create with the help of an NDI filter
@David a few years ago in this thread you mentioned you would add to the help patchers advice on showing an optimal way of adding texture conversion. I am wondering if you might have some suggestions since I don't see this has been added to the help files. And many thanks for your great work!!
If you look at the jit.ndi.send PTZ example in the help file that gives an example of outputting from jit.world.
Thanks for the quick reply. I meant to ask about jit.ndi.receive~. But now I am wondering if the object is already outputting a texture since I can send it directly to jit.world.
jit.ndi.receive outputs a standard matrix. jit.world will convert this into a texture, the classic way is to use jit.gl.videoplane.
Perfect! That's exactly what I thought, but it's great to have your input.
And one more question... I am working on an NDI router in Max in order to route a dozen NDI live performers from a call-in system called LiveToAir for live streaming performance. I want to make the setup as efficient as possible because the CPU load can be extensive for multiple NDI inputs in Max. I only need four live performers at any one time, but the routing allows me to rotate 12 performers from the call-in system into four on-air slots. I am bringing the performers into Max in order to composite them in virtual sets. I'm wondering if you have any suggestions for minimizing CPU load such as, for example, switching off one or more of the four inputs if they are not being used, or perhaps strategies for integrating the four independent textures.
I am only receiving NDI, there is no sending involved. The final program feed from Max is sent via the Jit.world screen output converted to SDI for encoding.
Many thanks and sorry if this is too large of a question!
Your performance bottlenecks in this process will be the actual NDI decode (which is reasonably efficient but still has a cost), and then the GPU texture upload. Minimise these to the minimum of what you actually require at any one time. Providing you don't need to switch between guest feeds whilst live, then only use 4 jit.ndi.receive~ objects, and switch them out by changing between NDI sources. Additionally, if you're only integrating guest feeds into a 3D scene and don't plan on ever showing the feeds full frame, then you could probably reduce the NDI source resolution considerably, which will help performance.
You are right, there are only four performers (guests) "on stage" at any one time, so I can switch the sources accordingly, much like a 4 voice allocation synth. Sometimes, a single performer might be in two or more "on stage" slots in order to multiply their image. So it might be possible that one performer takes up two slots with two other performers each taking up one slot. LiveToAir has settings for different resolutions for NDI output so I am able to experiment.
When I have a working draft of the router I'll post it here for everyone to see and comment on.
Many thanks!!!
For anyone having trouble finding the NDI library necessary to run the objects, I made a video showing the process.
I'd like to ask David if there's actually a better way to have the objects working
https://youtu.be/ws7EAPHhbzE
Many thanks!
@David, the link you provide to the NewTek NDI Runtime is broken.
NewTek are really terrible about maintaining these links, it's not the first time they've gone down. According to the below they're aware of the problem and will fix it. These URLs are actually embedded in their SDK source code so it's intensely frustrating that they don't take better care of keeping them active.
https://forums.newtek.com/threads/http-new-tk-seems-down-and-thus-http-new-tk-ndiredistv5-http-new-tk-ndiredistv5apple.167380/
I can't see any other obvious links on their site to the runtime, so I would suggest just to install the NDI tools package which includes the runtime.
@David have you ever been in touch with NewTek tech support about this? I understand why this is so frustrating, particularly since sometimes even after installing NDI Tools, the Jit.ndi externals are still not active, i.e. Federico's experience.
actually the NDI runtime can be found, for Windows 10 an up, using the `winget` command (which is the official equivalent of `brew` on windows), the command line to install it is: `winget install NDI.NDI5Runtime`
I don't know if the runtime can be installed the same way on OSX with `brew`... BUT there is another solution...
You can install the NDI Tools (which has installers for Windows and OSX) and it will install the runtime binary and some useful tools too...
That's how I got it to work...
Where can we find the runtime for MacOS? Or has it been discontinued, i.e. only NDI Tools available?
It seems NewTek lost control over the domain for their shortened link, so it's now available at http://new.link/NDIRedistV5Apple. Would have been nice of them to actually announce this to developers, I only find out because a third party eventually posted it on the dev forum...
Hey there,
any clues where to find the runtimes anymore? Do they still exist or we need to install the NDI toolbox?
Cheers,
M
Maybe -
Mac - https://github.com/obs-ndi/obs-ndi/raw/d462e9f83f0e06837a83331b1f71053b2132e751/runtime/libNDI_5.5.3_for_Mac.pkg
PC- https://github.com/obs-ndi/obs-ndi/raw/d462e9f83f0e06837a83331b1f71053b2132e751/runtime/NDI%205.5.3%20Runtime.exe
B
<solved - set type to char>
Hi there, I'm trying to use jit.ndi.send~, it works perfectly when using the example file, or when plugging a jit.movie object into it, however when I try to plug in my output from jit.world, I get the error
" jit.ndi.send~: mismatch type "
My jit.world object is set to "output Matrix" enabled, I also tried setting it to "output Texture" and then running that through jit.gl.asyncread, but I still get the "mismatch type " error, I'm unfamilliar with this error and assume its not really related to the ndi node, but I'm not sure what the type mismatch could be. Any assistance is appreciated.
Hello!
Has anyone else had some audio clicking from NDISend?
It sounds like a clocking error but i can't for the life of me figure out what could be causing it.
In one application i'm encoding 4 live video streams into separate NDISends.
In another i've made a switcher that chooses which stream to view.
Audio via any other NDI source is crystal clear but from Jit.NDISend its very glitchy.
Thanks
Hello Ethan. I use Jit.ndi only for video, but I am curious how you have created your ndi switcher. I have a couple of switchers I have built in Max and it would be great to know how you approached it, since ndi uses a lot of resources and is taxing on CPU. thx!
Hi Randall
On the receiving end i've used jit.gl.layer and jit.pworld to send video rendering to the GPU when possible, and used ubuttons to select the NDI streams.
On the send, i've just 4 different named ndi sender objects .
Thanks Ethan, interesting to see how you are doing it. I'll post my router for comparison.
I just use a jit.matirix before the NDI output devices.
There are some new built in integrations for NDI with OBS-studio.
VZ.Grabber and NDI-Virtual input is how I get NDI signal back into MAX as a live UVC input.