Intel Realsense development
Hi Dear community!
First and foremost you guys rock, I've been using/learning from the community for a long time! Been working a lot with 3D cameras of various type in Max and recently got a hold of the newcomer Intel RealSense F200 and R200, unfortunately they are Windows based and I converted to Mac a while ago, still these two cameras forced me back to my PC and as it turned out at least the F200 is easy to use and comes with many end-user software to do most of the things that we've been doing with Kinect and Asus. OSCing the data from my PC to the Mac was my only working option so far and was wondering if anybody is working on a wrapper for Max, if so I will be gladly running tests. (hint: maybe Jean-Marc Pelletier or Dale Phurrough are thinking about another magical object by the name of jit.realsense?) Here is a single webpage with the drivers and SDK:
Thank you all and happy new year!
I briefly considered writing an object for it. I did research into the number of people using it. So far, I haven't read about any significant number of people using it to justify the time/effort I would need to put into a quality release.
If you know instead that there is a large number of people using it, please do share with me so I can re-evaluate my previous choice.
Thanks for you reply Dale, the number of people using the tech is definitely not even remotely close to Kinect users, that is expected though knowing that the F200 recently became available to purchase (a note to friends reading this, please pay attention to the specified range of this camera it's super short range, not usable as is for performances), having said that with the publicity behind it such as DevLabs all around here in the US, also the fact that they are making laptops with the built-in F200 and tablets with built-in R200 (the long range camera) it seems it might become the only choice for a portable better resolution USB powered 3D camera. In either case I'll be happy to help in anyway necessary if you decided to work on it.
It's rather sad to see that Cycling '74 team does not take care of these things. I recently took a closer look at TouchDesigner and I saw that they have objects that come with the package for every device possible, this one included.
Here in Max we need outside people like Dale to take from their own time to build these objects.
That is not very good. Not good for the company. And definitely not good for the users.
YGREG +1
@YGREQ I think it's a resource issue. I suspect increasing the marketshare of Max would resolve a few of these things.
Actually I wanted to address that in another post as well.
I actually find it very unwise for Cycling '74 to offer Max for only one month in demo mode. That is not enough time for a beginner to get their hands around Max in order to realise that buying it is worth the effort. I only became a Max user because of the fact I found a way to crack it. In the end I bought it. In case someone is suggesting it is cheap so one can afford it, 1 it is not cheap for many of us, 2 why buy it if you are not sure you are going to use it?
TouchDesigner comes with a free version that you use non-commercially indefinitely. Once you realise you want to make money out of it or you realise you want to use the the power features (that you don't have in the free version, like using a bigger resolution) you buy the app. Which costs lots more that Max by the way. In the meantime the user becomes a power user and everyone is happy.
The conclusion is that by adopting this strategy, Cycling '74 is loosing a lot of potential users from my point of view.
Good news!
Here is an interesting update, Intel realsense seems to be finally available to us Mac users! ( I haven't tested it yet though!) here is a link to the library
By the way, while (to an extent) I agree to what has been discussed still I think to rely too much on Cycling74 to provide and cover for every device out there is not realistic, I myself would prefer some tutorials on how to do such a task so we could do it for every device that we want to. I think that's the point of having a community after all.
Cheers
The link says:
"This project is separate from the production software stack available in the Intel® RealSense™ SDK, namely that this library only encompasses camera capture functionality without additional computer vision algorithms."
I think that means we get the camera stream inputs but don't get our hands on all the tasty tracking, gesture and scanning stuff sadly :(
you can just rent the software if it is interesting you? its quite cheap / month
There is already an external made for it , it gives you acces to the depth image but thats it https://github.com/blueyeti/jit.realsense
Hi all,
do you know if with this external is it possible to use multiple devices on a single computer?
if the usb driver is multiclient? which i wouldnt expect...
In theory it's possible with the SDK
https://dev.intelrealsense.com/docs/multiple-depth-cameras-configuration
It's just a matter now to check if the external is capable of handling multiple devices
So, did somebody try and succeeded in using multiple Intel Realsense cameras with a single computer?
If yes, how?
I believe it is possible to use multiple devices using the "jit.realsense" external programmed by BlueYeti, since in the code it states a device index to be used, which can also be changed through an attribute.
The problem is that it doesn't work, returns an invalid pointer error in the Max console.
Thank you for any help!
What I've discovered so far:
- in jit.realsense.cpp there is a variable called "device", which can be assigned to an integer (the device index),
by default is set to "0". This variable can be modified by the attribute "rs_index" inside Max.
- When modifying the attribute from Max I get the error "invalid pointer", which I think could originate from this function:
// Try to get the new device
auto devs = ctx.query_devices();
auto n_dev = devs.size();
post("There are %d connected RealSense devices.\n", n_dev);
if(n_dev <= device)
{
post("Device %d is not connected.", device);
return;
}
dev = devs[device];
........
try
{
// Get and check the data.
if(!x || !x->dev)
{
error("No device");
return JIT_ERR_INVALID_PTR;
}
So, apparently ctx.query_devices(); doesn't return any valid device.
Looking more attentively I can see another error thrown out in the Max console:
realsense: info Firmware Version not supported by the device!
Which originates from:
catch(const std::exception & e)
{
error("realsense: %s\n", e.what());
if(dev)
dev = nullptr;
}
right under the "rebuild" function.
But this is very strange, since the device are exactly the same.
I'll keep posting updates here which could possibly be helpful to someone in the future.
It seems I'm able to change device by using the "rs_device" attribute, and it seems to switch between them.
But, it will always use the stream of only one of the cameras, so even if it says "using device 1" it actually uses device 0.
You can see they are actually different devices from the serial number.
Using device 1, an Intel RealSense D435
Serial number: 923322072590
Firmware version: 05.11.01.100
There are 3 connected RealSense devices.
Using device 0, an Intel RealSense D435
Serial number: 919122071639
Firmware version: 05.11.01.100
Hi! I know this thread is quite old now but I am getting crazy trying to compile the jit.realsense object for macOS.
Did anyone succeeded in this? Please let me know, there is couple of things that sadly I do not understand :-(
I've started to take a look at this project and will be evaluating if I can commit some time to it. For now I've got the mac version of the external built and functioning as expected (as far as I can tell with some very basic tests).
Please give it a try:
https://drive.google.com/file/d/1ZY1-XqI2a6QwyNfmmB_gimsovvR2Js_4
Hi Rob, thanks a lot! Wonderful, really really thanks!
rob - I have a real sense 430 and made my own object to get the depth image but would prefer to use this if it works out. so far it seems to be working really well on the Mac. let me know if you need any testers in the future
Hi Rob, nice job. I just tested it with a D435 and it works as expected.
I also tested it with the Lidar L515 and while the docs say it uses the same SDK, the external doesn't like it:
realsense: Couldn't resolve requests.
but it didn't crash. nice.
Hi ROB,
I have installed on mac (mojave) Max 8 latest version, where should I put the jit.realsense.mxo file? I tried in the library and it doesn't recognize the object.
it makes me a strange thing: as soon as I create a new object in a new patch and write "jit.realsense" the MAX8 application unexpectedly closes
the clean thing would be a package:
create a package folder with an externals folder inside and drop the mxo in there:
~Documents/Max 8/Packages/yourPackage/externals/jit.realsense.mxo
restart max
now it should work. I assume once the object is production ready it will be distributed this way..
Hi Rob
I tried to build your updated repo but I failed with the cmake step:
maybites@maybitessMBP172 build % cmake /Users/maybites/Arbeiten/02_code/library/max/jit.realsense -DCMAKE_BUILD_TYPE=Release
-- The C compiler identification is AppleClang 11.0.3.11030032
-- The CXX compiler identification is AppleClang 11.0.3.11030032
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
CMake Warning (dev) at CMakeLists.txt:27 (set):
implicitly converting '' to 'STRING' type.
This warning is for project developers. Use -Wno-dev to suppress it.
CMake Warning (dev) at librealsense/CMake/lrs_options.cmake:12 (option):
Policy CMP0077 is not set: option() honors normal variables. Run "cmake
--help-policy CMP0077" for policy details. Use the cmake_policy command to
set the policy and suppress this warning.
For compatibility with older versions of CMake, option is clearing the
normal variable 'BUILD_UNIT_TESTS'.
Call Stack (most recent call first):
librealsense/CMakeLists.txt:5 (include)
This warning is for project developers. Use -Wno-dev to suppress it
CMake Warning (dev) at librealsense/CMake/lrs_options.cmake:14 (option):
Policy CMP0077 is not set: option() honors normal variables. Run "cmake
--help-policy CMP0077" for policy details. Use the cmake_policy command to
set the policy and suppress this warning.
For compatibility with older versions of CMake, option is clearing the
normal variable 'BUILD_EXAMPLES'.
Call Stack (most recent call first):
librealsense/CMakeLists.txt:5 (include)
This warning is for project developers. Use -Wno-dev to suppress it.
-- Internet connection identified
-- Info: REALSENSE_VERSION_STRING=2.38.1
-- Performing Test COMPILER_SUPPORTS_CXX11
-- Performing Test COMPILER_SUPPORTS_CXX11 - Success
-- Performing Test COMPILER_SUPPORTS_CXX0X
-- Performing Test COMPILER_SUPPORTS_CXX0X - Success
-- Setting Unix configurations
-- using RS2_USE_LIBUVC_BACKEND
-- Found usb: /usr/local/lib/libusb-1.0.dylib
CMake Error at librealsense/src/gl/CMakeLists.txt:43 (include):
include could not find load file:
/Users/maybites/Arbeiten/02_code/library/max/jit.realsense/CMake/opengl_config.cmake
CMake Error at librealsense/examples/CMakeLists.txt:14 (include):
include could not find load file:
/Users/maybites/Arbeiten/02_code/library/max/jit.realsense/CMake/opengl_config.cmake
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE
-- Could NOT find apriltag (missing: APRILTAG_INC APRILTAG_LIB)
-- Unable to find apriltag library, skipping pose-apriltag example
CMake Error at librealsense/tools/CMakeLists.txt:28 (include):
include could not find load file:
/Users/maybites/Arbeiten/02_code/library/max/jit.realsense/CMake/opengl_config.cmake
-- Building with TM2
-- Fetching recommended firmwares:
-- D4XX_FW_VERSION: 5.12.7.100
-- SR3XX_FW_VERSION: 3.26.1.0
-- T26X_FW_VERSION: 0.2.0.951
-- L5XX_FW_VERSION: 1.5.0.0
-- http://realsense-hw-public.s3.amazonaws.com/Releases/RS4xx/FW/D4XX_FW_Image-5.12.7.100.bin
-- Download firmware 0;"No error" for D4XX_FW_Image-5.12.7.100.bin
-- http://realsense-hw-public.s3.amazonaws.com/Releases/SR300/FW/SR3XX_FW_Image-3.26.1.0.bin
-- Download firmware 0;"No error" for SR3XX_FW_Image-3.26.1.0.bin
-- http://realsense-hw-public.s3.amazonaws.com/Releases/TM2/FW/target/0.2.0.951/target-0.2.0.951.mvcmd
-- Download firmware 0;"No error" for target-0.2.0.951.mvcmd
-- http://realsense-hw-public.s3.amazonaws.com/Releases/L5xx/FW/L5XX_FW_Image-1.5.0.0.bin
-- Download firmware 0;"No error" for L5XX_FW_Image-1.5.0.0.bin
-- Configuring incomplete, errors occurred!I wanted to look into the possibility to output the pointcloud as a matrix to be used with [jit.gl.mesh]. or maybe even one step further: the possibility of [jit.gl.realsense], where the object renders the mesh directly like a 3d model.
the readme in the repo is incorrect for building Mac. I won't be providing any support on this anytime soon, but it should be pretty straight forward to get going. steps are something like:
git clone, including all submodules
cd /path/to/jit.realsense
mkdir build && cd build
cmake -G Xcode ..open the generated xcode project inside the build folder and build
you may need to run the cmake step twice (i did, errors the first time)
one thing i noticed, for some reason the xcode project Debug configuration has optimizations enabled, so if you want to debug you'll have to manually disable the optimizations setting (Optimization Level setting) or figure out why cmake is generating with those enabled.
Thanks for the hints. I indeed had to redo the first cmake command.
the readme in the repo is incorrect for building Mac.
I was able to build the external with the original readme command. both ways seem to work.
I have forked the repo and will update the readme with your suggestions plus a fix to the 'CMakeLists.txt'
-> https://github.com/tecartlab/jit.realsense
TNX @MAYBITES
I tried to make the folder with externals in PACKAGEs but it doesn't work it quits unexpectedly as soon as i try to create an object with "jit.realsense". I reinstalled max8. Can someone help me?
I use Mojave - max8
Please, Can you post your external here: "jit.realsense.mxo"? I will copy it to my externals folder
thnx!
THNX MAYBITES
I copied the folder in Package/jit.realsense/externals but this happens. WHY IT?
What do you think about it?
have you restarted max?
Yes I restarted max.
Should I install Intel® RealSense™ SDK 2.0 (v2.39.0) first of all this?
I think thats not needed. the brown color indicates max is not finding the object-external.
restart Max, create a blank patch, add a jit.realsense object, go to the max console, select all, copy, and paste the contents here
I would like to do what you tell me but does not allow it closes immediately as soon as I create the object jit.realsense and press return.
Is it 32bit?
I use max8
tnhx
please, can you create another build version for MAC OS of jit.realsense?
you have a screenshot posted above. there is a jit.realsense object that is tinged orange. I just want to know what the Max console says in that screenshot. Help us to help you!
yes, that is the situation in OCT 08 2020 | 10:56 AM. but now MAX8 closes immediately as soon as I create the object jit.realsense and press return.
hi I build the external by cmake but this is
remove the quarantine attribute via xattr - https://cycling74.com/articles/using-unsigned-max-externals-on-mac-os-10-15-catalina
also your folder structure in the Packages folder should be Packages / realsense / externals / jit.realsense.mxo
i would just use the -d option and remove it on the entire folder.
unfortunately it doesn't work...,
I tried it but terminal display returns this:
xattr: /Users/admin/Documents/Max 8/Packages/externals: No such xattr: com.apple.quarantine
hi,
I had trouble too. This is what helped for me:
1. You need Catalina (OS X 10.15.7)
2. Use xattr as described above (Type in terminal: xattr -d -r
com.apple.quarantine then drag the folder containing
jit.realsense.mxo into the terminal window and hit enter)
3. install homebrew https://brew.sh/index_de
4. install libusb with brew install libusb
5. not sure if the installing the xCode command line tools is necessary too?
My understanding is, that libusb is required to use jit.realsense.mxo.
hmm, interesting point about libusb!
(i think xcode command line tools is necessary to install homebrew anyway)
thank youThank you rob! I have been debugging in the terminal tool for 1 day, but I can’t use sdk in maxos normally. Fortunately, I searched the forum and found jit.realsense here. It works very well and is perfect! grateful!
Hi all,
did someone succesfully use the external with 2 devices connected to the same computer? i 'm on Windows10 64bit and use the compiled version from blueyeti..(eventually it's not the newest version)
Federico raised this question too some time ago... i can switch rs_device but the output stays the same.
maybe someone can provide a already built version for Win10 64bit?
thanks
mat
Hi Mat, I can confirm you that I managed to get at least 3 devices working on the same computer.
The main point are the cables. Be sure to be using powered USB cables.
i will try to find some time in the near future to clean up the versions I have and post them somewhere useful.
ok. interesting. because in touch i can sitch easily both devices.
but lets see. i will try that.
so there's no point in built the external from rob's git?
thanks Federico for this supermegafast answer.
thanks rob! great!
@maybites I downloaded your version of jit.realsense. It's recognised by Max 8 just fine but clicking the toggle object to start the stream produces the following error in the Max console.
There are 1 connected RealSense devices.Using device 0, an Intel RealSense D415
Serial number: 821312061537
Firmware version: 05.13.00.50
realsense: Couldn't resolve requests
wait_for_frames cannot be called before start()
jit.realsense: error type unspecified
wait_for_frames cannot be called before start()
jit.realsense: error type unspecified
Is Anyone else experiencing the same behaviour? I have realsense sdk, max-sdk, and libusb installed. I'm running MacOS 11.7.1
hello,
i have also a problem getting the jit.realsense external to work.
1) downloaded the external here
https://github.com/tecartlab/jit.realsense/releases/tag/v5
2) moved the external to Packages / realsense / externals / jit.realsense.mxo
3)remove the quarantine attribute via xattr with
xattr -d -r com.apple.quarantine [path to external folder]
4) installed libusb with brew
5) i create a jit.realsense object in an empty max patcher.
then i get the following error:
jit.realsense: unable to load object bundle executable
2023-01-17 12:28:03.124 Max[3142:33962] Error loading /Users/pet/Documents/Max 8/Packages/realsense/externals/jit.realsense.mxo/Contents/MacOS/jit.realsense: dlopen(/Users/pet/Documents/Max 8/Packages/realsense/externals/jit.realsense.mxo/C
ontents/MacOS/jit.realsense, 0x0106): Library not loaded: '/usr/local/opt/libusb/lib/libusb-1.0.0.dylib'
Referenced from: '/Users/pet/Documents/Max 8/Packages/realsense/externals/jit.realsense.mxo/Contents/MacOS/jit.realsense'
Reason: tried: '/usr/local/opt/libusb/lib/libusb-1.0.0.dylib' (no such file), '/usr/local/lib/libusb-1.0.0.dylib' (no such file), '/usr/lib/libusb-1.0.0.dylib' (no such file)
it seems to be that brew have not installed the libusb at the location, where the external expecting it ...
checked the following paths manually:
/usr/local/opt/libusb/lib/
/usr/local/lib/
/usr/lib/
there is no file "libusb-1.0.0.dylib"
i am on
"version" : "Version 8.3.3 (6f7f6680766) (x64 mac rosetta)"
"osversion" : "Mac OS X Version 12.6.2 (Build 21G320) x86_64 Rosetta"
perhaps someone has a hint how i could manage my realsense D415 to work.
do have to install some driveres from intel first?
all the best,
petcode