Intel Realsense development


    Jan 02 2016 | 6:38 am
    Hi Dear community!
    First and foremost you guys rock, I've been using/learning from the community for a long time! Been working a lot with 3D cameras of various type in Max and recently got a hold of the newcomer Intel RealSense F200 and R200, unfortunately they are Windows based and I converted to Mac a while ago, still these two cameras forced me back to my PC and as it turned out at least the F200 is easy to use and comes with many end-user software to do most of the things that we've been doing with Kinect and Asus. OSCing the data from my PC to the Mac was my only working option so far and was wondering if anybody is working on a wrapper for Max, if so I will be gladly running tests. (hint: maybe Jean-Marc Pelletier or Dale Phurrough are thinking about another magical object by the name of jit.realsense?) Here is a single webpage with the drivers and SDK:
    Thank you all and happy new year!

    • Jan 04 2016 | 1:44 am
      I briefly considered writing an object for it. I did research into the number of people using it. So far, I haven't read about any significant number of people using it to justify the time/effort I would need to put into a quality release. If you know instead that there is a large number of people using it, please do share with me so I can re-evaluate my previous choice.
    • Jan 04 2016 | 2:01 am
      Thanks for you reply Dale, the number of people using the tech is definitely not even remotely close to Kinect users, that is expected though knowing that the F200 recently became available to purchase (a note to friends reading this, please pay attention to the specified range of this camera it's super short range, not usable as is for performances), having said that with the publicity behind it such as DevLabs all around here in the US, also the fact that they are making laptops with the built-in F200 and tablets with built-in R200 (the long range camera) it seems it might become the only choice for a portable better resolution USB powered 3D camera. In either case I'll be happy to help in anyway necessary if you decided to work on it.
    • Apr 14 2016 | 7:54 pm
      It's rather sad to see that Cycling '74 team does not take care of these things. I recently took a closer look at TouchDesigner and I saw that they have objects that come with the package for every device possible, this one included.
      Here in Max we need outside people like Dale to take from their own time to build these objects.
      That is not very good. Not good for the company. And definitely not good for the users.
    • Apr 15 2016 | 12:46 am
      YGREG +1
    • Apr 15 2016 | 3:11 am
      @YGREQ I think it's a resource issue. I suspect increasing the marketshare of Max would resolve a few of these things.
    • Apr 15 2016 | 6:59 am
      Actually I wanted to address that in another post as well.
      I actually find it very unwise for Cycling '74 to offer Max for only one month in demo mode. That is not enough time for a beginner to get their hands around Max in order to realise that buying it is worth the effort. I only became a Max user because of the fact I found a way to crack it. In the end I bought it. In case someone is suggesting it is cheap so one can afford it, 1 it is not cheap for many of us, 2 why buy it if you are not sure you are going to use it?
      TouchDesigner comes with a free version that you use non-commercially indefinitely. Once you realise you want to make money out of it or you realise you want to use the the power features (that you don't have in the free version, like using a bigger resolution) you buy the app. Which costs lots more that Max by the way. In the meantime the user becomes a power user and everyone is happy.
      The conclusion is that by adopting this strategy, Cycling '74 is loosing a lot of potential users from my point of view.
    • Apr 27 2016 | 8:00 am
      Good news! Here is an interesting update, Intel realsense seems to be finally available to us Mac users! ( I haven't tested it yet though!) here is a link to the library
      By the way, while (to an extent) I agree to what has been discussed still I think to rely too much on Cycling74 to provide and cover for every device out there is not realistic, I myself would prefer some tutorials on how to do such a task so we could do it for every device that we want to. I think that's the point of having a community after all.
      Cheers
    • Apr 27 2016 | 2:47 pm
      The link says: "This project is separate from the production software stack available in the Intel® RealSense™ SDK, namely that this library only encompasses camera capture functionality without additional computer vision algorithms."
      I think that means we get the camera stream inputs but don't get our hands on all the tasty tracking, gesture and scanning stuff sadly :(
    • Nov 17 2016 | 10:51 pm
      you can just rent the software if it is interesting you? its quite cheap / month
    • Dec 10 2017 | 10:22 am
      There is already an external made for it , it gives you acces to the depth image but thats it https://github.com/blueyeti/jit.realsense
    • Oct 03 2019 | 5:38 pm
      Hi all, do you know if with this external is it possible to use multiple devices on a single computer?
    • Oct 03 2019 | 9:26 pm
      if the usb driver is multiclient? which i wouldnt expect...
    • Oct 04 2019 | 7:42 pm
      In theory it's possible with the SDK https://dev.intelrealsense.com/docs/multiple-depth-cameras-configuration It's just a matter now to check if the external is capable of handling multiple devices
    • Oct 16 2019 | 10:12 am
      So, did somebody try and succeeded in using multiple Intel Realsense cameras with a single computer?
      If yes, how?
      I believe it is possible to use multiple devices using the "jit.realsense" external programmed by BlueYeti, since in the code it states a device index to be used, which can also be changed through an attribute.
      The problem is that it doesn't work, returns an invalid pointer error in the Max console.
      Thank you for any help!
    • Oct 16 2019 | 11:13 am
      What I've discovered so far:
      - in jit.realsense.cpp there is a variable called "device", which can be assigned to an integer (the device index), by default is set to "0". This variable can be modified by the attribute "rs_index" inside Max. - When modifying the attribute from Max I get the error "invalid pointer", which I think could originate from this function: // Try to get the new device auto devs = ctx.query_devices(); auto n_dev = devs.size(); post("There are %d connected RealSense devices.\n", n_dev);
      if(n_dev <= device) { post("Device %d is not connected.", device); return; }
      dev = devs[device]; ........ try { // Get and check the data. if(!x || !x->dev) { error("No device"); return JIT_ERR_INVALID_PTR; }
      So, apparently ctx.query_devices(); doesn't return any valid device. Looking more attentively I can see another error thrown out in the Max console: realsense: info Firmware Version not supported by the device!
      Which originates from: catch(const std::exception & e) { error("realsense: %s\n", e.what()); if(dev) dev = nullptr; }
      right under the "rebuild" function.
      But this is very strange, since the device are exactly the same. I'll keep posting updates here which could possibly be helpful to someone in the future.
    • Oct 16 2019 | 11:35 am
      It seems I'm able to change device by using the "rs_device" attribute, and it seems to switch between them. But, it will always use the stream of only one of the cameras, so even if it says "using device 1" it actually uses device 0. You can see they are actually different devices from the serial number.
      Using device 1, an Intel RealSense D435 Serial number: 923322072590 Firmware version: 05.11.01.100 There are 3 connected RealSense devices.
      Using device 0, an Intel RealSense D435 Serial number: 919122071639 Firmware version: 05.11.01.100
    • Sep 09 2020 | 2:41 pm
      Hi! I know this thread is quite old now but I am getting crazy trying to compile the jit.realsense object for macOS. Did anyone succeeded in this? Please let me know, there is couple of things that sadly I do not understand :-(
    • Sep 11 2020 | 3:08 am
      I've started to take a look at this project and will be evaluating if I can commit some time to it. For now I've got the mac version of the external built and functioning as expected (as far as I can tell with some very basic tests).
    • Sep 11 2020 | 1:54 pm
      Hi Rob, thanks a lot! Wonderful, really really thanks!
    • Sep 24 2020 | 8:06 pm
      rob - I have a real sense 430 and made my own object to get the depth image but would prefer to use this if it works out. so far it seems to be working really well on the Mac. let me know if you need any testers in the future
    • Oct 05 2020 | 2:28 pm
      Hi Rob, nice job. I just tested it with a D435 and it works as expected.
      I also tested it with the Lidar L515 and while the docs say it uses the same SDK, the external doesn't like it: realsense: Couldn't resolve requests.
      but it didn't crash. nice.
    • Oct 06 2020 | 9:06 am
      Hi ROB, I have installed on mac (mojave) Max 8 latest version, where should I put the jit.realsense.mxo file? I tried in the library and it doesn't recognize the object.
    • Oct 06 2020 | 9:11 am
      it makes me a strange thing: as soon as I create a new object in a new patch and write "jit.realsense" the MAX8 application unexpectedly closes
    • Oct 06 2020 | 9:21 am
      the clean thing would be a package:
      create a package folder with an externals folder inside and drop the mxo in there:
      ~Documents/Max 8/Packages/yourPackage/externals/jit.realsense.mxo restart max
      now it should work. I assume once the object is production ready it will be distributed this way..
    • Oct 06 2020 | 2:09 pm
      Hi Rob
      I tried to build your updated repo but I failed with the cmake step:
      maybites@maybitessMBP172 build % cmake /Users/maybites/Arbeiten/02_code/library/max/jit.realsense -DCMAKE_BUILD_TYPE=Release
      -- The C compiler identification is AppleClang 11.0.3.11030032
      -- The CXX compiler identification is AppleClang 11.0.3.11030032
      -- Detecting C compiler ABI info
      -- Detecting C compiler ABI info - done
      -- Check for working C compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc - skipped
      -- Detecting C compile features
      -- Detecting C compile features - done
      -- Detecting CXX compiler ABI info
      -- Detecting CXX compiler ABI info - done
      -- Check for working CXX compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ - skipped
      -- Detecting CXX compile features
      -- Detecting CXX compile features - done
      CMake Warning (dev) at CMakeLists.txt:27 (set):
        implicitly converting '' to 'STRING' type.
      This warning is for project developers.  Use -Wno-dev to suppress it.
      CMake Warning (dev) at librealsense/CMake/lrs_options.cmake:12 (option):
        Policy CMP0077 is not set: option() honors normal variables.  Run "cmake
        --help-policy CMP0077" for policy details.  Use the cmake_policy command to
        set the policy and suppress this warning.
        
        For compatibility with older versions of CMake, option is clearing the
        normal variable 'BUILD_UNIT_TESTS'.
      Call Stack (most recent call first):
        librealsense/CMakeLists.txt:5 (include)
      This warning is for project developers.  Use -Wno-dev to suppress it
      
      CMake Warning (dev) at librealsense/CMake/lrs_options.cmake:14 (option):
        Policy CMP0077 is not set: option() honors normal variables.  Run "cmake
        --help-policy CMP0077" for policy details.  Use the cmake_policy command to
        set the policy and suppress this warning.
        For compatibility with older versions of CMake, option is clearing the
        normal variable 'BUILD_EXAMPLES'.
      Call Stack (most recent call first):
        librealsense/CMakeLists.txt:5 (include)
      This warning is for project developers.  Use -Wno-dev to suppress it.
      -- Internet connection identified
      -- Info: REALSENSE_VERSION_STRING=2.38.1
      -- Performing Test COMPILER_SUPPORTS_CXX11
      -- Performing Test COMPILER_SUPPORTS_CXX11 - Success
      -- Performing Test COMPILER_SUPPORTS_CXX0X
      -- Performing Test COMPILER_SUPPORTS_CXX0X - Success
      -- Setting Unix configurations
      -- using RS2_USE_LIBUVC_BACKEND
      -- Found usb: /usr/local/lib/libusb-1.0.dylib  
      CMake Error at librealsense/src/gl/CMakeLists.txt:43 (include):
        include could not find load file:
        /Users/maybites/Arbeiten/02_code/library/max/jit.realsense/CMake/opengl_config.cmake
        CMake Error at librealsense/examples/CMakeLists.txt:14 (include):
        include could not find load file:
        /Users/maybites/Arbeiten/02_code/library/max/jit.realsense/CMake/opengl_config.cmake
      -- Looking for pthread.h
      -- Looking for pthread.h - found
      -- Performing Test CMAKE_HAVE_LIBC_PTHREAD
      -- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
      -- Found Threads: TRUE  
      -- Could NOT find apriltag (missing: APRILTAG_INC APRILTAG_LIB) 
      -- Unable to find apriltag library, skipping pose-apriltag example
      CMake Error at librealsense/tools/CMakeLists.txt:28 (include):
        include could not find load file:
        /Users/maybites/Arbeiten/02_code/library/max/jit.realsense/CMake/opengl_config.cmake
      -- Building with TM2
      -- Fetching recommended firmwares:
      -- D4XX_FW_VERSION: 5.12.7.100
      -- SR3XX_FW_VERSION: 3.26.1.0
      -- T26X_FW_VERSION: 0.2.0.951
      -- L5XX_FW_VERSION: 1.5.0.0
      -- http://realsense-hw-public.s3.amazonaws.com/Releases/RS4xx/FW/D4XX_FW_Image-5.12.7.100.bin
      -- Download firmware 0;"No error" for D4XX_FW_Image-5.12.7.100.bin
      -- http://realsense-hw-public.s3.amazonaws.com/Releases/SR300/FW/SR3XX_FW_Image-3.26.1.0.bin
      -- Download firmware 0;"No error" for SR3XX_FW_Image-3.26.1.0.bin
      -- http://realsense-hw-public.s3.amazonaws.com/Releases/TM2/FW/target/0.2.0.951/target-0.2.0.951.mvcmd
      -- Download firmware 0;"No error" for target-0.2.0.951.mvcmd
      -- http://realsense-hw-public.s3.amazonaws.com/Releases/L5xx/FW/L5XX_FW_Image-1.5.0.0.bin
      -- Download firmware 0;"No error" for L5XX_FW_Image-1.5.0.0.bin
      -- Configuring incomplete, errors occurred!
      I wanted to look into the possibility to output the pointcloud as a matrix to be used with [jit.gl.mesh]. or maybe even one step further: the possibility of [jit.gl.realsense], where the object renders the mesh directly like a 3d model.
    • Oct 06 2020 | 4:55 pm
      the readme in the repo is incorrect for building Mac. I won't be providing any support on this anytime soon, but it should be pretty straight forward to get going. steps are something like: git clone, including all submodules
      cd /path/to/jit.realsense
      mkdir build && cd build
      cmake -G Xcode ..
      open the generated xcode project inside the build folder and build you may need to run the cmake step twice (i did, errors the first time)
      one thing i noticed, for some reason the xcode project Debug configuration has optimizations enabled, so if you want to debug you'll have to manually disable the optimizations setting (Optimization Level setting) or figure out why cmake is generating with those enabled.
    • Oct 06 2020 | 5:53 pm
      Thanks for the hints. I indeed had to redo the first cmake command.
      the readme in the repo is incorrect for building Mac.
      I was able to build the external with the original readme command. both ways seem to work.
      I have forked the repo and will update the readme with your suggestions plus a fix to the 'CMakeLists.txt'
      -> https://github.com/tecartlab/jit.realsense
    • Oct 07 2020 | 9:05 am
      TNX @MAYBITES I tried to make the folder with externals in PACKAGEs but it doesn't work it quits unexpectedly as soon as i try to create an object with "jit.realsense". I reinstalled max8. Can someone help me?
      I use Mojave - max8
    • Oct 07 2020 | 10:02 am
      this setup works for me (in my case I made a shortcut to the folder - it should work either way)
      this setup works for me (in my case I made a shortcut to the folder - it should work either way)
    • Oct 07 2020 | 12:17 pm
      Please, Can you post your external here: "jit.realsense.mxo"? I will copy it to my externals folder thnx!
      this is my.
      this is my.
    • Oct 08 2020 | 8:12 am
    • Oct 08 2020 | 8:56 am
      THNX MAYBITES I copied the folder in Package/jit.realsense/externals but this happens. WHY IT? What do you think about it?
    • Oct 08 2020 | 9:32 am
      have you restarted max?
    • Oct 10 2020 | 6:49 am
      Yes I restarted max. Should I install Intel® RealSense™ SDK 2.0 (v2.39.0) first of all this?
    • Oct 10 2020 | 12:28 pm
      I think thats not needed. the brown color indicates max is not finding the object-external.
    • Oct 10 2020 | 3:46 pm
      restart Max, create a blank patch, add a jit.realsense object, go to the max console, select all, copy, and paste the contents here
    • Oct 12 2020 | 1:59 pm
      I would like to do what you tell me but does not allow it closes immediately as soon as I create the object jit.realsense and press return. Is it 32bit? I use max8 tnhx
    • Oct 15 2020 | 8:11 pm
      please, can you create another build version for MAC OS of jit.realsense?
    • Oct 15 2020 | 8:43 pm
      you have a screenshot posted above. there is a jit.realsense object that is tinged orange. I just want to know what the Max console says in that screenshot. Help us to help you!
    • Oct 17 2020 | 10:44 am
      yes, that is the situation in OCT 08 2020 | 10:56 AM. but now MAX8 closes immediately as soon as I create the object jit.realsense and press return.
    • Oct 23 2020 | 10:53 am
      hi I build the external by cmake but this is
    • Oct 23 2020 | 5:57 pm
      also your folder structure in the Packages folder should be Packages / realsense / externals / jit.realsense.mxo i would just use the -d option and remove it on the entire folder.
    • Oct 24 2020 | 7:53 am
      unfortunately it doesn't work..., I tried it but terminal display returns this:
      xattr: /Users/admin/Documents/Max 8/Packages/externals: No such xattr: com.apple.quarantine