Send audio via network to Hololens 2 (Unity)

    MaxMSP

    Robert Hart's icon
    Robert Hart's icon
    Robert Hart
    May 31 2022 | 9:48 am
    Hello everyone, I am currently developing an application where I need to send data between a Hololens 2 and Max. Basically I send control information via OSC from the Hololens (Unity application) to my patch.
    At the moment I'm looking for a good way to send Max's audio output to the HoloLens. Does anyone know a good solution for this? In summary, I need a way to send my audio output over a local network where only the sender can run Max. I would be happy if someone could help me there. Many Thanks.

    • Alice's icon
      Alice's icon
      Alice
      Jul 14 2022 | 2:19 am
      Hi Robert,
      How's your work going? I'm also trying to send audio via network using unity. A few toolkits I found useful: - https://github.com/Iam1337/extOSC - https://github.com/keijiro/OscJack
      Tho I found some latency issues from the sender side, may due to Holo's hardware affordance.
      Happy to discuss with you if you are keen! Cheers.
      Share
    • Robert Hart's icon
      Robert Hart's icon
      Robert Hart
      Aug 16 2022 | 9:02 am
      Hi Alice,
      Unfortunately I haven't been very successful so far. For the controls I do use extOSC. But I don't think this suits for audio transmission. I managed to set up a WebRTC process that uses a virtual cable to capture the system audio/max output and send it to the lens via WebRTC and a local server. Unfortunately the received audio seems to be highly optimised for voice transmitting as one can hear a lot of processing for it. As I need to handle music files this was not an option.
      I think I could have looked at the depth of the source code for the WebRTC implementation and eventually found a way to make it less processed, but for now I've replaced the process with rendering using a Unity Spatializer on the HoloLens itself.
      However I'm still trying to implement that transmission part at some point because Max gives so much more control. How is your progress?
    • Alice's icon
      Alice's icon
      Alice
      Nov 23 2022 | 11:54 pm
      Hey Robert, Sorry for my late reply - I put this network project aside for a while since last post - now I'm back - happy to discuss and share audio/dsp dev issues in holos. Just want to clarify - you want the audio generated from laptop Max send to holo in *real-time*? So that the audio can be played in holos directly ?
      I reckon there is a huge different between sending a real-time generated audio to holo for playing, versus sending a sample( music files which you mentioned). This also implies very different technical issues. Cheers, Alice