Florian Demmer

With the increased importance of social distancing, many people are facing the need to teach or present work using Max remotely. Many schools have moved their courses online, and streaming video has emerged as a popular way to connect with people socially across distances. This article will provide some practical tips on how to set up audio and video streaming from your computer in order to share it with others.

Modern video conferencing services and solutions usually apply intelligent filters and compression algorithms in order to optimise the stream for low latency and speech intelligibility, usually at the cost of audio quality. While the following won’t provide an audiophile streaming experience to your audience it does provide a solution where both the latency and quality are good enough to illustrate what is going on with your patch.

While modern browsers are equipped with technology (WebRTC) that allows peer-to-peer communication between users, it’s usually more difficult to set up and furthermore prone to issues with the networking layer if people are behind proxies or firewalls that can often be found in educational institutions. This guide therefore opts for a more classic streaming approach in order to increase accessibility and ease of use.

 Check out the other streaming article - Streaming Tips - The Jitter Edition 

The key idea here is to use local software in conjunction with a streaming platform for creating a video stream that can be accessed by others by means of visiting a url in their browser. Additional to providing a video stream most services include features for interacting with your audience like a live chat or others. In order to achieve this you will need to install the following:

OBS - Open Broadcaster Software as local streaming client

A virtual or physical audio input, which can be used to route the audio output from Max into OBS

examples for virtual AudioDrivers on OSX would be BlackHole or Loopback

An account with a streaming platform of your choice, two popular options are Twitch and YouTube

In this guide we’ll be using BlackHole as a virtual audio driver, OBS and Twitch as a video streaming service. Twitch offers us the possibility of 160Kpbs stereo audio, as opposed to video conferencing services that usually work with mono and even further compressed audio streams.

If you do not already have an account please visit https://www.twitch.tv and create a new user account to use for your stream.

We’ll start by setting up the audio in Max in order to be able to use its output as an input within OBS. Assuming that BlackHole has been successfully installed simply open the Audio Status Window (Options => Audio Status) and select BlackHole as the Output Device. Also make sure that the Sampling Rate is set to 44100.


Adjusting the OBS settings for streaming to Twitch

After successfully installing and opening OBS we need to adjust the settings for streaming to Twitch. In case you are greeted by the Configuration Wizard on your first launch of OBS you can simply skip that for now. To adjust the settings for our Twitch stream open the Preferences (OBS => Preferences) and navigate to the Stream section. Within that section select Twitch as the service, leave Auto as the Server.

Next click on the Link next to Stream Key , which will open your web browser and navigate to the Settings Page of your Twitch Account. Copy your Stream Key from here and paste it into OBS.

Next up we’ll adjust the parameters for our stream within OBS. For this guide we’ll be streaming using a resolution of 1080p and 160kbps audio. However, this might be demanding for your machine and network so please refer to the encoding recommendations provided by Twitch https://stream.twitch.tv/encoding/ in order to adjust the parameters accordingly.

Navigate to the Output section of the preference screen and make sure that Simple is selected as the Output Mode. We’ll use a Video Bitrate of 6000 kbps according to the recommendations. Leave the Encoder as Software (x264) and set the Audio Bitrate to 160. Also make sure to enable the Advanced Encoder Settings and Enforce streaming service bitrate limits and select veryfast as the Encoder Preset.

Next we’ll head to the Audio section to ensure that we are using a Sample Rate of 44.1 kHz to match our setting in Max and that we are streaming in stereo. Underneath this we can set up our Audio Devices. In order to route the output from Max into OBS we select BlackHole for Mic / Auxiliary Audio. If you’d like to additionally use a microphone input you can select this as the second device.

We’ll now head to the video settings and set our resolution. In order to stream with 1080p we set the Base and Output resolution to 1920x1080 and the Common FPS Value to 60.

With that, we have prepared OBS for our stream and can close the Preferences Window.

At the bottom of OBS’ main window you can adjust the sources of video input that should be included in our stream. OBS is a powerful tool that allows you to combine various sources of video input into a single stream. For the sake of this guide we’ll stick to simply sharing the entire screen of the streaming computer, however feel free to adjust this according to the needs of your stream. 

If there are already sources added to your stream simply remove them by selecting and clicking the Minus Button. To add a capture of your screen click on Plus Button in the Sources section and select Display Capture. Click OK in the opening dialogue and next up you will see a window that allows you to adjust the properties of the Display Capture. This would for example allow you to select the display to use in case you have multiple displays connected, hide the cursor or perform any cropping. You can simply click on Defaults in the bottom left corner and then click on OK. You should now see your display getting shown within OBS and can simply use the red border to resize the captured display to fit into the stream.

To start our stream, click on Start Streaming in the bottom right corner of the OBS window. Once the stream is successfully connected and live you should see a bright green square on the bottom right corner. 

You should now be able to access the Creator Dashboard on Twitch to verify your stream is up and running by visiting https://dashboard.twitch.tv/ and running with a good quality. From here you will also be able to interact with your viewers using the Live Chat. During your actual live stream it might be a good idea to turn off the local preview of your stream in order to save additional bandwidth.


In order to be able to monitor the audio you will be streaming we need to ensure that OBS' monitoring is routed and setup correctly. First off open the Preferences again and switch to the Audio pane. In the Advanced section you can use the dropdown to select the Monitoring Device of your choice.

Next up we will need to enable the Monitoring for our stream. For that  open OBS' Advanced Audio Properties via the Edit Menu and set the option for Audio Monitoring to Monitor and Output on the Mic/Aux device.

A fundamental limitation you will potentially encounter is the speed of your upstream connection. Most people have faster download than upload speeds. For example, an example cable modem speed test might say you have 130Mbps download and 12Mbps upload. Therefore it is recommended to test the speed of your connection before your are asking others to join your stream. You will end up having to adjust your video output settings accordingly. As a general help please refer to resources like Twitch’s Encoding Guidelines.

We chose to illustrate how to stream your Max patch using Twitch for this guide given the simplicity and accessibility of the service. However, if you have certain requirements like unlisted or private streaming you might want to consider using a different platform.

You can use OBS to route multiple audio sources into your stream but can of course also use an adc~ within your Max patch and combine it with the audio produced by your patch if that’s easier for your setup .
We hope this guide helps you get started using Max in a live streaming environment, and encourage you to leave a comment if you run into trouble or have any other tips you can offer.

max-livestream.png

Many are facing the need to teach or present work using Max remotely, this article shows how to set up streaming from your computer.

tips-for-streaming-your-max-patch

Tips for Streaming Your Max Patch

StvDee

Max was used to capture live camera video from 6 remote computers, and use a TCP/IP protocol to stream the information. Also, two mobile phones were used to stream live video directly into Max (using this approach).

Thus, the main application (also in Max) extracted the video frames from all 8 sources, and with a mixing utility the user could compose the final composition. Multiscreen and projection mapping was also implemented into Max to transform the venue into an immersive space.

this-citys-centre

This City's Centre

Garrett

The work uses a combination of Max, MSP, Jitter and Java.

netscapes

Netscapes

Julien Bayle

I designed UI,  ans user actions controls.

I used the great Olaf Matthes' oggcast external.

the-streamer

The Streamer

tomoya matsuura

External objects for Cycling'74 Max to send MSP signal over network using rtp protocol. A modern alternative to legacy netsend~ & netreceive~ objects.

Transmit mc audio signal over IP network using rtp protocol. An internal sample format is an uncompressed PCM, not floating-point but a 16bit integer.
The rtp packets are sent on UDP protocol. Users can choose the way of initializing connection from either raw RTP (No handshake) or RTSP (handshake with TCP).

Currently, a number of channels is fixed by an attribute "channels", an auto-adaptation depending on input channels is not available due to a limitation of min-api.
Also, sending signals over NAT (= Global Network) is not supported (consider using VPN).

The object internally uses ffmpeg C API through C++ code, thus you can also receive audio signal by using ffmpeg or ffplay.


https://github.com/tomoyanonymous/rtpsendreceive 

External objects for Cycling'74 Max to send MSP signal over network using rtp protocol. A modern alternative to legacy netsend~ & netreceive

mc-rtpsendreceive~

mc.rtpsendreceive~

Rob Ramirez

In this article we’re going to continue exploring some options for streaming content from your Max patch to the internet. This time we’ll look at how to use Jitter texture output as a streaming input source, and go over some techniques for creating and sending to a virtual camera device for use in webcam apps.

 Check out the other streaming article - Tips for Streaming 

On both platforms we will again let OBS manage the streaming I/O. OBS is a fantastic open source project with tons of functionality either built-in or available via plugins. Additionally, we’re going to use Syphon/Spout texture sharing to send our output to the streaming app. OBS on Mac supports Syphon sources natively, and on Windows there is a plugin to support Spout sources.

Windows - install the Spout package and this OBS Spout plugin

Note - On Windows machines with dual graphics cards, care must be taken to ensure all apps are set to use the high performance GPU rather than the integrated GPU.

Previously we looked at how to stream to a single destination (Twitch) via OBS. I was recently clued in by C74 homie Chris Martin on how to turn that into multiple destinations via a service called Restream, which the Coaxial crew uses for their Experimental Half Hour show.

Syphon/Spout usage is detailed in the Best Practices for recording article and we’ll use many of the same concepts in order to achieve streaming Jitter content. In both situations we must decide which patch elements to send down the pipes, and how they should be composited, and then capture those elements to a texture that feeds the server object (a jit.gl.syphonserver on Mac, or a jit.gl.spoutsender on Windows).

In my recent lockdown related streaming experiments I implemented a very basic background separation and monochrome effect that I streamed out of OBS to Restream which went live on Facebook and YouTube simultaneously. Let’s look at the steps necessary to make that happen. 

A quick note before jumping in, HD video streaming requires a beefy machine with a beefy gpu and internet connection. Mileage will vary wildly depending on these specs.

The recording article mentioned above starts by asking “What to Record”, and we must again decide on the what before moving to the how. In the provided patch I’ve mashed a few different sources and effects together just to keep things spicy. The core of the patch is a simple foreground video echo effect. 

We start with a jit.grab camera source (output_texture enabled of course) run through a background subtraction algorithm and then piped through to some Vizzie effects. We also implement a mesh-shatter effect to demonstrate how geometry can be captured and composited with texture streams via jit.gl.node @capture 1. Throw in a glitchy little ISF transition and some parameter animation via jit.mo and an ATTRACTR module to polish off.  To run the patch make sure you've installed the ISF Max package and ISF library files, or substitute the transition object with a Vizzie XFADR.

Now that we’ve got a sparky live video patch to play with, let’s send it off. The basic rule of thumb with Jitter texture streaming is any object that outputs a texture can be patched to a syphon/spout server object. If you want to send off geometry, capture it to a texture with jit.gl.node, or if you want to just send the entire contents of your jit.world window, enable output_texture and connect the server object to its leftmost outlet.

Once your patch has passed off its texture through the sender server, any client software running on your machine can grab the stream. Max is pretty much out of the picture from this point. OBS streams out to Restream and Restream handles sending to any other desired platforms. After logging in to Restream client services can be added to the streaming session in the Destinations section.

For OBS setup follow the instructions outlined in the previous streaming article. Copy the RTMP URL and stream key from Restream and enter in the OBS Settings window Stream section. Adjust any stream and video settings (e.g. bitrate, dimensions) as desired for your setup. Add a video source to your OBS scene by clicking the plus menu under the Sources section and select Syphon Client on Mac or Spout2 Capture on Windows and setting Max as the source (making sure you’ve installed the plugin linked above if on Windows). If more than one sender server is running on your machine, open the source properties by clicking the gear icon to select the appropriate server.

Note that it is possible to stream from Max via an OBS Window Capture source instead of using Syphon or Spout, but in my testing the frame rate dropped significantly. It also requires the Jitter window dimensions to match the streaming dimensions, so a less viable solution in most cases.

One last note, OBS makes it dead simple to make a local disk based recording of your stream, which generally is a good idea. By default it will match the recording settings to the stream settings, and therefore run more efficiently.

Ok so we’ve achieved our goal of streaming Jitter content through some different streaming services, but we live in the age of the Zoom meeting, how does one liven up their daily teleconferencing? For this we are going to need to create a virtual camera device to pipe Jitter through. Fortunately there are free programs on each OS that manage this device and receive Syphon and Spout textures streams.

On Windows install Spout Cam via the Spout installer

The setup is exactly the same as above except instead of sending to OBS, we will send to our virtual camera device. On Mac with Camtwist make sure you’ve set the video size and framerate in the preferences window before enabling your input. This should match the dims and framerate of your texture output.


Once you've updated your preferences, select Syphon as the source in the section titled Step 1, add effects if desired in Step 2, and then select your Max server as the source in Step 3. That’s it, you can now select Camtwist as your video device in your teleconferencing app (e.g. Zoom, Skype, Google meetups, etc).

On Windows we're going to run the Spout installer (note this is different from the Spout Max package), which will install a utility called Spout Cam. After installing, you will be able to select Spout Cam as the camera in Zoom. Spout Cam should match the dimensions of your texture input by default. As mentioned above, you must make sure all apps are using the high performance GPU for this to work. 

Not all webcam apps are able to detect and use Spout Cam directly, e.g. Skype did not in my testing. For these cases we can install the SplitCam app to act as a shim. SplitCam creates its own virtual camera device, and is also able to receive textures from Spout Cam. So with SplitCam running and its input set to Spout Cam, we are able to send directly to Skype.

In this final section let’s go even further down the rabbit hole and investigate how to improve the background subtraction. The patch above does a decent job of this, but it requires controlled conditions in order to be effective. Any changes in lighting and shadows, or any small movements of the camera cause the effect to break down. Modern day camera apps typically use smarter algorithms, likely built on machine learning and extensive datasets to handle the segmentation. To up our game we can install the popular (and free) Snap Camera app, and search for a chromakey lens.

Before we dive deep into this let me say, in my experimenting I encountered different problems on both platforms. The following is presented primarily as an exercise in problem solving, rather than a practical solution. If that's not interesting to you feel free to skip this section.

On Mac the OBS NDI plugin and NDI Runtime 

First, launch Snap Camera and enable the chromakey lens of your choosing. On Windows, jit.grab can directly receive the output from Snap Camera, but I was unable to get it working directly on Mac (not sure why). To workaround we can again make use of OBS, but this time instead of piping Jitter to OBS, we pipe OBS to Jitter using something called NDI. Install the NDI plugin and runtime linked above, then in OBS create a new Video Capture Device and set the device to Snap Camera. Click on the Tools menu and select NDI Output Settings and click the checkbox next to Main Output. Next launch the NDISyphon app and in the NDI Clients section you should see your OBS source. Click on that and enable it. Finally in your max patch you can receive the chromakeyed image with a jit.gl.syphonclient object.

If you want to take this monstrosity one step further, we can actually use OBS to both send from Snap to Jitter via NDI, and from Jitter to a streaming service. The trick is to enable Studio Mode by clicking that button in the lower right section of the interface. Then in the NDI Output Settings select the Preview Output instead of the Main Output. Finally you will set the Snap source as the preview and the Syphon source as the program.

Despite the ridiculousness of this setup, it does actually work and as proof I live-streamed out a little test clip using my patch above and replacing the background subtraction sub-patcher with the Snap Camera chromakey lens output. Check it out here.

You might be wondering how all this streaming stuff works out in practice. For that I'm going to turn it over to our good buddy Sam Tarakajian, who recently participated in an "online micro festival" called More Kicks Than Friends (Sam's bit starts around minute 38) presented by Max for Live master Ned Rush.

I wanted to perform something with my friend Alex, but of course the challenge was that we couldn't jam together in person. Our solution might sound a bit convoluted, but I was very pleasantly surprised by how well it turned out. First, Alex streamed his musical improvisation to me over Twitch. I kept that stream tab open in my browser, and fed the sound into Max using Loopback. If you don't want to buy Loopback, you could just as easily use Soundflower or BlackHole.

Once I had the sound in Max, I did some audio analysis to drive a couple of envelope followers, adding some audio reactivity to my Jitter patch. Then, so Alex could get a sense of what I was doing (albeit with some latency), I used exactly the technique that Rob describes: Max to Syphon to OBS to Twitch. Since Ned was asking for prerecorded video, I also used Syphon Recorder to record video locally (Rob describes using OBS for this, which would have been pretty much the same result). After we finished, we edited Alex's local audio together with my local video, and sent the result off to Ned. Luckily Syphon Recorder can record audio as well, which helped Alex edit the video properly.

On my machine I was simultaneously generating live visuals with Jitter, performing GPU accelerated face tracking, streaming video from Twitch, streaming video to Twitch, and recording video to disk. I'm amazed it was able to work at all, let alone as well as it did. But far from being a huge technical headache, all of the streaming tools we used were surprisingly easy to use. I'm looking forward to trying more streamed performance in the future.

restream-cover.jpg

Tips and techniques for sending Jitter video content to streaming platforms and teleconferencing apps. 

Search Results