Cycling '74

Tom Hall

Get an introduction to programming in the Max environment in this series of eight tutorial videos:

Episode 1: An Introduction to Programming in the Max Environment (see below)

Episode 2: Creating Some Basic Audio Effects

Episode 5: Creating a Velocity Sequencer MIDI Effect

Episode 6: Creating a Wobble Bass MIDI Instrument

Episode 8: Creating a Live API-based Step Sequencer

#1 Max For Live Decoding

programming-in-max-for-live-video-series-1

Programming in Max for Live: An Introduction to Programming in the Max Environment, Episode 1

Gregory Taylor

As a Max programmer, I spend quite a lot of time making patches that some people might find a little odd; rather than a large "instrument" that I toil over at great length or "the patch is the piece" outings, I love to make Max patches that don't make any noise or play any movies or create OpenGL scenes. Instead, I make things that are generative - working from the idea that Max is really just about messages, numbers, and lists and how you move them about; I love creating ways of generating and organizing variety, and then exploring what those generative structures do when hooked up to audio or video or other outputs.


LFO Tutorial 1: The Zen of the Silent Patch

Just as there are always several ways to do any given thing in Max, so there are several different ways to end the sentence, "One of the cool new things in Max 5 is...."

One of those things that piqued my interest is the ability to specify time in Max in ways that make so-called "musical" sense; in addition to creating metronomes that send out a bang every 100 milliseconds, you can now work with lots of new and interesting ways to specify time: ISO 8601-format hh:mm:ss.ms-style time, samples, "ticks" and note values. There's a long list of objects you can work with (you can find that list here,  both older Max objects as well as some new objects that help you take advantage of them (timepoint, when, and combine).  While lots of people are interested in using the new Max 5 transport object to arrange and trigger events in time, I'm personally a lot more interested in the ability to synchronize processes in Max using time values that resemble musical note values to create control structures that can be easily time synced.

This may be because I have a deep and abiding love for some of the interesting and esoteric control voltage uses of the LFO (Low Frequency Oscillator) in traditional analog synthesis. I'm thinking of analog modules like this - a cool set of LFOs that can be combined to achieve interesting effects (Anybody want to sell me a beat-up old AKS Sythi on the cheap?  No? And you wonder why I patch...). The great thing about Max is that you can produce the Max equivalent of these great general-purpose modules, and use them in any way you want. In fact, I'll often create patches like this and then spend the next few weeks hooking them up to all the patches I've got sitting around to see what they can do.

This tutorial is about making one of those kinds of modules - a quartet of synchronized LFOs whose outputs I can sample individually for several kinds of data (triggers for waveform start, LFO outputs that I can sample at variably synchronized rates, and a nifty summed waveform I can use for more exotic kinds of control).

Note: The downloadable materials that accompany this tutorial contain not only the large patch I'm going to talk about making, but a number of the little steps along the way - feel free to take the pieces and tear them apart and make things that are interesting to you, in addition to following along.

Here's what I usually use to create LFOs in Max.

You might have been expecting to see a  cycle~ object here instead of the  phasor~ object. While you could certainly use cycle~ objects for your waveform output, I prefer to use the phasor~ - a Max object that outputs a ramp that cycles between the floating point values 0. and 1.0 at a rate you specify.

The  cos~ object will take the output of a phasor~ object and produce a nice sinusoidal waveform with the standard -1.0 to 1.0 output range. The nice thing about using a phasor~ is that I can also easily produce a sawtooth LFO output by adding a pair of objects that change the range of the phasor~ object's output. (For extra credit, you might think about how you could generate a triangle wave using a phasor~ as a source. Yes, I know - you could just slap a triangle~ object below the phasor~ in the patch, but I'm encouraging you to think about signal math. If that seems too easy for you, how would you produce a square wave?

Now that we've got two waveforms, what kinds of useful information can we extract for our use? Since the waveform's output is happening at audio rate, we'll need to sample its output. Personally, I prefer to use the snapshot~ object, which samples a waveform output at whatever millisecond interval I provide as an argument. I'll probably need to do some linear mapping of the resulting numerical output into a more useful range of numbers, but I'll get to that in a minute. I'd also be interested in sending a bang message every time my waveform began its cycle. The example above shows how you can do this using two simple Max objects – a >=~ (greater than/equal to comparison signal object) followed by the handy  edge~ object that detects zero to non-zero signal transitions (and vice-versa, too). Beginning Max users might wish to meditate on this pair of objects in the course of their quest for enlightenment.

But there's another good reason to use the phasor~ object to create LFOs in Max, as shown in the left-hand part of the next example.

The  rate~ object takes a phasor~ as an input source, and produces the same kind of output (0. - 1.0) time-scaled by the number specified as an argument or sent to the object's right inlet. In the previous Max 4.x world, this was the only reliable way to synchronize any two LFOs. While there's a new way to do that now (shown on the right-hand side of the patch below), the rate~ object will still be really useful, as you'll soon see.

As I mentioned earlier, Max 5 gives you new ways to specify time in Max - some of which use terminology that is more explicitly "musical”; besides the old school metronomes that send out a bang every 100 milliseconds, you can now work with different “kinds” of time: ISO 8601-format hh:mm:ss.ms-style time, samples, and the time format we'll be using in this tutorial - note values. Since I'm interested in the ability to synchronize processes in Max using time values that resemble musical note values, I’ll use note values – whole notes, half notes, etc. – to set the rate of my LFO. Here's a simple example:

This patch contains the familiar phasor~ object, but instead of using an argument to specify a frequency for the ramp, I've used different arguments to each phasor~ object (1n and (2n), and I've also added an attribute called (lock), set to a value of 1 (on). The arguments 1n) and 2n) refer to note values - in this case, a whole note and a half note (you can find a complete listing of these note numbers  here).

Each phasor~ object also has a  umenu object connected to it containing a listing of all the possible note values. By selecting a note value from the menu, I can reset the phasor~ on the fly.

But what are the phasor~ objects locked to?

In Max, a central timekeeping object called  transport is used to handle metrical timing. The transport object not only keeps track of time, but broadcasts information to any Max objects in your patch that are "listening" or expecting timing information. When we use a note value argument with the phasor~ object, we're telling the object that it will need to listen for timing instructions.

While you can add transport objects to your patch (and even have different transport objects running at different rates controlling different sets of Max objects in a single patch), Max 5 includes a handy Global Transport - a default source of timing information that is ready any time you launch Max. You can find the global transport by choosing GlobalTransport from the Max Extras menu

The Global Transport has an on/off switch (the button to the right of the word Activate", a flashing LED indicator, displays bar/beat/unit timing and temp, and a display that shows the current "position" of the transport since you've started it.

NOTE: Since these tutorial patches are time-critical, you should have Overdrive turned on when you run them. If Overdrive is off, you may find that playback gets jerky when the density of notes is high or you perform other tasks on your computer (such as opening another patch).

To see the musically time-synced phasor~ objects in action:

 Click on the Global Transport's activate button.

Once the Global Transport is running, you'll see a pair of synchronized waveforms, and you can use the umenu objects to change their relative rates on the fly.

Now that we've got a great way to product perfectly synchronized LFO waveforms, let's go back to our earlier LFO examples and add some additional functionality. Of course, since our locked phasor~ objects are producing beautifully choreographed ramp outputs, they'll also be outputting beautiful sine waves when we use the cos~ object. And while we don't really need the rate~ object to synchronize the phasor~ objects' outputs any more, it's still useful. Suppose we want to produce a nicely synchronized waveform that repeats at a period that's greater than a dotted whole note (1nd, the largest time increment we can specify using note values)? Easy - we can use our old friend the rate~ object to scale the output of the locked phasor~ objects to longer time periods.

And while we're at it, it would also be nice to be able to invert the waveform phase outputs, too. That's an easy one - just multiply the signal by a value of -1.0. The following example patch adds all of these features, and a set of switches that let us configure our output:

It's hard to stare at any number of simultaneously synchronized LFOs and not feel the urge to sum those waveforms to produce lovely new shapes, if only for the pleasure of watching them. It's easy enough to do - you just scale the output of each waveform by the reciprocal of the number of waveforms you want to sum (1/2 or .5, 1/3 or .33, etc.) and then add them together.

And for one final touch, I think I'd like to be able to sample the LFO waveform output at a rate which is independent of the waveform itself, but still synced to the "global" rate of the Global transport. In this example patch, I'm sampling the waveform output with a snapshot~ object, and using the scale object to map the waveform's output data range from -1.0 - 1.0 to a more MIDI-friendly range. That data stream is then sent to the right inlet of an int object, whose current value will be output any time a bang is sent to the left inlet of the int object. The source of that bang comes from another Max object that can take advantage of the new timing values - the metro object. I've used a note value argument to set the rate at which the metro object sends bangs rather than the more common number argument (which specifies milliseconds), so the metronome is now also outputting its bangs at a rate derived from the Global Transport.

All of these little component level solutions are used in the giant example patch called LFOur.maxpat - which, as its title implies, contains FOur different and independent LFOs. Although the patch is pretty large, it contains all the little techniques you've seen so far.

You should feel free to open up the LFOur patch and poke around “under the hood.”

Max 5 makes it really easy to create user interfaces using the  Presentation Mode.  You can find a good tutorial on it  here, and Andrew Benson has written  a great tutorial on creating interfaces, as well. Working in Max 5, I now find that using the Presentation Layer to prototype interface designs accounts for a lot of my patching time these days; the patch that you see when you launch the LFOur patch is the presentation layer arrangement I decided on. Since I've set this patch to open up in Presentation mode and show you the user interface, you’ll need to do two things:

 Click on the Lock icon in the patcher toolbar to unlock the patch;

 Click on the Presentation Mode icon to display the LFOur patch in patching mode.

(Note: You’ll notice that when you click on the Presentation Mode icon, all the UI objects gracefully glide to their original position in the larger patch. If you’re curious about how to do this in your own patchers, choose Patcher Inspector from the View menu and double click in the Setting column for the Box Animate Time setting. Typing in a large number will create the glide as you toggle between patching and presentation modes).

While you’ll probably be able to recognize parts of the larger patch from the examples above, there are some useful things I’ve added to the patch that may be less obvious:

Each user interface item in the LFOur display in presentation mode includes a hint that describes what the object displays or does, and gives you hints about how to use it. Those hints were added by entering text into the Hint area in each object’s Inspector.

I also thought it’d be useful to be able to create preset combinations of waveforms and rates, so I added support for the pattr family of objects. Although the pattr family of objects are deep and subtle, it was actually quite simple to add this to the LFOur patch in just four steps:

I opened the Inspector for each and every UI object I wanted to save as part of a preset, and entered a unique name for the Inspector’s Scripting Name setting.

 I added a  pattrstorage object to my patch and gave it a name (in this case, LFOs).

 I added an  autopattr object to my patch (which has the effect of adding all the objects with scripting names to the pattr registry automatically).

 To make storing and recalling presets super easy, I added a  preset object to the patch (and added it to the Presentation layer, of course), opened its Inspector, and entered the name of the pattrstorage object (LFOs) to the value column for the pattrstorage setting. Once I did this, I could use the preset object to load and store presets for my patch.

Of course, this patch doesn't make any noise at all, although it is interesting to play with and restfully instructive to watch. LFOur is useful because of a key idea in Max: data spaces are global; any time you have a send object sending data to a destination named "suitcase", anyreceive object in any Max patch you have open and running with a receive object that specifies "suitcase" as its source will receive and output the message. So I run my LFOur patch alongside whatever Max patch I want to use it with, and as long as I've added receive objects with arguments that correspond to the names of the send objects in the LFOur patch, I've got data to work with. In addition, I've added a few receive objects that will let you control some functions of the LFOur patch from your patch.Here's a listing of the data that my LFOur patch makes available when it is running:

 There are four outputs (named nvA - nvD) that correspond to the note values for LFOs A, B, C, and D. The messages sent correspond to what you see on the control panels for each of the four LFOs (e.g. the output is sent to an r nvA object would be 1n).

 An output named wavelist continually outputs the current value of the scaled integer values (in the range 0 - 127) shown in the moving window display.

 LFOur has a single  send~ object that sends the summed output of the four combined LFOs at audio rate to the destination audiowave. The output corresponds to what you see in the oscilloscope output.

 There are four outputs (sampA - sampD) that correspond to the most recently sampled waveform output value (in the range 0-127) driven by the notevalue sampling rate. These values are displayed above the display on the far right.

 There are four outputs (named bangA - bangD), each of which triggers at the start of the waveform's output. Each bang is displayed above the third output display.

The LFOur patch also contains several receive objects you can control from your own Max patch.

 Sending a bang with a destination L4sync will automatically synchronize all four LFOs to their starting point (0.) simultaneously. It is the equivalent of clicking the sync button located below the third output display.

 Sending a zero or one with a destination L4start will toggle audio processing in the LFOur patch.

 Sending a zero or one with a destination L4samp_on will enable or disable sampling the LFO waveform ouputs. When sampling is off, you can use the rightmost display to set the sliders to that the same value will be output every time an LFO starts its waveform and sends a bang message.

 Sending a number in the range 0 - 23 with a destination L4samp will change the note value rate at which values are reported (you might want to copy a umenu object from the original LFOur patch and connect its left outlet to the send object instead of trying to remember what numbers to send). Since the rate at which the LFO waveform’s position is sampled is not tied to the rate at which values are reported, setting different note values for the sampling will produce different sets of values = at very high samples rates [e.g. 64n], the outputs will essentially be in sync with the third waveform display.

Next time, we'll create a simple patch that uses the LFOur patch to make some noise.


I'm personally a lot more interested in the ability to synchronize processes in Max using time values that resemble musical note values to create control structures that can be easily time synced. This tutorial is about making one of those kinds of modules - a quartet of synchronized LFOs whose outputs I can sample individually for several kinds of data (triggers for waveform start, LFO outputs that I can sample at variably synchronized rates, and a nifty summed waveform I can use for more exotic kinds of control).

lfo-tutorial-1-the-zen-of-the-silent-patch

Florian Demmer

With the increased importance of social distancing, many people are facing the need to teach or present work using Max remotely. Many schools have moved their courses online, and streaming video has emerged as a popular way to connect with people socially across distances. This article will provide some practical tips on how to set up audio and video streaming from your computer in order to share it with others.

Modern video conferencing services and solutions usually apply intelligent filters and compression algorithms in order to optimise the stream for low latency and speech intelligibility, usually at the cost of audio quality. While the following won’t provide an audiophile streaming experience to your audience it does provide a solution where both the latency and quality are good enough to illustrate what is going on with your patch.

While modern browsers are equipped with technology (WebRTC) that allows peer-to-peer communication between users, it’s usually more difficult to set up and furthermore prone to issues with the networking layer if people are behind proxies or firewalls that can often be found in educational institutions. This guide therefore opts for a more classic streaming approach in order to increase accessibility and ease of use.

 Check out the other streaming article - Streaming Tips - The Jitter Edition 

The key idea here is to use local software in conjunction with a streaming platform for creating a video stream that can be accessed by others by means of visiting a url in their browser. Additional to providing a video stream most services include features for interacting with your audience like a live chat or others. In order to achieve this you will need to install the following:

OBS - Open Broadcaster Software as local streaming client

A virtual or physical audio input, which can be used to route the audio output from Max into OBS

examples for virtual AudioDrivers on OSX would be BlackHole or Loopback

An account with a streaming platform of your choice, two popular options are Twitch and YouTube

In this guide we’ll be using BlackHole as a virtual audio driver, OBS and Twitch as a video streaming service. Twitch offers us the possibility of 160Kpbs stereo audio, as opposed to video conferencing services that usually work with mono and even further compressed audio streams.

If you do not already have an account please visit https://www.twitch.tv and create a new user account to use for your stream.

We’ll start by setting up the audio in Max in order to be able to use its output as an input within OBS. Assuming that BlackHole has been successfully installed simply open the Audio Status Window (Options => Audio Status) and select BlackHole as the Output Device. Also make sure that the Sampling Rate is set to 44100.


Adjusting the OBS settings for streaming to Twitch

After successfully installing and opening OBS we need to adjust the settings for streaming to Twitch. In case you are greeted by the Configuration Wizard on your first launch of OBS you can simply skip that for now. To adjust the settings for our Twitch stream open the Preferences (OBS => Preferences) and navigate to the Stream section. Within that section select Twitch as the service, leave Auto as the Server.

Next click on the Link next to Stream Key , which will open your web browser and navigate to the Settings Page of your Twitch Account. Copy your Stream Key from here and paste it into OBS.

Next up we’ll adjust the parameters for our stream within OBS. For this guide we’ll be streaming using a resolution of 1080p and 160kbps audio. However, this might be demanding for your machine and network so please refer to the encoding recommendations provided by Twitch https://stream.twitch.tv/encoding/ in order to adjust the parameters accordingly.

Navigate to the Output section of the preference screen and make sure that Simple is selected as the Output Mode. We’ll use a Video Bitrate of 6000 kbps according to the recommendations. Leave the Encoder as Software (x264) and set the Audio Bitrate to 160. Also make sure to enable the Advanced Encoder Settings and Enforce streaming service bitrate limits and select veryfast as the Encoder Preset.

Next we’ll head to the Audio section to ensure that we are using a Sample Rate of 44.1 kHz to match our setting in Max and that we are streaming in stereo. Underneath this we can set up our Audio Devices. In order to route the output from Max into OBS we select BlackHole for Mic / Auxiliary Audio. If you’d like to additionally use a microphone input you can select this as the second device.

We’ll now head to the video settings and set our resolution. In order to stream with 1080p we set the Base and Output resolution to 1920x1080 and the Common FPS Value to 60.

With that, we have prepared OBS for our stream and can close the Preferences Window.

At the bottom of OBS’ main window you can adjust the sources of video input that should be included in our stream. OBS is a powerful tool that allows you to combine various sources of video input into a single stream. For the sake of this guide we’ll stick to simply sharing the entire screen of the streaming computer, however feel free to adjust this according to the needs of your stream. 

If there are already sources added to your stream simply remove them by selecting and clicking the Minus Button. To add a capture of your screen click on Plus Button in the Sources section and select Display Capture. Click OK in the opening dialogue and next up you will see a window that allows you to adjust the properties of the Display Capture. This would for example allow you to select the display to use in case you have multiple displays connected, hide the cursor or perform any cropping. You can simply click on Defaults in the bottom left corner and then click on OK. You should now see your display getting shown within OBS and can simply use the red border to resize the captured display to fit into the stream.

To start our stream, click on Start Streaming in the bottom right corner of the OBS window. Once the stream is successfully connected and live you should see a bright green square on the bottom right corner. 

You should now be able to access the Creator Dashboard on Twitch to verify your stream is up and running by visiting https://dashboard.twitch.tv/ and running with a good quality. From here you will also be able to interact with your viewers using the Live Chat. During your actual live stream it might be a good idea to turn off the local preview of your stream in order to save additional bandwidth.


In order to be able to monitor the audio you will be streaming we need to ensure that OBS' monitoring is routed and setup correctly. First off open the Preferences again and switch to the Audio pane. In the Advanced section you can use the dropdown to select the Monitoring Device of your choice.

Next up we will need to enable the Monitoring for our stream. For that  open OBS' Advanced Audio Properties via the Edit Menu and set the option for Audio Monitoring to Monitor and Output on the Mic/Aux device.

A fundamental limitation you will potentially encounter is the speed of your upstream connection. Most people have faster download than upload speeds. For example, an example cable modem speed test might say you have 130Mbps download and 12Mbps upload. Therefore it is recommended to test the speed of your connection before your are asking others to join your stream. You will end up having to adjust your video output settings accordingly. As a general help please refer to resources like Twitch’s Encoding Guidelines.

We chose to illustrate how to stream your Max patch using Twitch for this guide given the simplicity and accessibility of the service. However, if you have certain requirements like unlisted or private streaming you might want to consider using a different platform.

You can use OBS to route multiple audio sources into your stream but can of course also use an adc~ within your Max patch and combine it with the audio produced by your patch if that’s easier for your setup .
We hope this guide helps you get started using Max in a live streaming environment, and encourage you to leave a comment if you run into trouble or have any other tips you can offer.

max-livestream.png

Many are facing the need to teach or present work using Max remotely, this article shows how to set up streaming from your computer.

tips-for-streaming-your-max-patch

Tips for Streaming Your Max Patch

Darwin Grosse

Max 7 includes a new set of tools developed by Matthew Davidson for his work as a Berklee School of Music instructor. The idea is to have a set of objects that would be as easy to use as the Vizzie modules, but would mirror some of the functionality of a Eurorack modular synthesizer. The result is BEAP: the Berklee Electro Acoustic Pedagogy - a fancy way of saying “A Modular Synthesizer in Max!”

We are going to highlight individual BEAP modules over the next few months, but we should start off with a basic patch: our first BEAP synthesizer. Follow a step-by-step demo of the creation of a simple synth, and use it to explore the BEAP system on your own.

Max 7 includes the BEAP modules, developed by Matthew Davidson for his work as a Berklee School of Music. 
Learn more about the video tutorial series here: 
https://cycling74.com/2015/09/15/a-few-minutes-with-beap-tutorial-series

A Few Minutes with BEAP: Create a Simple Synthesizer (Part 1)

a-few-minutes-with-beap-tutorial-series

A Few Minutes with BEAP, Part 1

Les Stuck

Ever wanted to play multiple notes (voices) of your synthesizer or sampler at the same time?  In Max, this is accomplished by using the poly~ object. In this video tutorial, we'll get you up to speed on how to use poly~, so you can start making your own polyphonic instruments.

How poly~, adsr~, and thispoly~ work together.

Cycling74 Max/MSP/Jitter Tutorials: Polyphonic Synth

poly-synth-tut.jpg

polyphonic-synthesizer-video-tutorial

Polyphonic Synthesizer Tutorial

Andrew Benson

One of the most feared and respected objects in the Jitter collection, jit.expr arrived on the scene as part of Jitter 1.5.  In some circles, there is a belief that harnessing its power will bring you great powers and enable you to achieve untold wonders.

The fact is, jit.expr is a really amazing tool, but unfortunately many users are intimidated by the syntax, which includes a lot of square brackets, parentheses, and special variables.  While these sorts of things are common for programmers, they can seem very esoteric and strange to many Max users.  Many avid readers of the Jitter Recipes got stuck when jit.expr was used to generate the quads in the TinyVideo and Shatter recipes.  Upon getting familiar with the expressions, however, it will become an extremely handy tool that makes seemingly difficult and complicated tasks much simpler.  Through this article, I hope to demystify this elusive object and help you to achieve the wonders that you are dreaming of.  We'll start by covering the basic elements of jit.expr expressions and then use these to create some basic geometry and image masks.

Download the patches used in this tutorial.

The jit.expr object doesn't do much out of the box.  In order for it to output anything more than black pixels, you'll have to give it an expression to work with, using the 'expr' attribute. The expressions work in a similar fashion to jit.op where you are performing some arithmetic on each cell of the incoming matrix.  If you'd like to just pass through the input from the first inlet of the object, you can type '@expr in[0]'.  The in[0] variable stands for the pixel value from the first inlet.  The second inlet would be in[1], and if you have more inlets, they are numbered sequentially (in[2],in[3],etc.).  If you'd like to add two jit.matrix inputs together (similar to jit.op @op +), you can use '@expr in[0]+in[1]'.  Still with me?  This is where it starts getting fun.

In addition to accessing specific inputs, the 'in' variable also allows you to access a specific plane of the incoming matrix in your expression by using '.p'.  For example, if you use '@expr in[0].p[0]', that gives you the first plane of the first input.  To get the second plane, you could write '@expr in[0].p[1]' or '@expr in[1].p[2]' to get the third plane of the right input matrix.  So, if we use '@expr in[0].p[1]*in[1].p[2]+in[1].p[0]' we would be multiplying the second plane of the first inlet with the third plane of the second inlet, and then adding the first plane of the second inlet.  That was quite a mouthful, but hopefully you get the point.  This can be really useful when you have an expression that requires multiple variables, and can take the place of using jit.unpack/pack in some cases. 

	On a related note, you can also create a unique expression for each plane of the output matrix by simply passing a list of expressions to the 'expr' attribute.  Each expression is passed in as a symbol, so it is often beneficial to put quotes around your expressions, especially if it includes spaces.  For the most simple case, we could type '@expr in[0].p[0] in[0].p[1] in[0].p[2]', which will just pass the appropriate planes from the first inlet.  It's a lot of typing for something so simple, but we'll be making a lot of use of this feature later, so keep it in mind.  It's especially useful when generating geometry, since you will often need to use different equations for the x,y, and z values.

When generating geometry or image masks, you will often need to generate a whole matrix full of values based on an equation.  For the simple cases, you could probably use an object like jit.gencoord to generate unique values for each cell of your matrix.  One of the great features of jit.expr is the ability to generate cell values by working with the norm[],snorm[],cell[], and dim[] variables.  Which one you use will depend on what you are trying to accomplish.

These variables each require you to specify which dimension you would like to generate values across (horizontal=0,vertical=1,etc.) between the square brackets.  The dim[] variable simply passes the size of your matrix across the specified dimension.  This will be the same for all of the cells of the matrix, and is useful for situations where you would like to adjust values in proportion to the size of the incoming matrix.

The simplest of the other three variables is cell[].  This variable generates integer coordinates across the specified dimension.  So, if we have @expr cell[0] cell[1], the first plane of the matrix would have values {0.,1.,2.,3.,…} across the horizontal dimension (starting on the left) and the second plane would have values {0.,1.,2.,3.,…} across the vertical dimension (starting from the top).  While this might not be so useful as an actual output value, it makes certain things like modulo (%) expressions a lot easier to write when you are working with integers.

Note:  You may have noticed the decimal point in all those values.  This is to remind you that jit.expr uses floating-point arithmetic internally, and will convert to whatever data type you are using upon output.

Now that we know how to use cell[], norm[] is pretty simple.  This handy variable outputs values between 0. and 1. across the specified dimension.  Writing @expr norm[0] will generate the numbers   {0.,…,1.} across the horizontal dimension, starting on the left.  This also creates a nice horizontal linear gradient.  This expression is the equivalent of @expr cell[0]/dim[0] but requires much less typing. Our final variable, snorm[] is very similar to norm[], but with the numbers stretched out to cover values between -1. and 1.  This is especially useful when generating vertex values for OpenGL geometry.

In working with imagery, it is often necessary to do things like generating an oval mask for an image, or other shapes that allow you to escape the oppression of rectangles.  Now that we've gone through how to use the built-in variables and put together simple expressions, we'll use them to create a few different simple mask shapes.  To start, let's try creating a simple oval mask.

Now, in order to create an oval mask shape, we'll need a way to generate values that radiate out from the center of the matrix, rather than just generating linear values.  To do this, we can use the hypot() function built into jit.expr, which calculates the length of the hypotenuse of a right triangle, often used as a distance equation for two points.  If we use @expr hypot(snorm[0]\,snorm[1]), we'll get values that increase as they get further from the center.  Remember that snorm[] creates values between -1. and 1. so the center cell will be 0.  So, we're getting close, but we have the opposite of what we really want (although an inverted oval mask can be pretty useful too).  To invert the values, all we have to do is alter our expression slightly: @expr 1.-hypot(snorm[0]\,snorm[1]).  Now we have a nice radial gradient.  Now, you might notice the "\" in that expression.  That's there because Max likes to interpret commas in a special way, and we need to include the comma as part of the expression.  The backslash prevents the comma from getting eaten up by Max.  Suppose we would rather have a hard-edged mask instead of a soft gradient.  We could easily add a threshold value by altering our expression again: 

@expr (1.-hypot(snorm[0]\,snorm[1]))>in[1]

This little extra portion sets the output value to 1. if our expression result is greater than the value at the second inlet.  The parentheses are there to enforce the appropriate order of operations.

We could now take this a step further and create other mask shapes based on simple geometry functions.  The first one we will look at is using sin() to generate another gradient shape.  If we use sin(norm[0]*TWOPI), we will get a full-phase sine wave across the horizontal dimension.  In this expression, we are using "TWOPI", which is one of the built-in constants (2.*PI) for jit.expr, provided for our convenience.  There is also PI, HALFPI, etc. Multiplying the norm[0] by an input value will change the frequency of the sine-wave, which can be useful for creating repeated shapes.  We could then take this expression further by multiplying a horizontal sine gradient by a vertical sine gradient.  Now, if we are using this to generate imagery or masks, the negative values aren't going to do us much good.  To invert the negative values, we can use the abs() operator (absolute value) on the result.  Now we have a nice repeating lump gradient pattern.

By now you are probably noticing that these expressions can get pretty long and complicated with just a little tweaking.  An important thing to remember is that jit.expr is an object like any other object, and there is no rule that you have to do all of your math within a single expression.  In fact you will probably find situations where you need multiple jit.expr or jit.op objects to accomplish what you need.

As you are experimenting with jit.expr, you will probably want to have some sort of visual feedback to get a better sense of what is going on.  For simple cases, jit.cellblock is a great solution, since it allows you to peek at the number values of the output matrix.  You can also visualize your jit.expr output using a jit.window which will represent the values as color intensities.  Often times this gives the most intuitive feedback.  Another way that jit.expr really shines though is in generating 3D geometry that can be visualized using jit.gl.mesh.  Without going into too much detail, jit.gl.mesh allows you to create different sorts of 3D forms by passing in a Jitter Matrix.  The matrix is then translated in different ways depending on the chosen "draw_mode". By experimenting with different modes, you can visualize your jit.expr expressions in different ways.  And once you get comfortable with the different modes, you can start creating expressions specific to the draw_mode you would like to use.  Remember that jit.gl.mesh expects a 3 plane (x,y,z), float32 matrix for the vertex input, so you will need to think about how you would like to generate positions for each of the coordinates.  If you would like to apply a texture to your 3D shapes, you will also need to generate texture coordinates for each cell (2 plane float32, 2nd inlet).

A very simple thing, which is very useful in real life, is to create a simple textured plane (similar to jit.gl.videoplane).  To do this we can use a 5 plane float32 matrix, and use the following list of expressions in jit.expr: snorm[0] -snorm[1] 0. norm[0] norm[1].  If we break it down, we are just creating a simple grid for the x and y values by using snorm[].  You may have noticed that we left the third expression as "0."  Since we are creating a flat plane, we can just zero out the z values.  For texture coordinates, jit.gl.mesh expects values between 0 and 1., so we are just generating a grid of coordinates using norm[].  Simple, right.  Now, I'll leave it up to you to imagine different sorts of equations that might provide more exciting results.  You might need to brush up on your trigonometry, or if you are like me, fake it.  I'm including a patch called simple-shapes that demonstrates a couple of simple and potentially useful shapes using the tri_grid draw_mode.

I hope this is enough of a boost to get you started writing your own expressions with jit.expr.  In future articles, we'll look at some of the more advanced features of jit.expr and ways to combine it with other Jitter objects to create unusual and wonderful results.  With a little practice and study, you too can learn to tame this mysterious animal.


demystifying-expressions-in-jitter

Demystifying Expressions in Jitter

Rob Ramirez

In this article we’re going to continue exploring some options for streaming content from your Max patch to the internet. This time we’ll look at how to use Jitter texture output as a streaming input source, and go over some techniques for creating and sending to a virtual camera device for use in webcam apps.

 Check out the other streaming article - Tips for Streaming 

On both platforms we will again let OBS manage the streaming I/O. OBS is a fantastic open source project with tons of functionality either built-in or available via plugins. Additionally, we’re going to use Syphon/Spout texture sharing to send our output to the streaming app. OBS on Mac supports Syphon sources natively, and on Windows there is a plugin to support Spout sources.

Windows - install the Spout package and this OBS Spout plugin

Note - On Windows machines with dual graphics cards, care must be taken to ensure all apps are set to use the high performance GPU rather than the integrated GPU.

Previously we looked at how to stream to a single destination (Twitch) via OBS. I was recently clued in by C74 homie Chris Martin on how to turn that into multiple destinations via a service called Restream, which the Coaxial crew uses for their Experimental Half Hour show.

Syphon/Spout usage is detailed in the Best Practices for recording article and we’ll use many of the same concepts in order to achieve streaming Jitter content. In both situations we must decide which patch elements to send down the pipes, and how they should be composited, and then capture those elements to a texture that feeds the server object (a jit.gl.syphonserver on Mac, or a jit.gl.spoutsender on Windows).

In my recent lockdown related streaming experiments I implemented a very basic background separation and monochrome effect that I streamed out of OBS to Restream which went live on Facebook and YouTube simultaneously. Let’s look at the steps necessary to make that happen. 

A quick note before jumping in, HD video streaming requires a beefy machine with a beefy gpu and internet connection. Mileage will vary wildly depending on these specs.

The recording article mentioned above starts by asking “What to Record”, and we must again decide on the what before moving to the how. In the provided patch I’ve mashed a few different sources and effects together just to keep things spicy. The core of the patch is a simple foreground video echo effect. 

We start with a jit.grab camera source (output_texture enabled of course) run through a background subtraction algorithm and then piped through to some Vizzie effects. We also implement a mesh-shatter effect to demonstrate how geometry can be captured and composited with texture streams via jit.gl.node @capture 1. Throw in a glitchy little ISF transition and some parameter animation via jit.mo and an ATTRACTR module to polish off.  To run the patch make sure you've installed the ISF Max package and ISF library files, or substitute the transition object with a Vizzie XFADR.

Now that we’ve got a sparky live video patch to play with, let’s send it off. The basic rule of thumb with Jitter texture streaming is any object that outputs a texture can be patched to a syphon/spout server object. If you want to send off geometry, capture it to a texture with jit.gl.node, or if you want to just send the entire contents of your jit.world window, enable output_texture and connect the server object to its leftmost outlet.

Once your patch has passed off its texture through the sender server, any client software running on your machine can grab the stream. Max is pretty much out of the picture from this point. OBS streams out to Restream and Restream handles sending to any other desired platforms. After logging in to Restream client services can be added to the streaming session in the Destinations section.

For OBS setup follow the instructions outlined in the previous streaming article. Copy the RTMP URL and stream key from Restream and enter in the OBS Settings window Stream section. Adjust any stream and video settings (e.g. bitrate, dimensions) as desired for your setup. Add a video source to your OBS scene by clicking the plus menu under the Sources section and select Syphon Client on Mac or Spout2 Capture on Windows and setting Max as the source (making sure you’ve installed the plugin linked above if on Windows). If more than one sender server is running on your machine, open the source properties by clicking the gear icon to select the appropriate server.

Note that it is possible to stream from Max via an OBS Window Capture source instead of using Syphon or Spout, but in my testing the frame rate dropped significantly. It also requires the Jitter window dimensions to match the streaming dimensions, so a less viable solution in most cases.

One last note, OBS makes it dead simple to make a local disk based recording of your stream, which generally is a good idea. By default it will match the recording settings to the stream settings, and therefore run more efficiently.

Ok so we’ve achieved our goal of streaming Jitter content through some different streaming services, but we live in the age of the Zoom meeting, how does one liven up their daily teleconferencing? For this we are going to need to create a virtual camera device to pipe Jitter through. Fortunately there are free programs on each OS that manage this device and receive Syphon and Spout textures streams.

On Windows install Spout Cam via the Spout installer

The setup is exactly the same as above except instead of sending to OBS, we will send to our virtual camera device. On Mac with Camtwist make sure you’ve set the video size and framerate in the preferences window before enabling your input. This should match the dims and framerate of your texture output.


Once you've updated your preferences, select Syphon as the source in the section titled Step 1, add effects if desired in Step 2, and then select your Max server as the source in Step 3. That’s it, you can now select Camtwist as your video device in your teleconferencing app (e.g. Zoom, Skype, Google meetups, etc).

On Windows we're going to run the Spout installer (note this is different from the Spout Max package), which will install a utility called Spout Cam. After installing, you will be able to select Spout Cam as the camera in Zoom. Spout Cam should match the dimensions of your texture input by default. As mentioned above, you must make sure all apps are using the high performance GPU for this to work. 

Not all webcam apps are able to detect and use Spout Cam directly, e.g. Skype did not in my testing. For these cases we can install the SplitCam app to act as a shim. SplitCam creates its own virtual camera device, and is also able to receive textures from Spout Cam. So with SplitCam running and its input set to Spout Cam, we are able to send directly to Skype.

In this final section let’s go even further down the rabbit hole and investigate how to improve the background subtraction. The patch above does a decent job of this, but it requires controlled conditions in order to be effective. Any changes in lighting and shadows, or any small movements of the camera cause the effect to break down. Modern day camera apps typically use smarter algorithms, likely built on machine learning and extensive datasets to handle the segmentation. To up our game we can install the popular (and free) Snap Camera app, and search for a chromakey lens.

Before we dive deep into this let me say, in my experimenting I encountered different problems on both platforms. The following is presented primarily as an exercise in problem solving, rather than a practical solution. If that's not interesting to you feel free to skip this section.

On Mac the OBS NDI plugin and NDI Runtime 

First, launch Snap Camera and enable the chromakey lens of your choosing. On Windows, jit.grab can directly receive the output from Snap Camera, but I was unable to get it working directly on Mac (not sure why). To workaround we can again make use of OBS, but this time instead of piping Jitter to OBS, we pipe OBS to Jitter using something called NDI. Install the NDI plugin and runtime linked above, then in OBS create a new Video Capture Device and set the device to Snap Camera. Click on the Tools menu and select NDI Output Settings and click the checkbox next to Main Output. Next launch the NDISyphon app and in the NDI Clients section you should see your OBS source. Click on that and enable it. Finally in your max patch you can receive the chromakeyed image with a jit.gl.syphonclient object.

If you want to take this monstrosity one step further, we can actually use OBS to both send from Snap to Jitter via NDI, and from Jitter to a streaming service. The trick is to enable Studio Mode by clicking that button in the lower right section of the interface. Then in the NDI Output Settings select the Preview Output instead of the Main Output. Finally you will set the Snap source as the preview and the Syphon source as the program.

Despite the ridiculousness of this setup, it does actually work and as proof I live-streamed out a little test clip using my patch above and replacing the background subtraction sub-patcher with the Snap Camera chromakey lens output. Check it out here.

You might be wondering how all this streaming stuff works out in practice. For that I'm going to turn it over to our good buddy Sam Tarakajian, who recently participated in an "online micro festival" called More Kicks Than Friends (Sam's bit starts around minute 38) presented by Max for Live master Ned Rush.

I wanted to perform something with my friend Alex, but of course the challenge was that we couldn't jam together in person. Our solution might sound a bit convoluted, but I was very pleasantly surprised by how well it turned out. First, Alex streamed his musical improvisation to me over Twitch. I kept that stream tab open in my browser, and fed the sound into Max using Loopback. If you don't want to buy Loopback, you could just as easily use Soundflower or BlackHole.

Once I had the sound in Max, I did some audio analysis to drive a couple of envelope followers, adding some audio reactivity to my Jitter patch. Then, so Alex could get a sense of what I was doing (albeit with some latency), I used exactly the technique that Rob describes: Max to Syphon to OBS to Twitch. Since Ned was asking for prerecorded video, I also used Syphon Recorder to record video locally (Rob describes using OBS for this, which would have been pretty much the same result). After we finished, we edited Alex's local audio together with my local video, and sent the result off to Ned. Luckily Syphon Recorder can record audio as well, which helped Alex edit the video properly.

On my machine I was simultaneously generating live visuals with Jitter, performing GPU accelerated face tracking, streaming video from Twitch, streaming video to Twitch, and recording video to disk. I'm amazed it was able to work at all, let alone as well as it did. But far from being a huge technical headache, all of the streaming tools we used were surprisingly easy to use. I'm looking forward to trying more streamed performance in the future.

restream-cover.jpg

Tips and techniques for sending Jitter video content to streaming platforms and teleconferencing apps. 

streaming-tips-the-jitter-edition

Streaming Tips - The Jitter Edition

Andrew Pask

DMX, or more accurately, DMX 512, is a network protocol most commonly used for the control of stage lighting and effects. It is easily programmed in Max, using techniques similar to those used to program MIDI. With your Max skills, you can  replace and extend the functionality of dedicated DMX controller/mixer hardware boxes with patches.

This is the first article of three in which we shall explore the world of programming DMX with Max.

All the tutorials in this series: Part 1, Part 2, Part 3.

Over the course of these articles, we will present some basic DMX concepts and an introduction to some DMX hardware options that are usable with Max. The second article will feature interviews with Max programmers that have experience with DMX in professional environments. Finally, the third article will focus on development of a rather complex DMX control system.

The name “DMX” is short for “Digital Multiplex”.  It is a network protocol for passing packets of control data over device network. The “512” in DMX 512 represents the maximum number of devices (512) that can be controlled with a single packet. This network of 512 devices is called a DMX “universe”.

In a typical DMX 512 network, devices are daisy chained together. The master (or control) device is at the beginning of the chain, with a cable running to one device, and a cable run from that device to the next device, and so on. DMX always travels in one direction – it is unidirectional, and the end of the chain needs to be terminated.

Here’s a picture of the simple DMX network we used to make the video.

In this picture, we have a laptop running our Max patcher connected to a DMX controller, which in turn sends packets down the DMX network cabling to connected lighting devices.

Devices in a DMX universe are specified by channel, with up to 512 channels in a single packet. A packet contains a header followed by the control data expressed as 8-bit values (bytes). Since this is 8-bit data, we use the values 0-255 (in decimal) when programming DMX data in Max.

There is no index in a DMX packet, so if a device thinks it is on channel one, it will look at the first data byte in a DMX packet and choose that value to set its state. If a device thinks it is on channel 53 in the current universe, then it will choose the 53rd byte in the packet for its value.

Each DMX controllable device has built-in channel selectors, allowing you to specify its channel numbers within the universe. Let’s look again at our example DMX network. The first device is a 6-channel LED “spot”. The 2nd device is a 4 channel dimmer starting on channel 8, with 2 light bulbs hooked up on the first two channels.

As an example, if we send a DMX packet with the following data:

The result will be to set the RGB led to white, the first light in the dimmer to half way on, and the 2nd light in the dimmer to all the way on. Notice a few interesting things about this data packet:

We don’t need to send all 512 data values; but we do have to send data for all the devices in our DMX universe.

The LED spot only uses 3 channels of color data, even though it is a “6-channel” device.

There needs to be 7 channels of data allocated before we get to first channel in our dimmer (channel 8). Since we are only using 3 channels to drive the LED, we need to pad out the packet with data for the unused channels.

Many DMX devices are like the 6-channel RGB spot, which has multiple channels to control various parameters of their state; for example, a DMX led array can have hundreds of channels of control. When programming multi-channel DMX devices, you usually need to dig into the documentation for each device in order to find out how each channel is used.

The first thing we need is a DMX interface. In my case, I have a LanBox LCX by the Dutch company LanBox.

The LanBox-LCX is an extremely well equipped piece of equipment. It offers connections in a variety of formats, including Ethernet, MIDI, and USB, or can be operated in standalone mode. It can be used on an Art-Net network. It is a kind of “Swiss Army Knife” of DMX network control. The LanBox website contains a page with the freely available Max externals which are required for communication with the device, plus some examples of patching code. This unit, while not the cheapest DMX interface available, has all the bells and whistles, and feels solid and well built.

In addition to a controller, you’ll need devices to control and some cabling to hook it all up. DMX cables are a mix of 3 pin and 5 pin XLR, so expect that you’ll need to buy some adapters. There is an incredible range of DMX controllable dimmers, lights, fog machines, lasers, servos, and amplifiers out there. Two of the most common brands of DMX capable lighting gear are Chauvet and American DJ.

Once we have downloaded and installed the LanBox Max externs and examples from their site, setting up for patching is straightforward. You can follow along with the LanBox documentation concerning setting an IP address for the LanBox on your network, or you may be lucky enough to find out that the default IP address for the LanBox works fine at your location.

For my example, I have an American DJ P36 LED spot connected as the first device on my DMX daisy chain. This light has 6 channels of DMX control. Next comes an Elation DP 415 4 channel dimmer pack, and I have placed standard incandescent light bulbs on channels 1 and 2 of that device.

On the back of the 4 channel dimmer, I’ve set the DMX channel offset dipswitches so that the dimmer’s first channel is channel 8.

There are many ways you can manage lists of numbers in Max. To make things obvious, I’ve decided to go with message boxes.

The swatch object is set to “Output Old Style 0-255 values” in its inspector. This gives me the exact range I need for my DMX channels. The first 3 channels on my LED spot are for R, G and B, so I’m setting them up as the first three values in my message box labeled “channels 1-7”. I have to pad out the message box with unused channels because the first channel on the dimmer box is set to channel 8, so it will use the 8th value in the DMX packet.

The output from the sliders and toggles intended to drive the light bulbs is made into a similar message, which represents the 4 channels on my dimmer. I then join the two messages together and use them to set the contents of a third message box, which is the whole packet I need to set state in this system.

My LanBox has a hardware clock which runs at 20fps, so I’m using a qmetro to update the state of the DMX network every 50 ms.

Search Results