Andrew Benson

So, you've finished the tutorials, you understand the basics of digital audio, and you can imagine using a jitter matrix for something. Perhaps you are looking for a couple of new recipes to expand your repertoire...

The following is a collection of simple examples that began as weekly posts to the Max mailing list. Here you will find some clever solutions, advanced trans-coding techniques, groovy audio/visual toys, and basic building blocks for more complex processing. The majority of these recipes are specific implementations of a more general patching concept. As with any collection of recipes, you will want to take these basic techniques and personalize them for your own uses. I encourage you to take them all apart, add in your own touches and make these your own.

One of the most common questions with new jitter users is, "How do I bring audio over into the video world?" You'll notice that many of the following examples deal with this question in one way or another. Using the OpenGL features of Jitter is a great way to create data visualizations of all kinds, and can be integrated easily with other video processes. In this weeks example, we will use a couple of low-frequency cycle~ objects to dynamically reposition spheres in 3-D space.

We need to provide a named matrix for our coordinates to be stored and altered. As we need (x,y,z) coordinates in floating point values, we use a 3 plane float32 matrix. In the video world, we are familiar with using red,green, and blue planes to convey color information. In openGL, these same planes refer to x, y, and z coordinates respectively.

We use jit.poke~ objects to load our named matrix with values.

jit.slide is used to smooth the movement of the balls between positions.

We use jit.iter to break our matrix down into 3-item lists, which are then converted to position messages. We are also banging our jit.gl.gridshape with each message. This creates a copy of our sphere for each cell of our matrix. You will see this technique whenever we need to create multiple copies of a jit.gl-object.

We need to make sure that the jit.gl.gridshape has @automatic 0, so that it will be rendered only when it is banged.

When Jitter 1.5 shipped, I immediately set to work imagining all the things I could do with the new jitter audio features. This patch is the first answer I came up with. This example could be considered a basic recipe, in that it may not accomplish anything exciting on its own, but it outlines a general technique that can be applied to all sorts of different processes.

The jit.catch~ object grabs a vector of audio data and sends it out as a 1-D float32 matrix. By using "dstdim" messages to a jit.matrix object, we propagate these horizontal rows over the length of the matrix. This technique is the same as that used to create "slitscan" graphics.

Once we have this matrix full of audio vectors, we could do any number of things, such as running this through other jitter processing, interpolation, etc. Here we are using it as a buffer to randomly select audio vectors in our playback engine. To do this we select individual rows to send to the jit.release~ object.

The jit.release~ object is set to "varispeed" mode to allow pitch control with different metro speeds.

This patch loads frames from a downsampled Quicktime video into a 3-D matrix, and then uses jit.gl.volume to visualize this matrix. You may find that this will slow down upon switching to fullscreen mode. This is because jit.gl.volume must do per-pixel rendering, which can be a pretty heavy operation.

Once we load a video into jit.qt.movie, we pass it through the "alpha-channel" subpatch. This unpacks the 4 char matrix and replaces the alpha channel with a mask generated by performing a luminance gate (sort-of like a noise-gate for video). This is accomplished by using the jit.op @op >.

Using "srcdim" messages to the jit.matrix object, we are able to continuously replace frame-slices in a random order. A similar scheme can be used to create your own video buffer like jit.matrixset. In our case, we are merely looking for a source of data to fill a cube of pixels.

This downsampled video buffer is then sent to jit.gl.volume to be rendered as a nice soft collection of colored blocks.

This is a good example of applying one basic recipe in a variety of situations. Here we use the same basic technique as the Video Pillows example, using the jit.gl.isosurf object for a very different effect.

Because jit.gl.isosurf doesn't use the color of the 3-D matrix for its surface color, we must generate a texture to be mapped onto the resulting geometry. We have the same alpha-channel patch that we saw before, only this time, we are outputting the matrix after the jit.rgb2luma to generate the geometry. This gives us a one-plane matrix that we can use to look up isolevels.

We then use the luminance-gated video as a texture that is applied to our geometry. Once the texture is applied, the transparency mask makes it appear as if parts of the geometry have been removed, leaving jagged edges. This is a good technique for creating seemingly complex surfaces without requiring too many vertices.

This example continues to be one of my favorites, as it combines audio modulation techniques with basic matrix processing in a way that generates some very organic motion. At first glance this patch will probably appear to be vastly complex, but the mechanics of it are really pretty simple.

Because we are using a single audio signal to create our geometry, and we need to generate multiple dimensions of movement, we will use our audio signal to modulate other signals. I use the rand~ object to generate randomly ramping signals at various rates. These signals are then multiplied to fall within the dimensions of our matrix (50x50x50). Because we only need one plane of data, we use our input signal to modulate another rand~ signal.

As if it weren't enough to have this matrix full of modulated audio data, we've also added a "growth" matrix. By slightly downsampling the matrix and then mixing with the original, the pixels appear to grow outward. This also creates a low-pass filter which smooths the movement over time.

The resulting matrix is finally fed into the jit.gl.isosurf object to be converted into OpenGL geometry.

This example shows a technique for using the graphic representation of sound provided by jit.graph to render more complex and interesting visuals. A similar technique can be used to create your own Visualizer.

We are using jit.catch~ to convert audio vectors into matrix data, which is then passed off to jit.graph to plot the floating point values of each cell in a waveform shape.

The output of jit.graph by itself looks like a simple oscilloscope representation of our audio. As this on its own is pretty simple, we then pass it on for further processing.

The output of jit.graph is then passed on to the jit.rota object to be slightly rotated and then looped back to mix with the original using a lumakey object. The lumakey object renders all black pixels transparent so that the fed-back matrix will still be visible.

For added visual pleasure, try hooking up more video processing on the end of this chain, such as jit.robcross or jit.ameba.

This patch generates numerous copies of the same videoplane that dance around through space in response to the audio file. This example is one of the more complex patches, using techniques from a couple of other patches we have already seen. If we were to break down the individual components of this patch, you would most likely recognize most of the processing stages.

We are loading a matrix using the same technique as the TimeScrubber patch. This matrix is then downsampled and chopped up into 3 columns using jit.scissors. Each of these columns is then packed in as different planes for a 3-plane matrix. This allows us to generate (x,y,z) values from a single audio input.

On the right side of the patch is all of our rendering process. You'll recognize the "make-alpha" subpatch from the VideoPillows example.

Inside of the "sprite-generation" subpatch you will find the same setup we used with the gLFO patch to render multiple instances in various positions. We use a scale message to resize the "sprites".

Take note that the viewalign attribute of our jit.gl.render is turned on. This enforces "billboarding" of our sprites so that they always face the camera.

This patch was originally posted as a response to a fellow who was looking for a way to do time-lapse audio recording. While this may not be a complete solution to that problem, it does illustrate the ability of Jitter objects to be of great assistance in working with frequency-domain audio.

The whole process here should be pretty clear if you have looked at the previous jit.catch~ examples. The only trick is that we are using 512 sample FFT frames instead of time-domain audio signals. By fixing the framesize of jit.catch~, we make sure that we don't get frame offsets in our matrix. Note that we are capturing both amplitude and phase output from the FFT.

The "output" subpatch allows us to read back our captured FFT-buffer at varying speeds. One interesting experiment would be to upsample or downsample our matrix with interpolation and see what sorts of effects it has on playback.

All the books in this series:  Book 1, Book 2, Book 3, Book 4 

Max 7.1 (or later) Users: Install a selection of these recipes using the Package Manager inside the Max application.

Rendering multiple copies of geometry with a single object.

Recording vectors of audio to a jitter matrix.

Randomly resequencing these audio vectors for playback.

Loading a 3D matrix with multiple frames of video.

Visualizing translucent volumes using jit.gl.volume.

Creating dynamic alpha masks for transparency.

Visualizing 3D matrix values as OpenGL geometry

Creating dynamic alpha masks for transparent textures.

Using Amplitude Modulation to create more interesting response.

Book 1 contains some clever solutions, advanced trans-coding techniques, groovy audio/visual toys, and basic building blocks for more complex processing.

jitter-recipes-book-1

Pardon my ignorance, but how to use these tutorials?

They look like the basis for a live lecture, and not for my feeble attempts at self-instruction

Pardon my ignorance, but how to use these tutorials?
They look like the basis for a live lecture, and not for my feeble attempts at self-instruction


The Jitter Recipes began as occasional patches posted to the Jitter forum, and are intended to supplement the existing documentation and provide practical examples. For more thorough introductions to concepts, read the tutorials available in the Max Help system. If any of the patches interest you and you'd like to learn more about how it works, let's go in the forums and chat about it.

The Jitter Recipes began as occasional patches posted to the Jitter forum, and are intended to supplement the existing documentation and provide practical examples.  For more thorough introductions to concepts, read the tutorials available in the Max Help system.  If any of the patches interest you and you'd like to learn more about how it works, let's go in the forums and chat about it.


The "See all Recipes" link at the beginning of the article is now broken.

Screen Shot 2021-02-11 at 17.44.45.png

Hey There, so as you can see some of the functions are not available to me, is it because I am in a trial period of Max? 

this occurs when I am trying to practice one of the lessons. 

Hey There, so as you can see some of the functions are not available to me, is it because I am in a trial period of Max? 
this occurs when I am trying to practice one of the lessons. 
Thanks in advance

if you enable timestretch, then the attributes related to time stretching will be activated

Jitter Recipes: Book 1, Recipes 0-12