Lilli Wessling Hart

The OpenGL Architecture Review Board has released the official OpenGL 2.0 specification. The new version includes:

Many of these features are already available on certain graphics cards on certain platforms, but it is nice to see these things having made it into the specification.


The OpenGL Architecture Review Board has released the official OpenGL 2.0 specification.

open-gl-20-specification-released

Open GL 2.0 Specification Released

Andrew Benson

The second installment of Jitter Recipes is a collection of simple examples that began as weekly posts to the MaxMSP mailing list. Here you will find some clever solutions, advanced trans-coding techniques, groovy audio/visual toys, and basic building blocks for more complex processing. The majority of these recipes are specific implementations of a more general patching concept. As with any collection of recipes, you will want to take these basic techniques and personalize them for your own uses. I encourage you to take them all apart, add in your own touches and make these your own.

All the books in this series:  Book 1, Book 2, Book 3, Book 4 

Using jit.buffer~ to create simple audio solutions.

This example shows a couple of techniques that take advantage of the matrix interface of jit.buffer~.

On the left half we are recording a live signal into the upper buffer. This buffer is then output as a matrix, and fed through the jit.slide object to slowly fade between buffer states. This faded buffer-matrix is then loaded into the second jit.buffer~, which is what we will use for playback 

On the right half, we use a multislider and jit.fill to provide an easy drawing interface for waveshaping. Inside of the "use exprfill" subpatch, we've provided several common expressions that can be used to fill a jit.matrix with a windowing curve. This is particularly useful for doing granular synthesis.

Recording gestural parameter data over time

When recording high resolution videos from Jitter, it is often very difficult to get acceptable results from real-time manipulation of parameters. For this reason, it is often helpful to record real-time parameter changes on a frame-by-frame basis, and then play back these parameters at a rate that jitter can keep up with.

On the left half, we are using the pattrsorage object to save the state of all the named UI objects for each "frame". The benefit of this implementation is the ease and robustness of the pattr-family of objects as well as the ability to save the state of objects such as matrixctrl. 

The right half uses a matrix to store our parameter values, and then records frames into a jit.qt.record object.

Note that we're using jit.coerce to force our float32 matrix into an acceptable format for QuickTime (char).

Using Matrix feedback and distortion to create movement

This example is functionally very similar to the VideoSynth1 patch, except that we're using jit.repos instead of jit.rota to do the image distortion.

We are using jit.poke~ to perform automatic drawing on a named matrix (see VideoSynth1) 

This matrix is then distorted using jit.repos and then fed back to be drawn again using our named matrix.

To generate a constantly shifting repos-matrix, we've implemented a similar tactic as that used for the drawing (see bot2 subpatch). This matrix is then mixed with a matrix of normalized coordinates generated by the jit.expr object. This has the effect of gently repositioning the matrix at each frame, thus giving us smooth movement.

The output of jit.repos is then mirrored by sending it to jit.glue and inverting the x-dimension of the right input using jit.dimmap.

Using the stroke messages to jit.gl.sketch to render curves in OpenGL

Using a matrix to generate sketch commands for procedural drawing

With the introduction of JSUI, "stroke" commands were exposed for generating spline-curves. While this is pretty well-documented in the JavaScriptinMax PDF, many people aren't aware that you can also use these commands with jit.gl.sketch to do curvy line drawings.

The movement of the points is generated by adding a matrix of motion-vectors (see ParticleRave-a-delic) 

This matrix of point-locations is then sent to jit.iter to be broken down into 3-part lists.

For every matrix that is sent, we reset our sketch object and then begin our stroke using the "beginstroke line" message, and specifying the order (or smoothness of our curve) as well as color and width.

Each point of the matrix is then sent using the "strokepoint...."

Using expressions to generate values across a matrix

Reshaping a NURBS by setting control-point coordinates

Sometimes, when learning the ins and outs of OpenGL, it helps to be able to see just how this matrix of vertices is affecting our geometry. This example uses jit.cellblock to provide a visual interface to our matrix data. This allows us to perform precise distortions of our surface, which can be very useful for generating interesting graphics.

Using the "exprfill" message, we load our first two planes with signed-normalized values across the x and y dimension respectively. This gives us values between -1. and 1. By filling these matrices in such a way, we create a plane across the x and y coordinates of our GL-world. 

The z-plane is left empty for us to fill in as desired to create spatial distortion.

All of these matrices are then packed into one 3-plane matrix and sent on as the "ctlmatrix" for jit.gl.nurbs

We use jit.gl.slab to map a movie onto our jit.gl.nurbs shape. We could also load a shader program into this object to do processing on our texture.

Creating one's own particle system using matrix operations

Rendering multiple copies of the same geometry

Manipulating text using matrix operations

This patch was published for Xmas 2005, which should explain its holiday aesthetic. A particle matrix is generated which is then rendered as floating text-strings as well as vertices of a jit.gl.mesh. The text is also messed with a bit using some matrix processing.

The majority of these recipes are specific implementations of a more general patching concept. As with any collection of recipes, you will want to take these basic techniques and personalize them for your own uses.

jitter-recipes-book-2

Jitter Recipes: Book 2, Recipes 14-25

bronzebygold

Quasi-Granulizer is not a granular synthesizer, but it works on similar principles. Quasi-Granulizer slices buffered audio into large grains which can be repeated at random (but constrained) intervals through seven “banks” of polys. While granular synthesizers generally use grains only a few milliseconds in length, Quasi-Granulizer is designed to generate grains of a few hundred to a few thousand milliseconds in length, creating interesting textures and ambient sounds.

This work is licensed under the Creative Commons Attribution 3.0 Unported License. To view a copy of this license, visit http://creativecommons.org/licenses/by/3.0/


quasi-granulizer

Quasi-Granulizer

So, you've finished the tutorials, you understand the basics of digital audio, and you can imagine using a jitter matrix for something. Perhaps you are looking for a couple of new recipes to expand your repertoire...

The following is a collection of simple examples that began as weekly posts to the Max mailing list. Here you will find some clever solutions, advanced trans-coding techniques, groovy audio/visual toys, and basic building blocks for more complex processing. The majority of these recipes are specific implementations of a more general patching concept. As with any collection of recipes, you will want to take these basic techniques and personalize them for your own uses. I encourage you to take them all apart, add in your own touches and make these your own.

Max 7.1 (or later) Users: Install a selection of these recipes using the Package Manager inside the Max application.

Rendering multiple copies of geometry with a single object.

One of the most common questions with new jitter users is, "How do I bring audio over into the video world?" You'll notice that many of the following examples deal with this question in one way or another. Using the OpenGL features of Jitter is a great way to create data visualizations of all kinds, and can be integrated easily with other video processes. In this weeks example, we will use a couple of low-frequency cycle~ objects to dynamically reposition spheres in 3-D space.

We need to provide a named matrix for our coordinates to be stored and altered. As we need (x,y,z) coordinates in floating point values, we use a 3 plane float32 matrix. In the video world, we are familiar with using red,green, and blue planes to convey color information. In openGL, these same planes refer to x, y, and z coordinates respectively.

We use jit.poke~ objects to load our named matrix with values.

jit.slide is used to smooth the movement of the balls between positions.

We use jit.iter to break our matrix down into 3-item lists, which are then converted to position messages. We are also banging our jit.gl.gridshape with each message. This creates a copy of our sphere for each cell of our matrix. You will see this technique whenever we need to create multiple copies of a jit.gl-object.

We need to make sure that the jit.gl.gridshape has @automatic 0, so that it will be rendered only when it is banged.

Recording vectors of audio to a jitter matrix.

Randomly resequencing these audio vectors for playback.

When Jitter 1.5 shipped, I immediately set to work imagining all the things I could do with the new jitter audio features. This patch is the first answer I came up with. This example could be considered a basic recipe, in that it may not accomplish anything exciting on its own, but it outlines a general technique that can be applied to all sorts of different processes.

The jit.catch~ object grabs a vector of audio data and sends it out as a 1-D float32 matrix. By using "dstdim" messages to a jit.matrix object, we propagate these horizontal rows over the length of the matrix. This technique is the same as that used to create "slitscan" graphics.

Once we have this matrix full of audio vectors, we could do any number of things, such as running this through other jitter processing, interpolation, etc. Here we are using it as a buffer to randomly select audio vectors in our playback engine. To do this we select individual rows to send to the jit.release~ object.

The jit.release~ object is set to "varispeed" mode to allow pitch control with different metro speeds.

Loading a 3D matrix with multiple frames of video.

Visualizing translucent volumes using jit.gl.volume.

Creating dynamic alpha masks for transparency.

This patch loads frames from a downsampled Quicktime video into a 3-D matrix, and then uses jit.gl.volume to visualize this matrix. You may find that this will slow down upon switching to fullscreen mode. This is because jit.gl.volume must do per-pixel rendering, which can be a pretty heavy operation.

Once we load a video into jit.qt.movie, we pass it through the "alpha-channel" subpatch. This unpacks the 4 char matrix and replaces the alpha channel with a mask generated by performing a luminance gate (sort-of like a noise-gate for video). This is accomplished by using the jit.op @op >.

Using "srcdim" messages to the jit.matrix object, we are able to continuously replace frame-slices in a random order. A similar scheme can be used to create your own video buffer like jit.matrixset. In our case, we are merely looking for a source of data to fill a cube of pixels.

This downsampled video buffer is then sent to jit.gl.volume to be rendered as a nice soft collection of colored blocks.

Visualizing 3D matrix values as OpenGL geometry

Creating dynamic alpha masks for transparent textures.

This is a good example of applying one basic recipe in a variety of situations. Here we use the same basic technique as the Video Pillows example, using the jit.gl.isosurf object for a very different effect.

Because jit.gl.isosurf doesn't use the color of the 3-D matrix for its surface color, we must generate a texture to be mapped onto the resulting geometry. We have the same alpha-channel patch that we saw before, only this time, we are outputting the matrix after the jit.rgb2luma to generate the geometry. This gives us a one-plane matrix that we can use to look up isolevels.

We then use the luminance-gated video as a texture that is applied to our geometry. Once the texture is applied, the transparency mask makes it appear as if parts of the geometry have been removed, leaving jagged edges. This is a good technique for creating seemingly complex surfaces without requiring too many vertices.

Using Amplitude Modulation to create more interesting response.

This example continues to be one of my favorites, as it combines audio modulation techniques with basic matrix processing in a way that generates some very organic motion. At first glance this patch will probably appear to be vastly complex, but the mechanics of it are really pretty simple.

Because we are using a single audio signal to create our geometry, and we need to generate multiple dimensions of movement, we will use our audio signal to modulate other signals. I use the rand~ object to generate randomly ramping signals at various rates. These signals are then multiplied to fall within the dimensions of our matrix (50x50x50). Because we only need one plane of data, we use our input signal to modulate another rand~ signal.

As if it weren't enough to have this matrix full of modulated audio data, we've also added a "growth" matrix. By slightly downsampling the matrix and then mixing with the original, the pixels appear to grow outward. This also creates a low-pass filter which smooths the movement over time.

The resulting matrix is finally fed into the jit.gl.isosurf object to be converted into OpenGL geometry.

This example shows a technique for using the graphic representation of sound provided by jit.graph to render more complex and interesting visuals. A similar technique can be used to create your own Visualizer.

Book 1 contains some clever solutions, advanced trans-coding techniques, groovy audio/visual toys, and basic building blocks for more complex processing.

Search Results