Gen 3: The Fine Art of Surfacing
In the first and second tutorials tutorials in our Gen series, we've taken a look at the gen~ object for processing audio data and the jit.pix object (and its cousin jit.gl.pix) for processing 2d, 4-plane image data. This time out, we're going to have a look at the jit.gen object - while it's just like the jit.pix and jit.gl.pix objects in that all three objects process jitter matrices, the jit.gen object is the generic Gen matrix processing object - it can handle matrices of any type, dimension, and planecount.
Read the other tutorials in this series:
Expressionism
Before the arrival of the Jitter Gen objects, one of the best techniques for processing matrix data in interesting and complex procedural ways was to use the jit.expr object. Andrew Benson's tutorial on working with expressions using the jit.expr object did a great job of demystifying normalized numbers, and our last tutorial on image processing using jit.pix made use of those insights to help explain how Jitter gen objects work with matrix data.
Working with jit.gen uses those same basic skills, but we're going to look at how a little familiarity with the Jitter family of gen objects can make working with expressions that can seem bewildering in jit.expr a lot easier. We're going to generate a data surface - a mesh whose contours are described by a mathematical expression. This is a quick way to grasp just how much the arrival of the Jitter family of gen objects has made your life a more pleasant place.
To start, let's take a look at the jit.expr object in action, and review how we use it to represent matrix data (both in jit.expr and in the jit.gen world, as well. The patch contains all the logic we'll need to start - it provides the standard set of objects we use when generating and rendering matrix data in Jitter, along with a jit.expr object that we'll use to generate our data surfaces.
The patch contains a number of jit.gl.mesh attributes that you'll see in all of our example patches (after you've generated your data surface, you might want to take a few minutes to investigate how they alter the look of the rendered matrix if you're not already familiar with them). We'll create our mesh using a 3-plane matrix of 32-bit floating point data whose dimensions are 100 x 100. Each of the three planes represent a dimension of our 3d data, which we'll send to the jit.gl.mesh object:
x = width
y = depth
z = height/displacement
We'll use a jit.expr object to create our 3d plane by using expr messages to fill the matrix with data values that describe our plane. As we go along, we'll also use variations of those messages to create more interesting variations on the plane.
Let's start with something simple - a plane whose coordinates all lie within the range -1.0 to 1.0. As you may know from Andrew Benson's tutorial (or the previous jit.pix gen tutorial), we can do this by specifying signed normalized (snorm) values to specify the coordinate range for our data surface. Sending the message expr snorm[0] snorm[1] 0. to the jit.expr object does just that -the number in the square bracket indicates which plane we're specifying - since the plane is completely flat, all of the values for the z plane can be set to zero by including a value of 0. rather than snorm.
While that works well, it's not very interesting. We can create variations in our data surface by replacing the zero with a mathematical expression that will describe the z-plane values for every single point on our data surface. The expression sin(snorm[0]*PI) does just that. for every x value across the plane, the z offset is calculated by multiplying the x value by 3.14159 and then calculating the sine of that value (which will always remain in the range -1. to 1.) The result is a single cycle of a sine wave that ripples across the x dimension from left to right.
The expression sin(snorm[1]*PI) has a similar result, but the sine wave propogates along the y axis (from top to bottom).
Suppose we wanted to have the sine function propagate along both axes at the same time. How would we go about doing that? To combine the sine waves together, all we have to do is to combine the two expressions we have, but add the results together and take their average. The expression expr snorm[0] snorm[1] (sin(snorm[0]*PI)+sin(snorm[1]*PI))/2 will do that.
Moving Waves
While this data surface is certainly lovely, it's still a litle static. Don't worry - as I learned from Andrew Benson, the jit.expr object allows you to work with variables in our expressions. In the jit.gen_2 patch, we'll introduce variables and use them to do two things:
We'll let you set the number of iterations of the sine function across the x and y axes.
We'll add a scaling variable to change the amount of vertical offset for our plane.
You can specify input variables in a jit.expr expression by typing in[N] , where N is the number of the inlet associated with your variable. The number of inlets that the jit.expr object has will vary according to the number of variables we specify. They appear to the right of the jit.expr object's left (matrix) inlet, and and numbered starting with 1.
Although the message expr snorm[0] snorm[1] (sin(snorm[0]*(in[1]*PI))+sin(snorm[1]*(in[2]*PI)))/2*in[3] may look a little daunting, you'll like what it does - the values for in[1] and in[2] are used as multipliers to PI, and allow us to set the number of sine functions that are arranged along the x and y axes of our data surface.
Similarly, in[3] multiplies the result of adding and averaging the two sine functions, and lets us change how flat or wavy our data surface is. And, of course, the cool part is that we can do all this on the fly by changing the input values in real time (Note: Since we need to re-render the surface each time we change a variable, we add a trigger object for each of the input variables.
Welcome to the world of jit.gen
While the jit.gen_2 patch is pretty cool to play with, I have a confession to make - for me, the message syntax got really complicated pretty quickly, (especially if you don't have an iron understanding chokehold on your algebraic order of expressions!). After numerous tries at typing that mangle of brackets and rearranging parentheses, I started to grumble about wanting a better and faster way to do all this.
Don't worry - the jit.gen_3 patch uses our new friend the jit.gen object to do everything we've just done in a way that's a lot easier to understand.
Here's the inside of the jit.gen patch that duplicates what we just created using those jit.expr expressions:
Wow. That does look a little simpler, doesn't it? Let's take a look at the patch.
You'll probably find the contents of the patch similar to the patching we did in our previous jit.pix tutorial - using normalized numbers to describe the positions of every element in our matrix (we're using the snorm operator in this patch because we want values in the range of -1.0 to 1.0 rather than 0. to 1.0), and the swiz and vec operators to "unpack" and "pack" up the vector data. What's interesting about this particular jit.gen object is that we're working with only three planes of data - x , y , and z coordinates. The jit.gen adapts to the dimensions and plane count of its input matrix and lets us swizzle out data for each coordinate by using numbered indices (e.g., swiz 0 gives us the x coordinate, swiz 1 gives us the y coordinate, etc.).
Our jit.gen patch includes three parameters, defined using the param operator: the x_iterations and y_iterations parameters specify how many iterations of the sine function we'll use (these are what we used the in[1] and in[2] variables in the message to jit.expr to do in the previous patch), and the scaling parameter is used to modify the amount of "lumpiness" in the surface (which is what we used the in[3] variable for in our previous patch).
NOTE: You may have noticed this convention previously in some of the gen example patches, but I thought I'd mention it here: When you add a named param operator to the inside of your gen patch, then you can use that name when you perform operations inside of the gen patch without having to connect the param object directly.
It's a handy thing to do, but some beginners find it confusing because it's not necessarily "Max-like." You may have encountered something similar when you first learned to work with the buffer~ object and the family of MSP objects that accessed the contents of a buffer by specifying the buffer name as an argument (such as info~ or groove~).
Since our jit.gen patch calculates the z plane vertical offsets of our data surface based on the x and y coordinates, the patch really doesn't take any input apart from the parameter messages - we use the swiz operator to provide us with x and y coordinate values across the range of our matrix (swiz 0 gives us the x coordinate, and swiz 1 provides the y coordinate). We use those values to calculate the sine function across the x and y dimensions, average the resulting values by adding and multiplying them by 0.5, and then multiply by the scaling factor. The vec operator packs up the x and y coordinate values along with the z value we've calculated, wraps them all into a three-element vector, and then outputs the result. It's quick, easy, and (at least for me) simple to see precisely what's going on. And that's why the jit.gen object is destined to become one of your best friends when working with matrices. While it's great to have Jitter gen objects like jit.pix and jit.gl.pix that are specially created to work with image data, the great thing about jit.gen is that it's the Swiss Army Knife gen object - you can use it for any kind of matrix you can imagine. In upcoming tutorials, we'll be using jit.gen to play in the fields of OpenGL.
A Codebox Interlude
While this tutorial has tended to be about moving away from writing the kind of expressions we'd use writing code, I'd be remiss at this point if I didn't take some time out and to show a little love to the gen codebox operator.
Here's an example of a jit.gen object that implements the previous example using the codebox operator.
Apart from a few renamed variables here and there, I think you'll recognize the part of the datasurface code that does the heavy lifting as bearing a strong resemblance to the expression we used in our jit.expr version.
What might be less apparent to you that the contents of our codebox operator defines its internal function (datasurface) before it does anything else such as specifying parameters, etc. That's a requirement for any codebox that contains a user-defined function.
This example of how to define a function, use GenExpr equivalents of operators such as swiz, and to interact with user-defined parameters is provided for the edification of those of you who, having seen the inside of the jit.gen object in the jit.gen_3 patch, felt the tug of nostalgia for the joy of writing code. Please enjoy this codebox example responsibly.
(Data) Surf's Up!
Now that we've seen how much easier it is to work with the jit.gen object for generating data surfaces, what can we do with the 'em? We'll end our tutorial with some sample patches to help you get started imagining some possible futures.
The jit.gen_throb patch provides what's probably the most immediate and rewarding use - using audio data to control the jit.gen object variables as a replacement for that all-too-predictable VJ effect we like to call The Throbbing Techno Donut.
Instead of using the amplitude of an audio file to make a giant torus change its size, this patch controls the number of sine function iterations in our plane and also to control the amount of z plane warping. Of course, you can do all kinds of things to make the patch more interesting (splitting the audio into frequency bands and using different bands to drive individual parameters), but isn't it great to have an alternative to that donut?
The pair of patches labeled jit.gen_skater and jit.gen_surfer involve processes a little closer to my own heart - using the data surfaces as the starting point for generating interesting kinds of control data. In a way, what these patches do is aking to what was at the heart of the Max for Live device I created in this tutorial, which generated a data surface and then created ways of taking a section of that data and using it to generate melodies. These patches are both a little more ambitious and a little more general, but they both make use the same data surface we created using the jit.gen object.
The jit.gen_skater patch adds a subpatcher called vector-skater to the contents of our the jit.gen_3 patch that lets us define a vector. The vector consists of
An x/y start position (with values in the range -1.0 to 1.0).
A vector whose direction and velocity is specified by positive or negative floating point values.
A boundmode that describes how a vector will act when it encounters the data range boundaries. You can choose to have the vector wrap or fold at the boundary for different outputs.
The output of the vector-skater subpatch varies and interacts in interesting ways according to the complexity of the data surface and the parameters for the vector itself. Think of it as a really interesting mutant cousin of your LFO.
The jit.gen_surfer patch is a somewhat eccentric variant of the jit.gen_skater patch. I don't want to spoil the fun by describing it in too much detail, but let's just say that it does things a little differently. Instead of shooting vectors across the data surface in the same way, I've added x and y offsets and folding/wrapping boundmodes to the insides of jit.gen object in order to sample a moving data surface. It's not only fun to watch in action, but the output of the much simpler vector-fetch subpatcher produces some subtle and interesting variations.
That's going to do it for this tutorial. Happy patching!
by Gregory Taylor on March 16, 2012
