20Concepts Lesson 01 - Vizzie Part 1
(→Mix Movie Streams)
|Line 119:||Line 119:|
== Mix Movie Streams ==
== Mix Movie Streams ==
''This is a work in progress...''
''This is a work in progress...''
Revision as of 23:48, 24 January 2013
(Needed: Example Video)
In lesson #1, we use the Vizzie subsystem to allow people to quickly become familiar with the basic functionality of a visual programming language, and to learn some of the functionality of the Max environment. For this to be useful, a current version of Max - along with the Vizzie components (automatically installed with Max) - should be installed on the user's systems.
Introduction to Vizzie
Max takes a different approach to programming: it provides a visual, rather than textual, view of the code-creating process. There are some advantages to vusion programming:
- It engages visual-spacial interaction with the code.
- It combines the user-interface with the program logic.
- Sub-programs are easily - and visibly - integrated in new programs.
- It is often more approachable for non-programmers.
The basic workflow of Max programming (and other visual programming tools) is:
- Create objects
- Connect these objects to other objects
Visual programming works well with an iterative development process, where you create a small working section of code, then add new pieces as required. An important part of this is to create reusable subsystems; for example, it is typical to create a working Microsoft Kinect subsystem, then reuse it every time you need to work with a Kinect. These subsystems are built using Max itself, and are easily made available for future projects.
In this set of lessons, we use a subsystem called "Vizzie" (automatically installed with Max) that is used to manipulate visuals (movies, images and live video). This system is meant to provide an easy introduction to visual programming, and to provide a "fun" way to get started. There are some limits to the Vizzie subsystem:
- You are limited to the output video resolution (maximum of 640x480).
- It has limited means for saving state and reloading content.
- It is a little garish looking - but this is done on purpose, since it makes it easy for an instructor to quickly identify a program's logic.
Some people wonder if working with Vizzie is actually working with Max. We consider it to be so, since it works within the Max environment, can interface with lower-level Max code, it uses standard Max messaging, and the user and programming interfaces are the same.
Create a Basic Movie Player
We are going to start our programming experiments by creating a basic movie player. But first, we have to understand how to instantiate Vizzie objects. Vizzie objects aren't standard Max objects - rather, they are small, self-contained patches that we can include into our patches. There are two places to retrieve Vizzie objects:
- You can get them from the Vizzie search sections in the file browser (which is opened from the File menu in the Max application) and drag them into your patch. - You can paste them directly from the contextual menu (right-click/control-click in the patcher window, then select Paste From->VIZZIE-CLIPPINGS).
In either case, you are confronted with four named subsections - none of which are all that revealing (from a name standpoint). Right now, we are going to focus on the VIZZIE PVR section, where PVR stands for Players-Viewers-Recorders. This is the section of Vizzie modules that give us things like movie players, webcam input and windowed displays.
Using either selection method, create an instance of the PLAYR module. You'll notice that all of the modules are rather oddly named - they seem to have "action" names, but they never end in a proper "er". This is because a limitation of Max is that you cannot have multiple modules with the same name, and many people have already created modules named "Player", so by dropping the "e" we are able to avoid conflicts.
With a PLAYR modules in your patcher, we are ready to see how this operates. Drag a movie file (either from a file folder or from the "movies" section of the File Browser) onto the blank square that says "Drop a Movie Here". You will see the movie in the mini-screen, and it will automatically start playing. You've created your first functional patch! In order to learn more about the PLAYR module, let's make some adjustments to the controls.
If you try to move one of the controls, you will find that you just select the whole PLAYR module. This is because you are in Edit Mode, where you lay out the contents of the patch. In order to interact with the modules, you will need to 'lock the patch'. You can lock the patch by clicking on the little lock icon at the bottom left of the patcher, or you can use a keyboard shortcut (Control-E on Windows, Command-E on Mac). In either case, you will cause the patch to change into Performance Mode, and you can now move the controls on the PLAYR module.
The PLAYR has a number of controls, and many of them require different gestures to use them effectively. The speed/direction control determines how fast the video plays; when you move the slider to the left of center, it reverses the movie playback (because it is working at a "negative" speed). To the right of this control you find two push-buttons: one labeled "hi", and one with a flower icon in the center. Clicking on the flower will "refresh" the control to its default state, while switching on the "hi" control will cause the speed/direction slider to control a larger range of values - you can now increase the speed to 5x its normal speed.
Below the speed controls are the preview screen (which doubles as a drop zone for movie files), and the play/pause button for movie playback. Both of these are pretty easy to understand. One useful thing to note is that you can turn off the preview by clicking on the video display.
Below the preview screen, there is the playback/loop control. While it looks similar to a slider, it is actually a range control; you set the starting and ending points of the loop by clicking on a starting point in the range, then dragging the mouse to select an ending point. As with the speed control, you can reset the value by clicking on the connected refresh (flower) button.
If you want more information about any Vizzie module, you can hover your mouse cursor over the name of the module - it will give you a nice description. You can also hover over any control in order to get a description of that control.
Add a Movie Viewer to the Player
Now we have a working movie player, but the postage stamp size of it is kind of disappointing. This isn't going to be useful for a gallery installation, is it? In order to get a better view of the video content, we need to add another module - the VIEWR.
As with the PLAYR, you can find the VIEWR in the Vizzie PVR section of Max (either the File Browser or the contextual menu). Create an instance of a VIEWR module in an unlocked patch, and you will see that - in addition to showing up in our patch - it makes a new window that is just for our video. Move this new window out of the way, and let's continue editing the patch.
The PLAYR is continuing to play the video file, but we have to find some way to pipe it to our VIEWR window. This is done by patching the process of using a patch cord to connect from one module to another. You will notice that at the bottom of the PLAYR module is a little "nub"; this is an outlet, and is the output of data from a module. The top of the VIEWR has several similar nubs - these are inlets. This is an important concept: inlets are at the top, and outlets are at the bottom.
If you click on the left outlet of the PLAYR (which is labeled "video out"), then drag to the leftmost inlet of the VIEWR (which is labeled "video in"). When you get there, the connection will show a circle - you can release the mouse to complete the connection. Immediately, you should see the VIEWR window light up, and you have a new, more sizable, window for your video display.
One of the important things about the VIEWR is its ability to display video full-screen. If you click on the "Full Screen" button on the interface of the VIEWR module (after you've locked the patch, of course), the video will fill the entire display where the window is currently located. This is cool on your laptop screen, but it's even better if you move the display to a projector or second screen before you move into full-screen mode.
To leave full-screen mode, all you have to do is to hit the escape key.
Add Effects to the Movie Player
The next thing we will want to do is to add an effect to the Movie Player that we've created. From the contextual menu (or from the File Browser, if that is what you prefer), look at the contents of the VIZZIE EFX sub-folder. This is the location of most of the visual effects that we'll be using, and the heart of the fun in Vizzie.
Let's start with the ROTATR effect. Drop this into your patch - moving the PLAYR and VIEWR if necessary - so that it sits between the two previous modules. As we've learned so far, you activate modules by connecting them using patch cords. Click on the existing patch cord (the one that is connecting the PLAYR to the VIEWR) to select it. When selected, you will notice that the patch cord gets "handles" on it. If you grab one of the handles, you can detach it from its existing connection and move it to a new on. Let's start by dragging the handle connected to the PLAYR and reconnect it to the output of the ROTATR module. Then, create a new patch cord from the left outlet of the PLAYR and move it to the Video In of the ROTATR module. You'll see the ROTATR come to life with a slightly rotated version of the original video content.
Now you can move the controls of the ROTATR to change the rotation and anchor points of the video. You can also change the "mode" value to see what happens when the edge of one of the video displays is visible. You can clear the background, fold or wrap the video content, or even use the clip mode - which extends the last scan line of the edge into infinity. This one effects module provides an extensive amount of control in a single module package.
We can add a second effect to the stream; let's try the SLIDR module (again, from the VIZZIE EFX sub-folder). This slides pixel values from one frame to the next, and provides a smeary quality to everything that runs through it. Connect it between the ROTATR and VIEWR, creating an "effects chain" of two modules. Now, when you rotate the visuals in the ROTATR module, the result takes a moment to catch up, sliding the contents to their new values.
Again, play with the controls of the SLIDR module, and intuit what the "Slide Up" and "Slide Down" controls do. Additionally, you might want to try some of the presets (these are the little buttons underneath the preview display) to see what the developers came up with. EXERCISE: Using the ROTATR, SLIDR and other modules that you might find in the VIZZIE EFX sub-folder, create a combination of effects that make something interesting from a single video stream. Note: Some EFX modules, like the CHROMAKEYR, require two video inputs, so you might want to wait to use those for a bit!
Add Generators to Alter Effects
While using the controls of the Vizzie modules can give good results, part of what makes Max interesting is the development of generative (or self-modifying) systems. When done right, a generative system can surprise you with results you might never have produced through more traditional, thought-out means. The Vizzie modules set has a number of tools for creating generative systems - they are found in the VIZZIE GEN sub-folder.
Let start by creating an instance of the TWIDDLR module. You will immediately notice a few things:
- It does not have a display screen.
- It does have an on/off switch.
- It does not have any video input or output.
Without any of the trappings of video processing, we are obviously in new territory. What exactly does TWIDDLR (and any other Vizzie GEN module) do to help us create generative systems?
It generates numbers.
Now, it may not seems like it on the surface, but the Vizzie system is a heavily number-based system. We aren't confronted with numbers very often, but they are there, right below the user interface, controlling the rest of the system. What the GEN modules do is to great numbers that can be used to alter the controls of any module, simulating our movement of the sliders without requiring our intervention.
The use of GEN modules is based on the non-video inputs - the inlets that have small circles below them. If you hover your mouse over these circles (which are actually small buttons), you will get a description of the use of the inlet. If we connect the left outlet of TWIDDLR to an inlet (for example, the first input of a PLAYR, which will control the playback rate), nothing happens. This is because a generator is waiting to be run in order to generate numeric data. If you click on the checkbox (at the left side of the device), you will see the slider start to move - and you will also see that the playback rate of the PLAYR starts to move too! It is emulating your movement of the rate slider, but you don't even have to be in the room to make it go!
The TWIDDLR simple ramps from one value to another in linear fashion, and you can set the direction (up, down or up & down), ramp speed and min/max range, and see that the automation of the connected value will follow suit. You can connect the output of one GEN module to many different inputs, or have a large number of GEN modules controlling your entire video system. TWIDDLR and other VIZZIE GEN modules (along with the controlling inlets of the effects and PVR modules) give you the tools necessary to create a full-fledged generative video system.
Replace Movies with a Webcam
One of the fun things to do with Vizzie is to work with live video. Since most computers include a webcam (often built into the lid of a laptop), let's use a Vizzie module that can use its video stream as a Vizzie source.
The module that connects to your webcam is called the GRABBR, and is also found in the Vizzie PVR subfolder. The GRABBR module depends on a few things: you (obviously) have to have a camera hooked up to your computer. You should also have only one video source; if there is more than one, the GRABBR module will choose the first source available (often the first in a list of sources that is sorted alphabetically). Finally, the camera should not be compressed: the output of a CV camera is not a great option, because the compression (and subsequent decompression) can load your system significantly, and reduce the number of other effects that can be run in realtime.
Once you load a GRABBR module and verify that it is generating output, everything should work as expected. You can connect the Video Out to any other module's Video In and be working with live video with no other changes.
Mix Movie Streams
Now that we have the ability to have either a movie or a webcam as our video source, how about we do both? This sounds simple enough, but if we create two different video streams and plug them into a single Video In, the frames of the two video streams will "fight" for the attention of the connected module, and you will just end up with a glitchy mess. What we really need to do is to mix the two video streams together; to do this, we have to use the 4MIXR device, which is found in the Vizzie CTL subfolder.
This module allows us to mix up to four video sources together, and to control the way that it happens. First of all, let's connect our GRABBR and a PLAYR module up to Video In #1 and #2, and the output of the 4MIXR to our VIEWR module. With video coming in from both input modules, we can "turn up" the level on either of the videos to see a good result.
The menu at the bottom of the module provides us with some detailed options that are useful for doing different kinds of mixing. Unlike audio, where you just add two signals together, mixing video can take many forms. By default, the module uses the "ABSDIFF" mode, which does a different between the two videos to produce the mix. This may not be what you expect; if you change the mode to "+", you will probably get a more standard video mix. But playing with the variety of mixing modes will definitely inspire you with interesting mixed results.
This is a work in progress...
(Needed: text and links)
(Needed: text and exercises)
by Darwin Grosse and Cory Metcalf