Creating video and audio sequences / loops for a live performance
I'm diving into Max to create audiovisual accompaniment for a live performance. I'm very new to Max, but I am a software engineer and have some rudimentary Pure Data experience.
I know what I need and at an abstract level, I know how to implement it, but I'm having a bit of trouble finding the right entry points via tutorials and documentation to make this concrete in Max.
I'd appreciate any pointer or advice.
My use case:
- A live band playing over an hour or so
- A projection of video and some sounds
- The projected material is "grouped" by song.
- Each group is manually triggered to start, and to stop, by a person running the audio visual (the "controller").
- Each song "group" has these attributes - a tempo, a collection of videos, and optionally, a collection of audio.
- Video (and Audio) would loop in sequence within a "group". Eg, if 5 videos are loaded, and each is 30 seconds, the videos will sequence 1, 2, 3, 4, 5, 1, 2, 3, 4, 5, 1, 2, 3 ... until the "group" is stopped.
- The "controller" would have a UI, being, for each group, a button to start and to stop the group sequence/loop
There are more details (eg effects on the video stream), but I can work those out once I get this basic patch for controlling the "groups" sorted.
Anyway thoughts would be helpful, thanks.
Hi Paul, your question is a bit too general to get answers because there are so many ways of doing what you are asking. As you are a software engineer, you might be interested in using the JS object (Javascript in Max) or an extension I created for Max called Scheme for Max, which puts a Scheme interpreter in Max. It's like a better JS object (if you share my opinions, lol). You can use these languages to orchestrate other Max components to do everything you describe.
I have a large live rig I'm building in Scheme that uses controllers similar to what you are saying. I find for that kind of stuff (as a programmer), being able to work in a text language that steers the ship is very productive. Happy to answer any questions if that piques your interest.
Well what you want to put together is possible, Max is a graphical programming language, you can build what ever you want. That said you can easily get overloaded with the amount of work that you have to do as a single person, which can easily make things not fun.
It sounds like you are trying to create some kind of conductor app, that integrates the synchronization of live performers, with audio and video. What makes Max so much fun is that it is an interactive performance environment, so I would want to approach the development in that way.
An interesting idea to explore would be how the performance environment could be a space that has awareness. For example having audio inputs that come from the performers that can be used to control video. Or having control signals sent to performers, like a clock, that they can use if they want.
I would recommend putting together a simple frame work that has all the parts you want. Not all of them may be implemented, but at the beginning you want to get the feel for how you will be conducting. From there you can come up with what sub components need to be implemented.
I would definitely implement this prototype with JavaScript as much as possible so that you are not having to patch everything together. When you find the right blend of patching and coding, project development and maintenance is easier. I would stay away from scheme or other esoteric max add on languages unless there is a good reason to use them.
Feel free to message me if you have questions, anthony.palomba@gmail.com
Thanks Anthony and Iain. I hadn’t actually noticed the js object and therefore the ability to write a large chunk of a patch in a textual programming language. That is a huge win for me. I’m going to take your advice Anthony and try to build the framework of what I want in a js object - I made a start and I already feel more in control :)
Iain Scheme for Max looks awesome - I’ve done quite a bit of tinkering in Clojure (including Overtone) and Emacs Lisp. I’m going to stick with js now as it is built in but I will definitely keep an eye on that.
Hi Paul, if you know Clojure you might want to check out some of what S4M offers over JS. I actually used JS extensively but after discovering Clojure (and then later other lisps) I wrote S4M to overcome the limitations of the JS object. The big one to know is that the JS object *only* runs in the main (a.k.a. UI) thread, so you can't accurately interact with the scheduler - you will always be waiting for the next pass of the UI thread which will introduce jitter. The other biggie is that s4m supports a repl and interactive loading like Clojure, but JS doesn't. So you can load code from messages or from various different files interactively, or even from your text editor over OSC like a network repl. If you're used to lisp workflows, the JS object is really clunky in comparison. It also uses a dead old JS version, ECMA5, and there has not been any indication of that changing anytime soon. Though Cycling/Ableton are known to release big surprises! :-)
I have some demos of all this on the youtube channel. https://youtube.com/c/musicwithlisp
Here's a clip of running a repl from from a vim session, evaluating functions from a hotkey in vim.
https://www.youtube.com/watch?v=Wxfboeb7Jmw&t=1s&ab_channel=MusicwithLisp
While I respect Anthony's work, I have to politely disagree on "stay away from Scheme".
I did a huge project similar to what you describe in JS and really felt like I hit the wall and dead ends in a bad way. JS is just a really poorly designed language and is woefully inappropriate for music. Lisp languages have a very big history in music programming for good reason - they are much nicer for interactive programming and representing music. (e.g. Common Music, Common Lisp Music, Overtone, Open Music, Patchworks, Bach Project, etc).
A case in point:
Max uses symbols extensively - everything you do (messages) is symbols and numbers. Max's fundamental unit of work is the message, which is a list of symbols and numbers. Lisp family languages are built around manipulating lists of symbols and numbers. JS has no concept of a symbol, so you have to do all your symbol manipulation with strings. But both Max and JS string handling is terrible. In Scheme, you have strings, symbols, and keywords (symbols that always evaluate to themselves) making interacting with symbols in Max so much more elegant.
I've done music hacking now in many languages, and I spent over 10 years earning my living slinging Python and (advanced) JS, and JS just feels like pushing a rock up a hill. but YMMV!
To play 5 audio or video clips using few buttons, being next or dedicated 5 is as trivial task as it can be
Most work is organising files and their relation,
what of that should call for js or any other
complicated approach ?
but I don‘t see much sense in having 5 video clips instead of a single one if they anyway loop in same order all the time
Source Audio is correct that this task doesn't need a text language at all - you could absolutely do it in the patcher. I recommended S4M because OP said "I am a software engineer" (as am I) and in my experience, if you are used to text programming, there are many things in Max that you might prefer to do in a hybrid of patcher and a text language.
I have done all of visual patching, JS, Node for Max, C extensions, and S4M code and find uses for them all. (Though these days the only use I have for the JS object is the JSUI - it is definitely the ticket for that).
Try to have a look at jit.gl.polymovie, I used it with great results in a complicated show together with mc.playlist. By the way it uses javascript.
Yeah, not seeing it either? I'm slow though so that maybe it but i think it shouldn't be a whole lot more than this honestly?
My suggestion was based on my read of the requirements as the kind of thing that is likely to evolve into a more complicated ongoing performance rig. I have something in that vein (though not video related) that I have been working on for years, and it has very sophisticated input controllers. I spent time trying out various approaches - patching, JS, Node, etc, and this time was well worth while. In the end, complex modal input controllers have turned out to be much easier to develope and maintain in a hybrid of patching, Scheme, and C. BUT - that is the case for a long lived project. if you are just wanting to get what you have listed done as fast as possible, the patcher is probably your best bet - go get the Cipriani and Giri books and make a big pot of coffee. lol.
If on the other hand this is likely to be the beginning of a long lived performance rig, I would argue that spending a month exploring all the options is time well spent. Also for the OP, if you are a C programmer (language was not specified in the original post about software engineering) I would strongy recommend learning the C SDK. When I finally did it, so much of Max made way more sense. It is, however, fairly advanced C, if pointers to pointers sounds like no fun, you probably won't like it. :-)
here's a simple way to sequence preloaded movies using jit.playlist (mostly cribbed from the help patch "notifications" tab)
some basic logic will allow you to repeat this sequence however you want.
Thanks everyone, I'm on my way.