Granular video
Hi !
As i like granular audio treatment, i've created a patch that makes realtime video granular treatment. It is a very simple patch, but i've never seen such treatment before, so i wanted to share this...
Enjoy !
hello sound cool but when I downloaded it there was no extension, i added maxpat but cant get it to open is it a maxpat?
thanks
jjr
it was built on Max 4.5, i'm not sure it works on PCs (made on a Mac)
you have to make sure to save the file with a .pat after it. I had to add .pat to make it work. I cleaned it up but its in max 5
that is so cool!!!!!
thanx for sharing!
i'm glad you like it !
And Axiom, thank you for the Max5 version, but the values are not visible in your patch and i think i doesn't make sense to have just sliders without values. And I thought i gave a different name to my patch...
And sorry for the others if i have forgotten to make the .pat extension visible.
I changed the name simply for clarity of function, this is technically a random video delay and not quite a grain function, but it could become one pretty easily, you would have to look at audio grains and how they work and see how it differs from what you have here, basically a video grain would be a windowed slice of linear played video, so you would have a matrixset that is being conted through in a linear way, and then a way to make an envelope for the alpha so that it will blend with the other video grains. I might try and make one in a bit.
In the original sense of an audio grain, it is the smallest bit of sound which can be perceived as such. The equivalent in visuals is simply a picture, it doesn't need windowing, as sonic and visual perception are fundamentally different.
A blending between video bits could be cool as well, if you plan to extend it to longer phrases. I do that in audio, but in the end I wouldn't call it granular anymore...
Stefan
I disagree, windowing on video elements with an alpha or blend envelope would allow for imagery to overlap, and usually when I use granular audio I have the option of setting the length of the grains, so in that sense a short sequence of frames down to 1 single frame from the granulized video would be like a grain. also typically in most of my granular audio experience you have the ability to set the number of active grains, which would allow for lots of images from surrounding areas of the video buffer to be overlaid on top of each other which would create a really nice temporal blurring.
My typical analogy with granular in visual is particles. I think that particles work very similar to grains and could easily be created with intimate analogies for every parameter.
Do you agree that this patch functions a bit more like a random buffer delay? Or do you think it is like granular? I agree that it is very similar to granular but there are a few distinct characteristics of granular that are missing.
wow, whatever you guys decide on calling it is fine with me, this is amazing. exactly the sort of random choppy 'granular' effect i was looking for. cheers...
> My typical analogy with granular in visual is particles. I think that
> particles work very similar to grains and could easily be created with
> intimate analogies for every parameter.
Yes, mixing the terms is problematic any way, it depends much on the perspective. If I look at my 3d representation of the world around me, there is no such thing as a picture. I see objects, and I'd agree that particles would be a good analogy.
> Do you agree that this patch functions a bit more like a random buffer delay?
Yes, of course
> Or do you think it is like granular?
also yes, as that is a common practice in live audio to have the grains just be jittered and thus delayed randomly.
> I agree that it is very similar to granular but there are a few
> distinct characteristics of granular that are missing.
Anyway, how to name a thing isn't really important, I love the result...
When I work with composers, they often want to do granular synthesis. Each and every of them had a very different idea about it. Non of the final results I would call granular, but for them it still is...
To have a clear nomenclature, I'd restrict the term to the way Curtis Roads introduced it, and not extend it to visual. But in this case the inspiration was granular sound synthesis and the result is inspiring in itself no matter how it would be called...
To explain it to someone who isn't able to see it, I'd probably also say its a random buffer delay...
Stefan
are you referring to microsound? I love that book, as well as computer music tutorial. I don't remember his definition of a grain, but are you saying that the definition says it cant be longer then a certain time? also I thought Xenakis was the first to come up with granular synthesis definition? Do I remember incorrectly?
have you guys seen this, I say a dvd of there work and it looked visually perfectly matched to the grains, and they seemed to be able to perform small changes over time
BaltimoreMAX wrote on Tue, 28 April 2009 10:43have you guys seen this, I say a dvd of there work and it looked visually perfectly matched to the grains, and they seemed to be able to perform small changes over time
now thats what im talking about!! i can't believe i hadn't heard of these guys before. what a head fu*k!! am trying to track down a dvd now. looks like it would be the perfect movie to put on at a recovery session
do any of you smart people think it would be possible to be capturing live sound with this patch? that would be really cool!
Axiom-Crux wrote on Tue, 28 April 2009 16:21are you referring to microsound? I love that book, as well as computer music tutorial. I don't remember his definition of a grain, but are you saying that the definition says it cant be longer then a certain time? also I thought Xenakis was the first to come up with granular synthesis definition? Do I remember incorrectly?
As often with inventions which are "in the air" its hard to tell. And I bet the first thoughts in that direction could well be coming from Xenakis.
Microsound is much more recent than granular synthesis, but its summarizing the basic thoughts. The idea is to cut an exiting sound into as small as possible bits, which could have their own identity, the smallest part a sound is composed of. If you make it too short, you don't hear any characteristic of the original if you play it back solo. If its too long, it is already composed of sound bits.
The area of time is around the time where we would distinguish between single events and a tone, the area where we distinguish between spectral properties and time properties. This is around the lowest frequency we can hear. about 50 ms or 20 Hz. This area is blurred of course...
I'd say anything longer than 100 ms/10 Hz I wouldn't call a grain anymore, and also anything shorter than 20 ms is questionable (or do you hear the 50 Hz buzz as a fast rhythm?).
The main problem in granular synthesis are fast transients. As we do hear their spectral content, and they could be very short. A single click of 1 ms is a grain if you include its surrounding silence....
Finally what we do with these grains once we have them, is only limited by imagination. In audio we need to blend them with some sort of windowing, as a rectangle window would create clicks and change the spectral content completely. In visuals I guess its not really changing the spectral content if we just switch between pictures. But our visual sense is translating non abstract pictures into a 3D representation of a bunch of objects. In the end the challenge is to deal with the latter instead of the 2D pictures...
Stefan
Bufferstroker wrote on Tue, 28 April 2009 20:01do any of you smart people think it would be possible to be capturing live sound with this patch? that would be really cool!
Why not? Delaying audio is way easier than delaying video...
your right....im just being lazy.
I've been wanting to get a copy of microsound for a while but everytime i got to borders they say they have to order it in. looks like amazon is my only hope in australia. once again my instant gratification urge while have to be tamed.
This is amazing. I'm looking to use this in a college project I'm doing. Do you know how I would go about making the audio grain with the video.
Hi everybody, would be great to test this patch... but I don't see the attachment anymore... can anyone re-up ?
Thanks
Hello all,
i would also like to try this patch, so please could someone who has access to it repost on the forum. Many thanks.
this is what I have and believe being the one you are looking for:
Thanks :)
@giorgio awesome patch! :)
Thought I'd chime in as I'm doing something super similar! Here's a video prototype of what I've been working with!
Video portion of the patch is similar to the one @giorgio posted with some tweaks. I have some different algorithms and features which is goign to be in the next documentation. I'm in the midst of doing experiments where it's all on the GPU. I have audio synced in real-time but I'm not using any granular methods!
I'll post the patch when I complete the project and it's cleaned up :)
can't access your patch or video, or giorio's original patch. would love to see as I've been interested in doing waveforms in RISE with granular effects such as these. Why not granular video????
Hi Jeff,
Not sure why you can't access Giorgio's patch. I've just done so now and it is fantastic. Did you click on the Pasted Max Patch button? If it didn't open for you maybe try another browser.
Paul
i believe I've got it under tiny tool so I've renamed. as a total newbie, how would I switch out the webcam for movies and change the 2d to 3d image?
Thanks so much
Also how audio responsive?
...for the records, the patch is not mine
but Wilfried Thierry's (tinytool) - 2009
g
Hi Jeff, just change the "open" message to "read" and the replace jit.grab with jit.movie. You can then use it with movies.
I'm not so hot with jitter myself so can't help much with your other queries. That said I'm pretty sure there are some tutorials here about audio/visual interaction
@g Thanks for the info!
Paul
This patch is more like a temporal granular effect. You can achieve more efficiently the same effect with the 'textureset-example.maxpat' from Max7 (basically a GL version of the jit.matrixset object). Granular Video Synthesis gets lot more interesting in the spatial domain: https://www.youtube.com/watch?v=HN1HaVOpAn8
Thanks for the heads up 2K. That looks great.
Paul