Audible Flow - The audio equivalent of Optical Flow
Hi all,
I am curious to see if anyone has worked with the concept of 'Audible Flow'. I would like to analyze speed and direction (up or down) of the amplitude of n number of frequency bands. Then generate the realtime output based on the analysis: predicting the future!
This could probably be done with fft but may be very processor intensive.
Any thoughts on how you could do this would be great! Or better if anyone has done something similar.
Best,
Tim
My feeling is that FFT would work for this, or if you want to avoid this method you could use a bank of bandpass filters to split the frequencies of your source material. The standard fffb~ object would be well suited to this task.
I made a simple analysis program that compared the weighs of 4 bands which when properly tuned could identify bass, tom, snare hits.
best,
John.
Hi John,
That is pretty much how it needs to happen I think. fffb~ is great but are you aware of how you could do it with fft~ and grab say 256 bands of analysis and then generate 256 more bands and send it back into ifft~. In other words the same thing as I will do with the fffb~ but with fft~ as accurately as my little computer can process in realtime. Reconstruct a predicted sound at the highest resolution.
Best
Tim
Timothy Devine schrieb:
> This could probably be done with fft but may be very processor
> intensive.
What you're after is called envelope following, look into vectral~ and
framedelta~ within a pfft~ - should be perfect for it. Won't be too
hungry either...
Stefan
--
Stefan Tiedje------------x-------
--_____-----------|--------------
--(_|_ ----|-----|-----()-------
-- _|_)----|-----()--------------
----------()--------www.ccmix.com
Hey thanks guys. Much appreciated. I thought i was on the right track but needed some guidance from my fellow Maxers! Will let you know how i go.
Cheers,
Tim
Right.
I have been looking into various fft objects and patches including jitter matrixes. Though the process has been slow it has proven worthy. I now know exactly what I want to do. But not how to do it.
Adjust the amplitude and phase 'x' amount based on the direction of the previous 'n' number of frames.
For example if a pure tone of 600Hz is sent in and it increases constantly at 10Hz per second the system will adapt and if you look one second into the future the tone will be 10Hz higher then the present tone. If you look 2 seconds into the future then the tone will be 20Hz higher.
The way I imagine this happening is having a bank of fft bins and looking for movement up, down or constant and then translate this into an augmented signal for ifft.
I have attached a graph.
I thought vecrtal~ slide up 0.1 would work in some way similar to this idea but I couldn't hear or see it.
Best Wishes
Tim
[img]index.php?t=getfile&id=1004&private=0[/img]
Armchair speculation: So you want to find patterns between successive
fft frames that indicate moving frequencies. The differences in
instantaneous frequency between two spectra give you a dataset you
can analyze, but you need to account for phase wrapping - some kind
of 2D analysis is needed. Maybe some concoction of cepstral,
correlation and differentiation will do it?
On Nov 28, 2007, at 1:00 PM, Timothy Devine wrote:
> Right.
>
> I have been looking into various fft objects and patches including
> jitter matrixes. Though the process has been slow it has proven
> worthy. I now know exactly what I want to do. But not how to do it.
>
> Adjust the amplitude and phase 'x' amount based on the direction of
> the previous 'n' number of frames.
>
> For example if a pure tone of 600Hz is sent in and it increases
> constantly at 10Hz per second the system will adapt and if you look
> one second into the future the tone will be 10Hz higher then the
> present tone. If you look 2 seconds into the future then the tone
> will be 20Hz higher.
>
> The way I imagine this happening is having a bank of fft bins and
> looking for movement up, down or constant and then translate this
> into an augmented signal for ifft.
>
> I have attached a graph.
>
> I thought vecrtal~ slide up 0.1 would work in some way similar to
> this idea but I couldn't hear or see it.
>
> Best Wishes
>
> Tim
>
>
>
>
>
> [img]index.php?t=getfile&id=1004&private=0[/img]
Be seeing you
grrr waaa
www.grahamwakefield.net
Quote: lists@grahamwakefield wrote on Tue, 04 December 2007 19:43
----------------------------------------------------
> Armchair speculation: So you want to find patterns between successive
> fft frames that indicate moving frequencies. The differences in
> instantaneous frequency between two spectra give you a dataset you
> can analyze, but you need to account for phase wrapping - some kind
> of 2D analysis is needed. Maybe some concoction of cepstral,
> correlation and differentiation will do it?
>
That's kind of my thought. cepstrum, or "modulation spectrum" might be worthwhile. Neither of these have been heavily implemented in Max yet. There's a version that transforms into Mel bands -- look for Mel frequency cepstral coefficients -- that's popular in the speech community. I think Mels are a little suspect, but maybe some Bark decomposition would be worthwhile. Also look at John MacCallum's wavelet~ object on the CNMAT site.
mz
On Dec 5, 2007, at 5:15 AM, mzed wrote:
>
> That's kind of my thought. cepstrum, or "modulation spectrum" might
> be worthwhile. Neither of these have been heavily implemented in
> Max yet. There's a version that
There are some nice examples/tutorials in GABOR sub-library of FTM.