Forums > MaxMSP

very interesting video concerning FFT spectrum player

March 25, 2006 | 2:33 pm

http://www.youtube.com/p.swf?video_id=eRlhKaxcKpA&eurl=http% 3A//www.popmodernism.org/scrambledhackz/%3Fc%3D4&iurl=http%3 A//static16.youtube.com/vi/eRlhKaxcKpA/2.jpg

Application that links the sound from one medium to the spectrum
analysis of another; recall any sample in time as a 3D vector in time.

Similar techniques were discussed at a cycling lecture given in
Switzerland in 2005 by Puckett, so this may be of interest to folks here.
;)

Seb.


March 25, 2006 | 3:38 pm

The principle employed is very similar to what was used in the
European project Flying Cities, in which I was involved in 2003: Only
in this case the audio analysis and mapping was used to drive the 3D
animation of the waxing and waning of virtual space cities.

Check out the Flying Cities website at http://www.flyingcities.net.

Georg


March 25, 2006 | 8:00 pm

On the audio side, there’s a more inticate analysis and synthesis of
the sound from samples at
http://www.mat.ucsb.edu/~b.sturm/sand/VLDCMCaR/VLDCMCaR.html .

wes


March 27, 2006 | 5:45 pm

I visited the scrambledkackz site, that is pretty cool stuff.
I am curious to know how would one implement this is in Max.
Obviously the audio input would have to be loaded, analyzed,
and saved. Is there an external that allows you to save FFT
data to a file? What would be the best way to access this
potentially large data set: load it in a coll, or some DB?
If it were in a DB, what could I use to access it it?

Anthony


March 27, 2006 | 5:59 pm

Icidentaly there is an external out there called bonk~ ( by Puckette)
that does a similar thing. It allows you to trigger a sound with
the spectrum of another sound. But in this case one would need to
be able to handle a massive data set. I think bonk~ only uses RAM
to store FFT data.


March 27, 2006 | 6:06 pm

Jitter is very good for this sort of thing.
Check out Jitter Recipe 07:FFTCollector

http://scoop.cycling74.com/story/2006/2/6/114353/4118

for an idea of how this might be done. Depending on how long your
sample is, you may need a pretty large matrix. You can then save the
matrix as a jitter binary file using the write message.

Cheers,
Andrew B.


March 27, 2006 | 6:31 pm

The problem is, I do not have jitter. Another problem
is that I may be constrained by how much RAM I have.
What are some alternates methods to doing this Max/MSP?
I would need to redirect fft~ output to a file/db. What
kind of data does fft~ output? What format is this in?
If no RAM were to be used, I would need to manage data
storage/retrieval either to a flat file or a DB. Is there
any support for that in Max?

Anthony


March 27, 2006 | 8:30 pm

I can see from fft~ help that it outputs and audio signal
called real output and imaginary output. What can I use
to convert this data into something I can save? I want to
later be able to compare an incoming FFT~ data point with
a saved one. Any ideas?

Anthony


March 27, 2006 | 11:39 pm

finally a reason to get it!


March 28, 2006 | 4:46 am


March 28, 2006 | 10:03 am

apalomba@austin.rr.com wrote:
> I can see from fft~ help that it outputs and audio signal
> called real output and imaginary output. What can I use
> to convert this data into something I can save? I want to
> later be able to compare an incoming FFT~ data point with
> a saved one. Any ideas?

sfrecord~, record~, peek~ into a buffer~ and save…

Stefan

[][] [][][] [][] [][][]
[][][][][][][][][][][][][][][]

Stefan Tiedje
Klanggestalter
Electronic Composition
&
Improvisation

/~~~~~
\ /|() ()|
))))) )| | |( \
/// _/)/ )))))
___/ ///

————————-x—-
–_____———–|———–
–(_|_ —-|—–|—–()—-
– _|_)—-|—–()———–
———-()————x—–

14, Av. Pr. Franklin Roosevelt,
94320 Thiais, France
Phone at CCMIX +33-1-57 42 91 09


March 28, 2006 | 5:18 pm

apalomba@austin.rr.com wrote:
> What kind of data does fft~ output? What format is this in?

You can save it as a sound file, but I guess you want to make an
interpretation of an analysis and store that, and thats up to you.

I’d like to have fiddle~/bonk~ versions for use inside a pfft~, does
somebody tried that already?

> If no RAM were to be used, I would need to manage data
> storage/retrieval either to a flat file or a DB. Is there any support
> for that in Max?

There is a mysql external should be listed on Maxobjects.

Stefan

[][] [][][] [][] [][][]
[][][][][][][][][][][][][][][]

Stefan Tiedje
Klanggestalter
Electronic Composition
&
Improvisation

/~~~~~
\ /|() ()|
))))) )| | |( \
/// _/)/ )))))
___/ ///

————————-x—-
–_____———–|———–
–(_|_ —-|—–|—–()—-
– _|_)—-|—–()———–
———-()————x—–

14, Av. Pr. Franklin Roosevelt,
94320 Thiais, France
Phone at CCMIX +33-1-57 42 91 09


March 28, 2006 | 5:42 pm

On Mar 28, 2006, at 2:03 AM, Stefan Tiedje wrote:

> apalomba@austin.rr.com wrote:
>> I can see from fft~ help that it outputs and audio signal
>> called real output and imaginary output. What can I use
>> to convert this data into something I can save? I want to later be
>> able to compare an incoming FFT~ data point with a saved one. Any
>> ideas?
>
> sfrecord~, record~, peek~ into a buffer~ and save…

FWIW, in this case, be sure to write the saved FFT frame sound file
as float32 data to adequately store the spectrum. The default 16 bit
integer representation of soundfiles isn’t enough to store without loss.

-Joshua


March 28, 2006 | 10:07 pm

Do I have to go through sfrecord to do this? I was
hoping to take the real output from fft~ and save it
directly to disk. Using buffer~ would require me to
allocate a buffer that is the size of the output file.
If the sound file I am processing is 5 minutes long
that is a huge buffer (unless I write out chunks at
a time). All I really need to do is write each FFT
vaule out to disk. That way I do not have to remove
a sound file header either. Is there a "fileout" object?

Another question I have is, what is the actual output
of FFT~? If my window size is 512, I should get 512
floating point values representing the spectral content.
Yes? I could then process this file more off line. Perhaps
order the FFT windows to make it easier to search.

THis is what I have so far…

#P window setfont "Sans Serif" 9.;
#P user number~ 294 302 333 317 9 139 3 2 0. 0. 0 0. 50 0. 0 0 0 221
221 221 222 222 222 0 0 0;
#P window linecount 1;
#P message 331 147 49 9109513 open raw;
#P newex 331 211 89 9109513 prepend samptype;
#P user umenu 331 189 80 9109543 1 64 205 0;
#X add int8;
#X add int16;
#X add int24;
#X add int32;
#X add float32;
#X add float64;
#X add mulaw;
#X add alaw;
#P message 331 168 55 9109513 open wave;
#P message 331 125 65 9109513 record 1000;
#P user number~ 131 108 170 123 9 139 3 2 0. 0. 0 0. 50 0. 0 0 0 221
221 221 222 222 222 0 0 0;
#P toggle 58 239 15 0;
#P window linecount 2;
#P comment 330 95 66 9109513 � start/stop recording;
#P user number~ 92 302 131 317 9 139 3 2 0. 0. 0 0. 50 0. 0 0 0 221 221
221 222 222 222 0 0 0;
#P user ezdac~ 177 333 221 366 0;
#P window linecount 1;
#P newex 172 44 65 9109513 phasor~ 880;
#P newex 172 76 68 9109513 fft~ 512 512 0;
#P newex 151 22 114 9109513 buffer~ specbuffer 1000;
#P newex 264 331 82 9109513 peek~ specbuffer;
#P newex 91 272 88 9109513 record~ specbuffer;
#P newex 293 272 48 9109513 sfrecord~;
#P hidden fasten 12 0 0 0 336 187 324 187 324 247 298 247;
#P fasten 4 0 1 0 177 133 96 133;
#P fasten 4 0 10 0 177 100 136 100;
#P fasten 4 0 0 0 177 228 298 228;
#P connect 0 0 16 0;
#P hidden fasten 15 0 0 0 336 165 324 165 324 248 298 248;
#P hidden fasten 14 0 0 0 336 256 298 256;
#P fasten 13 1 14 0 406 207 336 207;
#P hidden fasten 11 0 0 0 336 145 298 145;
#P connect 5 0 4 0;
#P connect 1 0 7 0;
#P fasten 9 0 1 0 63 259 96 259;
#P window clipboard copycount 17;


March 30, 2006 | 9:15 am

apalomba@austin.rr.com wrote:
> Do I have to go through sfrecord to do this? I was
> hoping to take the real output from fft~ and save it
> directly to disk.

Thats what sfrecord~ is doing. But you need to know exactly what you’re
doing to not scramble the bins…

> Using buffer~ would require me to
> allocate a buffer that is the size of the output file.
> If the sound file I am processing is 5 minutes long
> that is a huge buffer (unless I write out chunks at
> a time).

Nowadays its not huge, its big maybe, but you could easily manage more
than that if you give your computer enough ram.

> All I really need to do is write each FFT
> vaule out to disk. That way I do not have to remove
> a sound file header either. Is there a "fileout" object?

If you read in a soundfile, you don’t need to worry about headers.

> Another question I have is, what is the actual output
> of FFT~? If my window size is 512, I should get 512
> floating point values representing the spectral content.
> Yes? I could then process this file more off line. Perhaps
> order the FFT windows to make it easier to search.

You should look into pfft~ which does all the windowing stuff for you
and gets rid of the duplicated bins (which do not carry additional
information, but need space)
And consider using Java with mxj~ within pfft~.
Its tough to get into that stuff, but its worth it…

Stefan

[][] [][][] [][] [][][]
[][][][][][][][][][][][][][][]

Stefan Tiedje
Klanggestalter
Electronic Composition
&
Improvisation

/~~~~~
\ /|() ()|
))))) )| | |( \
/// _/)/ )))))
___/ ///

————————-x—-
–_____———–|———–
–(_|_ —-|—–|—–()—-
– _|_)—-|—–()———–
———-()————x—–

14, Av. Pr. Franklin Roosevelt,
94320 Thiais, France
Phone at CCMIX +33-1-57 42 91 09


March 30, 2006 | 5:47 pm

Thanks for your feedback Stefan. I know Sfrecord~ saves things
to disk, I guess I am not used to thinking that sfrecord~ could
handle FFT data. I will give pfft~ a look as well.

Ultimately though, for this to work, I will need to be able to
do the following…

1. Process source soundfile and save FFT data.
2. Load source FFT data, run FFT on incoming signal
3. Search source FFTs for closest matching source FFT

I could load the FFT file in to buffer~ and use index~ to
access it. Searching would be linear which might be a problem.
It would be cool to post process the FFT sound file
so that the FFT windows were ordered, that way one could do a binary
search.

Anthony


March 30, 2006 | 7:04 pm


March 30, 2006 | 7:24 pm

Thanks for your advice, but again I do not have jitter.
When I save enough money to buy it I will give your
idea a try. Until then I will have to do this with
max/msp alone.

I want to be able to compare one FFT frame with another
and find the closest match. That means that I may
have to come up with some clever way to compare frames.
Maybe find some kind of equation that will take a frame
and compute a value that I could then compare.

Anthony


March 30, 2006 | 7:36 pm

Just use FTM for that :

http://recherche.ircam.fr/equipes/temps-reel/ftm/index.html


March 30, 2006 | 7:53 pm

I have seen FTM before but have not had much time to delve in
to it. Could you give me some pointers on what parts of FTM
I should look at?

Anthony


September 6, 2007 | 2:48 pm

Hello,
did somebody went deeper in that project?
i seen the video too, read about the concept, had a look on jitter example.. so i understand the concept, but don’t really sucess in patching the idea..
if somebody have help to give, info or patches ..it will be great

thanks

freeka


September 6, 2007 | 4:05 pm

Check out MATConcat…

http://www.mat.ucsb.edu/~b.sturm/research.html

It is still on me list of things to evaluate,
but it looks promising.

Anthony

—– Original Message —–
From: freeka
Date: Thursday, September 6, 2007 9:52 am
Subject: [maxmsp] Re: Spectral Recontextualization (was very
interesting video…)

>
> Hello,
> did somebody went deeper in that project?
> i seen the video too, read about the concept, had a look on jitter
> example.. so i understand the concept, but don’t really sucess in
> patching the idea..
> if somebody have help to give, info or patches ..it will be great
>
> thanks
>
> freeka
>


September 6, 2007 | 9:10 pm

Quote: Anthony Palomba wrote on Tue, 28 March 2006 15:07
—————————————————-
> Do I have to go through sfrecord to do this? I was
> hoping to take the real output from fft~ and save it
> directly to disk. Using buffer~ would require me to
> allocate a buffer that is the size of the output file.
> If the sound file I am processing is 5 minutes long
> that is a huge buffer (unless I write out chunks at
> a time). All I really need to do is write each FFT
> vaule out to disk.

all you have to do? :) for 5 minutes stereo, thats 52,920,000
numbers – you will need thousands of coll objects to store that data.

for mono "fft data" its a nice way to simply write a stereo file from within pfft, each signal in a channel. and if you have enough RAM in your hardware, buffer~ should be the easiest way.

dont forget that you have have to find a way of recording triggered by a signal, because you have to start recording when a new window begins. (which is why i suggested pfft above.)


September 7, 2007 | 7:51 am

thanks!!

any example patches??

f


September 7, 2007 | 7:54 am

Quote: freeka wrote on Fri, 07 September 2007 01:51
—————————————————-
> thanks!!
>
> any example patches using or not jitter??
>
>
> f
—————————————————-


Viewing 25 posts - 1 through 25 (of 25 total)