geko

I want to analyse a sound in realtime and extract the frequencies with the loudest amplitude in order to establish a frequency-related access to playback and further editing. My approach is to record fft transform data in matrices and take additional reference matrices containing the extracted informations for each time slice of the fft providing a flexible access.

I dont see how to achieve this in MSP. I followed a hint in the forum to Ircams FTM. It seems to be well suited for my item even if this will take some time to become acquainted with it. Since I have not a long experience with MSP I want to be more sure about this approach and like to know if there are other recommendable solutions which match my aims as well.

I want to analyse a sound in realtime and extract the frequencies with the loudest amplitude in order to establish a frequency-related access to playback and further editing. My approach is to record fft transform data in matrices and take additional reference matrices containing the extracted informations for each time slice of the fft providing a flexible access. 

I dont see how to achieve this in MSP. I followed a hint in the forum to Ircams FTM. It seems to be well suited for my item even if this will take some time to become acquainted with it. Since I have not a long experience with MSP I want to be more sure about this approach and like to know if there are other recommendable solutions which match my aims as well.


peak-frequency-evaluation-of-stored-data

If you can figure out the fft~ or pfft~ and get the frequency bins that you want, you should be able to use jit.catch~ to store that signal (frequency) information into a matrix of appropriate size (maybe just one row, with # of columns equal to the # of bins, each column a value between 0. and 1. or scaled to = 0 to 255). Or you could have multiple rows, each representing a certain slice of time, or use multiple matrices. Possibly a different matrix for each slice of time would be more flexible if you want to do more matrix-related processing, but that's up to you (you can always use jit.spill to get at the lists individually).

Quite possibly I am missing something though, not an expert on frequency-domain stuff...

If you can figure out the fft~ or pfft~ and get the frequency bins that you want, you should be able to use jit.catch~ to store that signal (frequency) information into a matrix of appropriate size (maybe just one row, with # of columns equal to the # of bins, each column a value between 0. and 1. or scaled to  = 0 to 255). Or you could have multiple rows, each representing a certain slice of time, or use multiple matrices. Possibly a different matrix for each slice of time would be more flexible if you want to do more matrix-related processing, but that's up to you (you can always use jit.spill to get at the lists individually).

Quite possibly I am missing something though, not an expert on frequency-domain stuff...


thanks for your reply. it looks like I have to deal with jitter.

jitter could be helpful for all this, but two approaches without jitter might be:

1) multislider: -1. to 1., with as many sliders as bins. you can use [pattr] with your multislider, so there's one kind of flexible access (interpolation)...store slots for time slices, store banks of slots (the JSON pattrstorage files) for each sample. plus you can manually edit the bins in the multislider UI, that could sound interesting...

2) buffer~: you could use peek~ to write the bins into a buffer~ with sample length=the number of bins. the value range would be 0. to 1., so it wouldn't be "audio", just a storage place for the lists. Then use peek~ again to grab any single sample/bin. This way, you could utilize some of the MSP operations and FX on it before you access the bins, so sort of parallel to the jitter FX, but of course different results. And you'd have [waveform~] to mess with for drawing your own bins, you could do something fun with the selection/loop tools, etc.

Question for the fft~ people on here, would oscbank~ re-create (with simplification) the "original" sound by setting the frequencies to each of the bin frequencies, and setting the amplitudes to the data in your list?

anyway, jit.fill/spill on a 1-dimensional list can get you into and out of matrix land, so you could have access to the jit.FX if you want it. But plenty of other fun options for storing lists of lists without it, with the added UI benefits.

thank you for your hints. I will try with jitter and buffer~. I want to work with a quite high frequency resolution 2^15 e.g. so I would touch the limits of multislider soon.

I want to compare the actual evaluated fft data (frequencies with the loudest amplitude) with some predefined trigger values and grasping a certain time-point of recorded data by a frequency value as well. To achieve this a matrix seems to be most straightforward. Will take a little to work through some tuts.

thank you for your hints. I will try with jitter and buffer~. I want to work with a quite high frequency resolution 2^15 e.g. so I would touch the limits of multislider soon.
I want to compare the actual evaluated fft data (frequencies with the loudest amplitude) with some predefined trigger values and grasping a certain time-point of recorded data by a frequency value as well. To achieve this a matrix seems to be most straightforward. Will take a little to work through some tuts.


peak frequency evaluation of stored data