thetechnobear

I've been messing about with fft's in rnbo, and have a couple of specific and general questions which hopefully something with more knowledge can help me on.

basically, I just want to extract the top frequencies and amplitudes

so I basically have this fft~ going into cartopol~ then into a codebox

Screenshot 2022-12-14 at 13.25.21.png

ok, so I started with ONE fft~ (so ignore the other for now;) ) , and basically Im using the amp from car-cartopol, and the phase from fft~ to give me the bin.

Q1) what does cartopol~ phase/theta represent?

I assumed this would also be the bin of the fft... but represented as an angle 

so in above bin would be something like bin = (theta/pi) * 512 + 512

(as phase output seems to be, surprisingly, bipolar, -pi to +pi?!)

but I expected this to match ~fft phase.. but it doesn't!?

(plotting it doesn't really reveal much about what it represents , at least to me)

the next part.... is I wanted to do two overlapping ffts to improve the outputs 

I looked up overlapping ffts, I could see the approach is to use a window to overlap the results. I was going to put in place a hanning window, but then noticed fft~ has a window attribute, which defaults hamming... this is a bit odd, since the fft~ in msp, does not have this.

does it mean we dont need to window, as we would do in msp?

so, Im wondering, does the fft~ in rnbo always apply a window, so what this step is unnecessary when creating your own overlapping ffts - its seems like a reasonable approach.

(eliminates an extra step, though is perhaps a little odd when using one fft?)

Q3) how do I combine the two fft in my use case?

this is a noob question... my assumption is... I can just add the two amplitudes together... and then divide by two, so average between them.

Q4) efficient way to track top N frequencies (and their amplitudes)?

I have pretty good way with one fft, where I use start n' end of phase, so I can keep a 'running track' of best N frequencies. however this wont work with multiple ffts , as there is no start n' end.

seems, Id have to keep track of all the bins, but then Id have to continually search all the bins(1024) for the top N. which is a pretty intensive operation. (as obviously its all unsorted)

there seems to also be very limited data structures (just lists?) in rnbo, and codebox... so not sure the best way to handle it...

.... well the only way I can think of, is actually to just make RNBO publish the freq/amps , and then write some C++ code to do the heavy lifting of tracking the top N frequencies.

this is ok, Im custom coding anyway - but a bit of a pity, since I was trying to demo how RNBO code be used to solve problems... so pity to then hit such a roadblock.

can we please have pfft~ in RNBO... this would seem useful to have so we can interface with gen~?

anyway, any pointers would be greatly appreciated.

basically, I just want to extract the top frequencies and amplitudes 

ok, so I started with ONE fft~ (so ignore the other for now;) )  , and basically Im using the amp from car-cartopol, and the phase from fft~ to give me the bin.
that seems to work nicely.

first question : ) 

Q1)  what does cartopol~ phase/theta represent?
I assumed this would also be the bin of the fft... but represented as an angle 
so in above bin would be something like   bin = (theta/pi) * 512 + 512
(as phase output seems to be, surprisingly,  bipolar, -pi to +pi?!) 

but I expected this to match ~fft phase.. but it doesn't!?

(plotting it doesn't really reveal much about what it represents , at least to me) 

the next part.... is I wanted to do two overlapping ffts to improve the outputs 

I looked up overlapping ffts, I could see the approach is to use a window to overlap the results. I was going to put in place a hanning window, but then noticed fft~ has a window attribute, which defaults hamming...  this is a bit odd, since the fft~ in msp, does not have this.

Q2) what does window do for rnbo fft~
does it mean we dont need to window, as we would do in msp?

so, Im wondering, does the fft~ in rnbo always apply a window, so what this step is unnecessary when creating your own overlapping ffts - its seems like a reasonable approach.
(eliminates an extra step, though is perhaps a little odd when using one fft?)

Q3) how do I combine the two fft in my use case?
this is a noob question... my assumption is... I can just add the two amplitudes together... and then divide by two, so average between them.
correct?

I have pretty good way with one fft, where I use start n' end of phase, so I can keep a 'running track' of best N frequencies. however this wont work with multiple ffts , as there is no start n' end.
seems, Id have to keep track of all the bins,  but then Id have to continually search all the bins(1024) for the top N. which is a pretty intensive operation. (as obviously its all unsorted)
there seems to also be very limited data structures (just lists?) in rnbo, and codebox... so not sure the best way to handle it...

.... well the only way I can think of, is actually to just make RNBO publish the freq/amps , and then write some C++ code to do the heavy lifting of tracking the top N frequencies.

this is ok, Im custom coding anyway -  but a bit of a pity, since I was trying to demo how RNBO code be used to solve problems... so pity to then hit such a roadblock.

Q5) final question for C74...  :) 
can we please have pfft~ in RNBO... this would seem useful to have so we can interface with gen~?

I've been messing about with fft's in rnbo, and have a couple of specific and general questions which hopefully something with more ... 

fft~-in-rnbo

The output of fft~ (and ifft~) gives you the real part, imaginary part, and bin number of an FFT calculation. The bin number is perhaps confusingly called phase by fft~ because that's how it's referred to in MSP and also because it's the "phase" of reading through a group of complex numbers (since FFT works on chunks/buffers of data). So for fft~: the phase outlet is the bin number.

For cartopol~: fft~ is giving you the real and imaginary part of a number for a specific bin. In order to get the magnitude of a complex number, cartopol~ converts from a complex number's real and imaginary parts into a magnitude and phase. 

https://en.wikipedia.org/wiki/Complex_number#Polar_complex_plane

The phase output of cartopol~ is saying how out of phase a spectral component is, which doesn't matter if you're just interested in how much of a spectral bin is present.

In short -- to get the frequency bin, use the third outlet of fft~. To get the amplitude of the corresponding bin, use the first outlet of cartopol~.

If you set `@window rectangular`, the outputs should be the same:

image.png

(with a Hann window, for example, the outputs will be different)

The defaults for fft~ in RNBO should have been rectangular (AKA no window) in order to behave by default like fft~ in MSP. This will be fixed in an upcoming version of RNBO.

Correct! It's simply a convenience that this is available for RNBO fft~ so you don't need to worry about indexing and windowing manually.

Can you explain what you mean by there isn't a start and end with multiple FFTs?

Agreed this would be useful! Request noted. :)

what does cartopol~ phase/theta represent?

For cartopol~: fft~ is giving you the real and imaginary part of a number for a specific bin. In order to get the magnitude of a complex number, cartopol~ converts from a complex number's real and imaginary parts into a magnitude and phase.  https://en.wikipedia.org/wiki/Complex_number#Polar_complex_plane 

I expected this to match ~fft phase.. but it doesn't!?

(with a Hann window, for example, the outputs will be different)


Q2) what does window do for rnbo fft~
does it mean we dont need to window, as we would do in msp?

Correct! It's simply a convenience that this is available for RNBO fft~ so you don't need to worry about indexing and windowing manually. 

Q4) efficient way to track top N frequencies (and their amplitudes)? I have pretty good way with one fft, where I use start n' end of phase, so I can keep a 'running track' of best N frequencies. however this wont work with multiple ffts , as there is no start n' end.
seems, Id have to keep track of all the bins,  but then Id have to continually search all the bins(1024) for the top N. which is a pretty intensive operation. (as obviously its all unsorted)

thank you for you fantastic reply... really cleared up so much...

yeah, I did wonder why rectangle had not been chosen as a default for fft~

"cartopol~ is saying how out of phase a spectral component is" 

hmm, thats I think the part I need to dig into ( I get the complex vs cartesian representation) , but when you say its out of phase, with respect to what?

If I have a sine wave, its only a fundamental .. so one bin = in phase with respect to signal... but once we get harmonics (from other waveforms), I guess those could be out of phase with the main harmonic?

start n' end ... so this is kind of an 'implementation' I did ....

the way fft~ is presenting the harmonics, is in order first to last bin, so thats the start n' end.

so I was collecting the top N frequencies, and then outputting them in my code, once I say the last bin.

this meant I didn't need to store all bins, and then search the all the bins (a 1024 vector) 

I could basically do the work... as each bin came in.

however, once we have overlapping fft's, each has its own start/end (by definition).

the brute force/unsubtle solution is to keep both sets of bins, sum and average them, and every frame search for the top N.... but that is pretty cpu intensive. (esp. compared to my single fft solution).

I could do things a bit more efficiently by have sorted (by amp) vectors, containing both freq/bin and amp.

but RNBO has only appears to have fairly basic list support, no other data structures... so that means we cannot store pairs (bin/amp) , and Im also not sure that the slicing/accessing of lists will be efficient enough to be used as a basis for other data structures (e.g.RB trees)

but as I say, its kind of an implementation thing... and perhaps not surprising that rnbo doesn't really support.

however, I guess the higher level thing is... 

I think its not uncommon to want from fft, to know the frequencies that have most energy, and its surprising that is not easier to find (efficiently) 

thank you for you fantastic reply... really cleared up so much... 

"cartopol~ is saying how out of phase a spectral component is" 
hmm, thats I think the part I need to dig into ( I get the complex vs cartesian representation) , but when you say its out of phase, with respect to what?

If I have a sine wave, its only a fundamental .. so one bin = in phase with respect to signal... but once we get harmonics  (from other waveforms), I guess those could be out of phase with the main harmonic?

start n' end ... so this is kind of an 'implementation' I did ....
the way fft~ is presenting the harmonics, is in order first to last bin, so thats the start n' end.
so I was collecting the top N frequencies, and then outputting them in my code, once I say the last bin.
this meant I didn't need to store all bins, and then search the all the bins (a 1024  vector) 
I could basically do the work... as each bin came in.

however, once we have overlapping fft's, each has its own start/end (by definition).
so that technique won't really work.

the brute force/unsubtle  solution is to keep both sets of bins, sum and average them, and every frame search for the top N.... but that is pretty cpu intensive. (esp. compared to my single fft solution).
I could do things a bit more efficiently by have sorted (by amp) vectors, containing both freq/bin and amp.

but RNBO has only appears to have fairly basic list support, no other data structures... so that means we cannot store pairs (bin/amp) , and Im also not sure that the slicing/accessing of lists will be efficient enough to be used as a basis for other data structures (e.g.RB trees) 

however, I guess the higher level thing is... 
I think its not uncommon to want from fft, to know the frequencies that have most energy, and its surprising that is not easier to find (efficiently) 
 

oh nice. didn't expect that might be possible to include someday. +1 from me in that case :D

you can put a simple >~ at the magnitude outlet of cartopol~ to detect bins that pass beyond a certain threshold(don't look for an 

 N number if you want to be as efficient as possible... but i don't mean to say it's not possible nor desirable to do that depending on context, either).

i'm reading some of your description and feeling like maybe these aren't 'RNBO'-specific questions(but not that it's a problem, just mentioning: you want to learn step-by-step, and diving into FFT straight away within RNBO might not be best). for example...

what do the words 'top' and 'best' mean to you? -> this could mean the highest in numerical order of bins? 'best' is even more subjective a word than 'top'? or(only because i read your last sentence talking about 'know the frequencies that have most energy') i'm also thinking it could also mean something like 'top-most energy'? (in which case, 'bins with the highest magnitude/energy' is a better description)

the semantics used here are important and lead me to believe it might be better to go over the basics first:

https://cycling74.com/tutorials/advanced-max-ffts-part-1

(and then, if that still feels sparse, the phase-vocoder tutorials on this site 

as well as the book, "Musimathics Vol. 2", can really boost understanding of FFT... there are also other avenues(i took a course on Coursera once that explained FFT within the context of python scripting as well - that's not to say i wasn't entirely confused most of the time, haha))

out of phase with respect to the center-frequency of the particular bin you're analyzing. this is covered nicely in the video from the last tutorial seven of the series linked above(this is also a tiny secret of FFT: you're not limited to just the center-frequencies of bins, you can find 'instantaneous frequencies' beyond the resolution/number of bins but limited to a certain degree of accuracy by that same resolution).

hope it helps(i don't consider myself an expert on this subject yet, either, so anyone should feel free to correct me if anything i've written seems mistaken here(not that i'd ever consider myself an expert on 

 subject, either, hahaha ...but i digress...)) 🍻

I think its not uncommon to want from fft, to know the frequencies that  have most energy, and its surprising that is not easier to find  (efficiently)

you can put a simple >~ at the magnitude outlet of cartopol~ to detect bins that pass beyond a certain threshold(don't look for an exact N number if you want to be as efficient as possible... but i don't mean to say it's not possible nor desirable to do that depending on context, either). 

so I can keep a 'running track' of best N frequencies...
I was collecting the top N frequencies...

i'm reading some of your description and feeling like maybe these aren't 'RNBO'-specific questions(but not that it's a problem, just mentioning: you want to learn step-by-step, and diving into FFT straight away within RNBO might not be best). for example...
what do the words 'top' and 'best' mean to you? -> this could mean the highest in numerical order of bins? 'best' is even more subjective a word than 'top'? or(only because i read your last sentence talking about 'know the frequencies that have most energy') i'm also thinking it could also mean something like 'top-most energy'? (in which case, 'bins with the highest magnitude/energy' is a better description)
the semantics used here are important and lead me to believe it might be better to go over the basics first:
https://cycling74.com/tutorials/advanced-max-ffts-part-1 
(and then, if that still feels sparse, the phase-vocoder tutorials on this site 1 and 2,
as well as the book, "Musimathics Vol. 2", can really boost understanding of FFT... there are also other avenues(i took a course on Coursera once that explained FFT within the context of python scripting as well - that's not to say i wasn't entirely confused most of the time, haha))

when you say its out of phase, with respect to what?

hope it helps(i don't consider myself an expert on this subject yet, either, so anyone should feel free to correct me if anything i've written seems mistaken here(not that i'd ever consider myself an expert on any subject, either, hahaha ...but i digress...)) 🍻

yeah, Im pretty clear on the basics of fft....

I think I said earlier I was talking, top as in highest energy/magnitude.

phase - interesting, I guess as a bin is a collection of a set of frequencies, this is therefore a 'skew' of that bin... that makes sense.

e.g. bin represents 400-480hz, it might still be skewed to 470hz at any particular time.

the thing Im mostly lacking is how to do this in max/rnbo... I can do it reasonably easily in C++. rather the issue is how rnbo (fft~/cartopol~) are providing the data, and then the facilities (particularly data structures) available in RNBO. e.g. I can easily store the data, and brute force search it, but thats going to be very inefficient, and difficult to do in constant time.

... something thats not hard at all, if I were doing in C++.

none, of this is really an issue ... partly what Im writing my own C++ vs RNBO generated code is arbitrary.

I'm encapsulating the RNBO generated code into my own 'host' anyway. so partly, this is still exploring where this boundary lies.

so its a valuable lesson, it feels here its better to get the raw data to my c++ layer early, then process it there... as Ive kind of reached the boundary of where codebox~ is useful.

---------------------------------------------

(*) my main purpose for RNBO is to use it as a kind of 'prototyping' environment. I do as much of the dsp within RNBO in early development, to allow me to 'play with ideas', which is a bit slower in C++. then as ideas solidify, so more of the code becomes bespoke c++, until finally, (potentially) all RNBO code is removed.

so its nice to do as much as possible in RNBO, but its not really a big deal, since as above, at some point, often the code will be re-written in C++ anyway.

yeah, Im pretty clear on the basics of fft....
I think I said earlier I was talking, top as in highest energy/magnitude. 

phase - interesting, I guess as a bin is a collection of a set of frequencies, this is therefore a 'skew' of that bin... that makes sense.
e.g.  bin represents 400-480hz, it might still be skewed to 470hz at any particular time.


the thing Im mostly lacking is how to do this in max/rnbo... I can do it reasonably easily in C++. rather the issue is how rnbo (fft~/cartopol~) are providing the data, and then the facilities (particularly data structures) available in RNBO. e.g. I can easily store the data, and brute force search it, but thats going to be very inefficient, and difficult to do in constant time.
... something thats not hard at all, if I were doing in C++.

none, of this is really an issue ... partly what Im writing my own  C++ vs RNBO generated code is arbitrary.
I'm encapsulating the RNBO generated code into my own 'host' anyway.   so partly, this is still exploring where this boundary lies.

so its a valuable lesson, it feels here its better to get the raw data to my c++ layer early, then process it there... as Ive kind of reached the boundary of where codebox~ is useful.

thanks for the help,  much appreciated

---------------------------------------------
(*) my main purpose for RNBO is to use it as a kind of 'prototyping' environment. I do as much of the dsp within RNBO in early development, to allow me to 'play with ideas', which is a bit slower in C++. then as ideas solidify, so more of the code becomes bespoke c++, until finally, (potentially) all RNBO code is removed.

so its nice to do as much as possible in RNBO, but its not really a big deal, since as above, at some point, often the code will be re-written in C++ anyway.


ah, i understand you now(you're studying the C++ output for comparison... a great idea).

(*) my main purpose for RNBO is to use it as a kind of 'prototyping' environment

ah, i understand you now(you're studying the C++ output for comparison... a great idea).
my fault for answering prematurely :)


no, thank you, your input was very useful.

tbh, I tend to use Max a bit sporadically, things like RNBO pull me back occasionally, so not always clear on what's possible - though, generally, I really like where they are going, thinks like gen~ and codebox, are really useful.

I do wish codebox would have some more data structures though...

something like associative arrays/maps, and it'd be great if it these could hold structures or lists as 'values'. iirc, you can do this with dict in MaxMsp (?) , so why not RNBO? 

(the translation to c++ using something like std::map would be fairly straightforward) 

I think this would make codebox much more powerful.

all that said, Im enjoying rnbo, and as above, I think codebox is really useful, and allows me to do things in lines of code that I found (frankly) pretty tedious in visual programming.

no, thank you, your input was very useful.

tbh, I tend to use Max a bit sporadically, things like RNBO pull me back occasionally, so not always clear on what's possible -  though, generally, I really like where they are going, thinks like gen~ and codebox, are really useful.

I do wish codebox would have some more data structures though...
something like associative arrays/maps, and it'd be great if it these could hold structures or lists as 'values'.  iirc, you can do this with dict in MaxMsp (?) , so why not RNBO? 
(the translation to c++ using something like std::map would be fairly straightforward) 
I think this would make codebox much more powerful.

all that said,  Im enjoying rnbo, and as above, I think codebox is really useful, and allows me to do things in lines of code that I found (frankly) pretty tedious in visual programming.

I read Isabel's explanation, but really, the third outlet of the rnbo~ [fft~] should be labelled "FFT bin" like for [ifft~]. The fact that it is labelled "phase" is confusing. Maybe some documentation writer made a shortcut with the third -argument- of fft~, which is also called phase, but yet another kind of phase.