hey - here comes an interesting problem for a change!

digiology wrote on Fri, 21 August 2009 18:42Hi all,

I'm trying to capture the amplitude, phase and bin index from the output of an FFT object into three buffers and send this data to a series of oscillators.

My problem is that it comes out sounding like crap : )

If you ignore the intermediate storage and retrieval steps it is essentially the output of fft~ going in to a cartopol~ object and into the oscbank~ object.

....

I just have a bit of maths going on to map the bin indexes to the oscillator frequencies also.

I had also tried multiplying the amplitude and phase by zero when the index was >= 128 but it hasn't helped.

Do the amplitude and phase have to be rescaled or something? Or is it that I don't have another overlapping fft object perhaps?

yes, you've got to map the phases from -pi/+pi to 0/+1 and (if you are interested in the original sound) scale the frequencies correctly. scaling the magnitudes is also a good idea, to prevent blowing your speakers. and yes, to get that completely right, you'd have to do it twice and implement an windowed frame overlap.

here is a quick example - without overlapping - which gets close to the original, but has issues at the beginning of every frame:

the problem is that if you want to use the correct phases of the fft analysis, then oscbank~ needs 'framesync' to be on (which you did correctly). but at some point of time the information got to be updated - this happens when the phase index input is 0 (at the beginning of every frame), which causes a small hiatus. this results in clicks. to avoid this, you need windowing and to avoid the amplitude modulation resulting from windowing you need the overlap.

in the end, this gets quite cpu intensive, and i don't know if this is really the right track for what you want to do.

incorporating phase information is often complicating the matter. that's why the phase-vocoder became so popular. i.e. the phase info is used to calculate the "true frequency" of the input, but is not used to set the actual phase of the resynthesis-oscillators.

does that helps a little, volker.

Thanks a million for the response. I'll have to look at the patch later and have another think about this but I think I get what you're saying.

I hadn't thought of the fact that oscbank~ only updates at the start of the frame. I had originally planned on creating 128 cycle objects and sending the parameters to them but smoothing the transitions with line objects. That is, until I found oscbank!.

Am I correct in assuming that oscbank updates each oscillators amplitude in sequence immediately if framesynch is off? (presumably smearing the spectrum). How important is retaining phase information to the sound quality?

One last question, are the negative frequencies important for reconstructing the signal or can these be omitted from the oscillator bank?

Thanks again.

digiology wrote on Sat, 22 August 2009 07:16 I hadn't thought of the fact that oscbank~ only updates at the start of the frame.

if 'framesync' is on, yes. but from how i understand the oscbank reference, you can still have amp- and freq-interpolation. quoting the reference: "When frame synchronous operation is enabled, a given index's values will only change or begin their interpolated ramps to the next value when the index input signal is 0..." only, depending on the fft-size and overlap factor, the update period might be very short, not leaving much time to interpolate.

Quote: Am I correct in assuming that oscbank updates each oscillators amplitude in sequence immediately if framesynch is off? (presumably smearing the spectrum).

i'm not sure i understand the question, but i think the answer is yes. quoting again: with 'framesync' off, "...a given index's values will change or begin their interpolated ramps to the next value when the index input signal is equal to that index."

Quote: How important is retaining phase information to the sound quality?

hm, i believe basically it's very important. but with many sound transformations it might be pretty complicated or even impossible to keep the original phase relations of the partials. then again, it really depends on the sound material. naturally transients are especially delicate in this respect.

Quote: One last question, are the negative frequencies important for reconstructing the signal or can these be omitted from the oscillator bank?

hm, hm, the concept of negative frequencies is pretty "detached" (if one can say this) and for the most part is still eluding me. but i think you refer to the mirror image(s) above the Nyquist frequency. in the oscillator bank scenario, i believe you are safe to ignore them.

Great stuff, I owe you one.

I wonder if it would be helpful to have the index signal go to zero more frequently perhaps even every second sample (so the signal could be: 0 1 0 2 0 3 ... 0 127 for example). If I ignored the negative frequencies all the oscillators would update within the same amount of time but do so immediately.

I doubt it would sound better to overlapping windowed ffts but it would be interesting to try out.

digiology wrote on Sat, 22 August 2009 15:16How important is retaining phase information to the sound quality?

This is not only important, its essential. Without phase information you get only a fixed single pitch with overtones... I would do all in a pfft~ by the way, which does all the necessary windowing and overlap for you...

I thought that the bins are at fixed frequencies anyway?

Perhaps the phase information is a way of approximating frequencies between the bins. My knowledge of the fft is limited.

Using pfft~ doesn't look feasible as I want control over which ocsillators are updated and be able to get this data from a buffer.

Can we go through the exercise of adding windowing. It would be nice to know how to do that.

stefantiedje wrote on Mon, 24 August 2009 02:24digiology wrote on Sat, 22 August 2009 15:16How important is retaining phase information to the sound quality?

This is not only important, its essential. Without phase information you get only a fixed single pitch with overtones...

i don't think the question is about doing it "with or without" phase information (whatever that's supposed to mean). a lot of spectral transformations preserve the original phase differences from frame to frame, thus the correct frequency information, but mess up the original phase _relations_ among the partials, which results in a "phasey" sound and smeared transients.

Quote: I would do all in a pfft~ by the way, which does all the necessary windowing and overlap for you...

well, in my experience, this is exactly what makes using pfft~ really difficult in this situation - the automatic windowing and overlap. pfft~ does some pretty funky things under the hood. but i'd be curious to see a working patch. anyone?

Anthony Palomba wrote on Tue, 25 August 2009 07:58Can we go through the exercise of adding windowing. It would be nice to know how to do that.

i haven't seen a patch, that uses oscbank~ in a windowed context with framesync on, and which works correctly. so i think this is an interesting topic. below is a first example. maybe someone else has some more to show.

the patch (pretty useless) does nothing but a transform to frequency domain and a resynthesis with an oscillator bank (including all the windowing/overlap stories). i've added a few comments. if you have more questions, go ahead and ask.

vb

I'll save someone 10 minutes of trouble shooting:

turn your vector size to 256.

nice patch!

Matthew Aidekman wrote on Tue, 25 August 2009 12:57I'll save someone 10 minutes of trouble shooting:

turn your vector size to 256.

nice patch!

hm? are you talking about the patch i've just sent? works fine here with any vector size. what excatly are the problems you're seeing/hearing?

strange indeed. When I originally posted it, it didn't work at 512. now it works fine at 512 but not 1024 or 2048.

sound is your typical inter-fft-frame clickiness.

I would probably put it in a poly anyway

Matthew Aidekman wrote on Tue, 25 August 2009 14:23now it works fine at 512 but not 1024 or 2048.

thanks, now i can see (and here) the problem. at 512 and higher i get an error message from [fft~] "fft~: phase must be multiple of 512, setting to 0" seems to be a limitation of [fft~], which is stated in the docs. i've never seen this before, as i usually keep my sigvs small.

so, sigvs must be 256 or smaller for the overlap to work properly. sorry for any inconveniences. vb

Nice patch.

Why did you window the input and output? I tried removing it (along with the sqrt on the window) and it sounds fine.

I would have thought only one stage of windowing would be necessary.

digiology wrote on Tue, 25 August 2009 17:11Nice patch.

Why did you window the input and output? I tried removing it (along with the sqrt on the window) and it sounds fine.

I would have thought only one stage of windowing would be necessary.

true, as long as you don't alter the analysis data, but just do a straight resynthesis, there is no need to window the input signal. however, if you plan to do modifications to the spectrum, e.g. change the freq-scale, or alter the magnitude of certain bins etc, you'd want to window the input, too, in order to minimize spectral leakage.http://en.wikipedia.org/wiki/Spectral_leakage