Huge live timbrical change with FFT: emulating other instruments
Hello friends, I’m thinking about possibility of changing by FFT the timbre of a live recorded instrument, and then changing its timbrical fourier synth to another instrument. Do you think is this possible?
Will be possible in all instruments? I mean, for example it isn’t the same the clarinet (just some parcials) than the oboe or posaune. From clarinet to posaune will be more easy, maybe? However, from posaune or oboe to clarinet I don’t know if it will be possible.
Do you know something about this made in MAx MSP?
(PD: May be, I think for sure, that the other possibility is better. Analyzing in real time the note is playing and resample it, also do you know something made with Max MSP?
Thank you.
Posaune = trombone, btw.
Hi
although I’m not deeply familiar with the FFT process I would imagine that what your are describing would be a highly complex process. There are, of course, numerous examples of physical models of acoustic instruments, which are incredibly effective (the PerColate library for Max comes to mind) but capturing the timbral fingerprint of a specific instrument and then processing/manipulating it to mimic that of another would raise the question of the mechanics of acoustic instruments. Digital controllers do a treat job of mimicking other instruments, since the WX7, but this isn’t what your pursuing, I guess. There are objects such as [fzero~] and externals such as [fiddle~] which will do the analysis for you, but I’m struggling to think of existing solutions to the remainder of the equation. Perhaps a poke around inside the Max/examples/fft-fun folder would provide inspiration. For both of us! : )
HTH
Brendan
HEllo Noob_Meister, yesterday late night I found this video http://www.youtube.com/watch?v=QkMAP-7W41s from Nicola Ferro. It’s more or less what I’m looking for. I supose in this video he uses some plug-ins for changing sound, go far to minut 4:20, he plays guitar inside trombone. An yes, I suposed to raise the question of the mechanics to the acoustic of each instrument. Let’s say we’ve got a preset with the mechanics, and then, you choses you’re instrument that you’re playing, and the output instrument changing the timbrical procedures… Too heavy maths maybe?
I’ve tried thos objects, fiddle~ and fzero~ However, I’m not trusting at all on them, I had some problems with fiddle~ in a live concert (maybe it was a previous version 3/4 years ago). I will check it now, but that will be the easy way, I’d like to solve or investigate the possibilities of complex one ;)
Hi
I understand the idea you are describing a little better now. It should not be very difficult at all to change the spectral content of one instrument so that it mimics that of another – I thought you were asking about changing one or more instruments into any other instrument. FFT is the way to go, and there is also the art of convolution; two concepts and practices I sadly know nothing about, but I withdraw my comment about it being highly complex!
If you intend to stay within orchestral families regarding mutation of the sound, from ‘cello to violin or from tuba to trumpet for example, then the other question of mechanics also conveniently fades into the background. Crossing families and physical properties would be more challenging though I guess. But in Max you can do most things, for example, simulating the noise of a plectrum on a guitar string, or a rapidly vibrating reed is relatively trivial. How you trigger them is up to you.
Brendan
Hi Lads,
I think you are describing different things.
I would imagine doing an FFT "finger print" in real time would indeed be very complex. While the relationship of the partials is useful for additive synthesis remember that FFT analysis takes only a snapshot of the sound. In synthesis you calculate the partials relationship and just synthesise them, the results would be similar to what you get with pure additive synthesis or FM. If you were to do transformations I reckon you’d need to keep track of each individual partial of the incoming sound and operate on it different to match the target timbre. For a realistic transformation you’d also need to keep track of the noise part of the signal and process it accordingly. Which would increase artifacts of the FFT process
Brendan talks about using physical models in which case you could use some sophisticated signal analysis to get more information than just the pitch and amplitude of the original signal and then carefully scale and map these features to the model’s parameters.
As for the provided example, judging from the video, I’d say what is going on is far simpler. The mute he is using wouldn’t necessarily have a high quality microphone. This would change slightly the original timbre of the trombone. Also, it is inside the instrument so the added characteristics of the bell wouldn’t be in the signal. Due to shared cultural associations, any clean periodical monophonic signal going through a guitar distortion pedal does sound guitar-like. He is using a set of effects like distortion, slight delays that are a typical guitar solo settings and he is playing idiomatic guitar phrases. I would guess that is what’s going on.
You can test this by taking any melodic traditional instrument recording, putting it into your favorite DAW and apply distortion, delay with settings that would normally be used for guitar. I’m sure the results would be quite convincing if you want a guitar sound. If you want a different timbre I still think it would be quite complex.
Another option would be to use cross-synthesis but then you need a recorded file of the target sound that is playing exactly what you would be playing live at the same tempo. This is actually a way of filtering a signal of a source through the timbre of another but you’d need perfect sync for it to work reliably.
Having said that, often the most interesting sounds are made by attempting to do these kind of things and failing in awesome ways that instead of reaching the original goal open new possibilities.
Just my 2p
– Miguel
Really doubt there’s anything more complex than that in the example.