I'm working on a project where audio analysis (not necessarily real-time) feeds parametric 3d models. I'm wondering if anyone has done or knows about audio analysis which goes further than the straightforward per-sample frequency and amplitude data.
I'm thinking of pitch, rhythmicallity (does this word exist?), texture (as in harsh/noisy versus smooth/soft), etc.
I'm aware of the fiddle, bonk and pitch externals which are candidates for developing a more complex analysis system.
I think a lot can be done just by analyzing frequency and amplitude over time. Any hints on algorithms are most welcome.