I think you are asking the wrong question-- FM synthesis with feedback is _always_ non-linear, and therefore chaotic, by definition. It's just that the simpler states (ie low, whole-number mod indexes, low feedback amplitudes etc) are easier to describe (via bessel functions to describe the amplitudes of partials etc) and, more importantly, to perceive and make sense of, than the more complex states. When it becomes 'noisy' or 'chaotic' is simply a cultural/human-perception thing. The noisiest, most complex signal feedback path in FM is still deterministic and periodic, albeit unpredictable. Maybe you should ask 'when does the fm spectrum become too complex for the human ear for it to perceive as having an unambiguous sense of pitch?' or something similar (and then apply for a research grant to answer it)

Hm... Here's what I probably would do to come a little closer to an answer: first, I would convert your patch into a discretized equation describing the signal flow -- that is, something that has the form F ( output_signal[n], fundamental, mod_ratio, mod_index, feedback_scalar, n ) = 0 (at this point, you should take into account that the send-receive that you have in your patch introduces a delay equivalent to the current vector size). Then I would convert this into a differential equation. If I already had the discretized function F(), then this is actually quite straightforward. As a next step, I would try to compute the biggest Lyapunov-exponent of the dynamic system described by the differential equation (I would expect to get a formula for this that depends on all of the scalar parameters of your patch at the same time). This can be _very_ tricky and hard, depending on the differential equation that you get. Once you have an expression for the biggest Lyapunov-exponent, you can tell the set of parameters that would cause your system to go into the chaotic region (at least, in the mathematical sense -- which might or might not do much with the acoustic sense of 'noisiness').

And Terry is right, I wouldn't jump into this unless I got a grant for it. ;-)

Cheers, Ádám

Both excellent answers, thank you. I clearly missed the the non-linear nature of feedback.

You might also try posting this on KVRAudio's forum. There's been a lot of work with filters and feedback (esp. delay-free loops) recently, and I can't help but wonder if there might be some connection that could be of value.

Thanks Peter. I guess I was naively asking for a simple formula to tie fbAmp to modFreq. Ask the respondents state I framed the question poorly. And I'm already under the yoke of one research grant at the moment :)

There is an old paper out there about doing a sawtooth via feedback FM. It's interesting to play with and might be a starting point for some interesting explorations.

...about 2/3rds the way down in this ccrma resource:

is a reference. But sadly, once Bessel functions are mentioned I glaze over. But all the formulas are sexy; like hieroglyphs.

Thanks for the assist

Brendan