Scheduling Jitter and Audio Processing

Hello,

I'm working on a jitter patch that iterates through a matrix and modifies values in each cell any time a midi note is played. The notes are also going to a (very clunky) polyphonic sampler, but if I play a chord there's a pause between each note while jit.iter processes. Is there a way to force the audio to process first, and then let the matrix update as it can? I have the same problem whether overdrive is on or off.

Here's the patch: