Grain size independent from pitch

Hey!
I am building a granular synth based on Sakonda's sugarSynth and I was wondering if there's a relatively easy way of separating grain size and pitch? I've studied a few granular synths, such as Nathan Wolek's and Brendan McCloskey's works and while I've been able to separate the two - I would then get problems when trying to change the number of poly voices or I'd get a tremolo-like effect when changing the size and then going back. Any help is greatly appreciated!

Thanks.