Help needed with stutter effect. ADSR is clicking sometimes

Hello. I'm working on a stutter type effect, designed to be used as an insert plugin.

You can see a youtube video of the current version in operation here:
http://nl.youtube.com/watch?v=o9mRM5dd1M8

And you can get the mxf file here:
http://www.bitbutter.com/public/basement_stutter.mxf.zip

As you can hear in the video, the stutter effect clicks sometimes and often the 'grains' play erratically (sometimes they do not stop when they are supposed to).

Advice i've gotten so far (thanks em411 guys!) suggests that the clicking and erratic quirks have to do with the fact that i'm using an adsr~ object, triggered by a control rate message, to control the amplitude envelope of each grain (i chose for the adsr~ because it suppresses clicks on retriggers), rather than an object that is running at audio rate.

So i'm wondering what would be the best way of controlling the amplitude of these grains, in a way that guarantees that there is no 'sloppiness' like you can currently hear.

I'm using a groove~ object to play the grains, so i could use the third 'position' output to drive something else, at audio rate, that could provide the information that would modulate the amplitude. Would an audio rate control over amplitude eliminate the messy artifacts i'm getting at the moment, do you think?

This is where my max/msp knowledge lets me down:

Should i be looking at a table? or a buffer? or would line~ be better?

If i use a buffer like thing, what would be an elegant way of getting a simple 'envelope' shape into it? and how could i avoid clicks if the 'envelope' retriggers before it's finished?

Thanks in advance to anyone who takes a look, or a crack at any of the questions!