There were a couple problems. You had joined all the groove~ sync outs, but because they're signals, they added so the result was between 9.0 and 10.0 instead of the 0.0 to 1.0 you were expecting. Secondly, because of the way that the end was being detected using delta~, it was re-triggering madly. I swapped delta~ for change~. I also changed the groove~s to point to a dummy buffer and only get pointed to their real buffer after polybuffer~ creates them.
All this said, the way I would approach this patch is to use poly~
[sel] doesn't send a bang out it's right outlet. It sends the things that were'n selected for. The [sel 1] instance would send the ints 2..9 out the right outlet.
I put the change there so that any given groove~ would only be "stopped" once, and not be re-stopped when each other groove~ was started. Thinking about that now, I see that that is misguided, as I would really want to set to the highest number in ms of the loaded sample, so that's a bug.