[sharing + question] Find average color, ignoring black. (selective jit.3m)

Attached is my inelegant solution to a problem I often face: How to calculate the average color of all pixels that aren't black. (How do I make jit.3m ignore black pixels?)

Context:
I want to detect the average color of certain objects. I performed background subtraction on a camera feed, leaving me with objects against a black background. After converting to HSL colorspace I wanted to get the average hue of the non-black pixels. (This isn't possible with jit.3m since it includes the black pixels in the average, watering-down the data.)

My Solution:
In the attached patch I iterate through the matrix, testing to see which pixels meet minimum saturation and luma conditions. If they pass the test, those pixels are added to a new 1D matrix which can be analyzed with jit.3m. It works perfectly but it's slow as molasses.

Question: Can somebody share a smarter approach?. (maybe js or gen?)

Patch and small sample video are attached. No camera or background-subtraction required. (The forum software wouldn't let me include my copy-compressed patch in the post, sorry.)