I don't really have a specific question just more of how to do this. We are trying to track new added color that's been added to an already wall of colors. So every time a color is added i would like to track that new color and not the other colors that have been previously added.

tracking-color

You could try jit.chromakey @mode !, and send that output to your tracker. in mode 1, it sends out a greyscale image, where the color you select to key is black. Therefore, if you have a red splotch and a green splotch on a white background the output will be black where the red splotch is, and white everywhere else.

Tracking color