video to audio conversion / matrix sonification

kampana's icon

hello,
i'm experimenting with converting video to audio, mostly looking into examples that demonstrate how jit.release~ and jit.peek~ work. however, my experience with jitter is limited and i'm kind of lost in finding the right way to use these objects for my purpose: i have a video with white background and black squares that constantly move around and form all kinds of geometrical shapes. i want to be able to detect the density of these black squares in the video, and use it to generate sound which will change in correspondence with the changing geometrical shapes.

could somebody please advise how to start off such patch?

or otherwise, if anyone can please point me out to threads or patch examples of video to audio conversion, video-to-audio synth, etc.

thanks!
K.