madbutter

I'm trying to write a patcher that processes a movie using a list of rectangle dims so that, for every frame of the movie, the pixel values within each rectangle are averaged out and then all replaced with that solid color. I will have the list of rectangles before starting the processing, but the count and placement of these rectangles will be different for every different movie (but consistent within a single movie that is being processed).

Say for instance I have a movie like the 320x240 bball.mov. I know for this movie I want to process 10 rectangular regions that are each 12x12 pixels scattered randomly around the matrix and I want to fill each of those regions with the solid color that is the average color within the region.

I know a brute force, cumbersome way to do this: I could use javascript to first get the list of rectangles, then step through each frame of the movie use getcell for each pixel in each region, average the value and then write them back using setcell. Or maybe I could get submatrices out for each rectangle, re-sample them to 1x1 with interpolation, then write those back to the rectangle resolution and position.

There must be a more clever way to do this! Using Gen (just starting to wrap my head around it)?

Anybody have any ideas for me? Thx in advance.

I'm trying to write a patcher that processes a movie using a list of rectangle dims so that, for every frame of the movie, the pixel values within each rectangle are averaged out and then all replaced with that solid color. I will have the list of rectangles before starting the processing, but the count and placement of these rectangles will be different for every different movie (but consistent within a single movie that is being processed).
Say for instance I have a movie like the 320x240 bball.mov. I know for this movie I want to process 10 rectangular regions that are each 12x12 pixels scattered randomly around the matrix and I want to fill each of those regions with the solid color that is the average color within the region.
I know a brute force, cumbersome way to do this: I could use javascript to first get the list of rectangles, then step through each frame of the movie use getcell for each pixel in each region, average the value and then write them back using setcell. Or maybe I could get submatrices out for each rectangle, re-sample them to 1x1 with interpolation, then write those back to the rectangle resolution and position.
There must be a more clever way to do this! Using Gen (just starting to wrap my head around it)?
Anybody have any ideas for me? Thx in advance.
-bob


average-color-values-replace-all-pixels-in-arbitrary-regions

@srcdim of jit.matrix will get you your subsampled rectangles if you want to avoid javascript.

@dstdim would allow you to overwrite into the original matrix at the specified rectangle coordinates.

Thanks Jesse, I forgot about jit.3m. Your srcdim / dstdim suggestion is what I had in mind in my second suggestion, but I had the added step of downsampling the submatrix to a 1x1 matrix to get the mean and then upsampling it back to the original dim of the rectangle to make it the right size to re-insert - using jit.3m may be more efficient.

My biggest challenge is to figure out how to scale this to perhaps hundreds of rects per frame. Maybe setting it all up as jitter objects in javascript may make it easier to iterate through the rectangle list (The processing does not have to be done in real time but then again I don't want it to take any longer than it has to!).

I was guessing that this is the type of thing that Gen could do efficiently - if it is I could buckle down and try to learn how to write such a thing! A Gen solution would have the advantage of being a highly optimized solution to this part of my work chain - just feed the beast a big list of rectangles, then blast it with consecutive frames and have it burp out processed frames...

Thanks Jesse, I forgot about jit.3m. Your srcdim / dstdim suggestion is what I had in mind in my second suggestion, but I had the added step of downsampling the submatrix to a 1x1 matrix to get the mean and then upsampling it back to the original dim of the rectangle to make it the right size to re-insert - using jit.3m may be more efficient.
My biggest challenge is to figure out how to scale this to perhaps hundreds of rects per frame. Maybe setting it all up as jitter objects in javascript may make it easier to iterate through the rectangle list (The processing does not have to be done in real time but then again I don't want it to take any longer than it has to!).
I was guessing that this is the type of thing that Gen could do efficiently  - if it is I could buckle down and try to learn how to write such a thing! A Gen solution would have the advantage of being a highly optimized solution to this part of my work chain - just feed the beast a big list of rectangles, then blast it with consecutive frames and have it burp out processed frames...


average color values replace all pixels in arbitrary regions