We just released a new, simple video software called DNA: http://www.lividinstruments.com/software_celldna.php
It’s all built with Jitter, and uses all OpenGL rendering and effects. (almost – the freeframe effects rely on texture readback)
The nice thing about it for Jitter users is that I’ve made it pretty simple to add your own Max patches into the system, so they can be integrated with the rest of the effects and video sources. You can read a bit more about it here: http://www.lividindustry.com/culture/?p=168
It’s not exactly open source, but you can pull apart the patches in the CellFX and Extras folders and learn how to add sources, effects, and controls to the existing structure of DNA.