P-frame motion vectors from mpeg file
Hi all,
I'm looking for a way to extract the motion vector data of a p-frame out of an mpeg stream. So far I have not been successful in finding out how to do this...
Has anyone done this before or does anyone know if this i feasible? I would like to get them stored in a matrix eventually.
rgds,
Brecht.
Hello Brecht:
very interesting question, unfortunately I don't know the answer. Maybe by binding an external library(java, c)?, do you know any framework that already has this feature implemented?
I know FFmpeg has the option to visualize the motion vectors when used through the terminal (command is in the manual). I don't know though whether ffmpeg's source is available anywhere or if there's an api like with vlc...
Unfortunately, I'm no programming hero so getting access to them in that way is far beyond my capabilities.
Hello brecht:
I checked the FFmpeg site and the source is available(you can get it by cloning with git). The source is in 'c' which already might be a positive sign. Most of the headers invoke libraries that seem to be wrapper for different utilities: graphic interfaces, etc. Maybe it would be matter of discovering which library is taking care of the motion vector extraction(maybe someone on the forum here?).
I am curious, is this type of analysis doable on real time(it should be slightly heavy, no?); how are the vectors's data represented?
As i mentioned before i am not(by no means) an expert but i am rather curious!
-emmanuel
Apparently Mplayer can visualize the motion vectors to. Those look like this:
http://blog.prashanthellina.com/2008/03/30/visualizing-mpeg4-motion-compensation-vectors-using-mplayer/
How they are represented internally, I have no clue, but I suspect that this would be pretty straightforward up/down left/right per 'block'
FWIW, mplayer uses ffmpeg (its libavcodec "sublibrary") to draw those arrows. I haven't found a way to easily extract the vectors using just mplayer (or ffmpeg's own executable), but it should be pretty easy to write a utility using 'libavcodec' that takes an mpeg movie and produces, say two plane matrix per frame with motion vector components (or three plane matrix with third plane empty). As far as I remember jit.qt.movie understands some form of uncompressed frames, that would be a perfect one to use here.
as far as I understand the vectors are returned by libavcodec during decoding as part of AVPFrame structure (see comments to motion_val, motion_subsample_log2, and may be more in the file avcodec.h).
half a year ago I was doing some ffmpeg based development and at that time I found a very nice series of tutorials on coding with it:
(but watch out that it is slightly out of date)
also, to look at the places where they use the motion_val to draw arrows:
(scroll to line 1845 if browser doesn't)
all projects that use ffmpeg's libavcodec include this same drawing code (even if in some it isn't accessible behind their own APIs). the drawing code is pretty messy since (a) it has to fit into generic playback pipeline and only run if some configuration was set, (b) drawing the arrow is done slightly differently depending on the size of macroblock (8x8,8x16,16x8,16x16). To get this data into jitter matrices you just ask avcodec to decode for you and look at the shape of motion_val returned and save it in some form you can get into jitter (the linked snippet is just for reference of how to access that data).