nonrealtime issues part 2

    Mar 31 2006 | 4:18 pm
    Removing sfrecord~ or replacing it with record~/buffer~ solved the non-realtime slowdown with render_node, but I discovered that the video frame recordings are not accurate anymore. Frames get dropped in the process. In a 1minute render, the video is gradually lagging behind to about 2 seconds at the end.
    To me it looks like the the non-realtime mode, which is now pushing my cpu to 100%, is killing the whole idea behind jit.qfaker. Can someone think of a way to ensure that all frames are rendered and written to disk, without having to slow it down 30x with sfrecord~?

    • Mar 31 2006 | 8:38 pm
      Difficult to say. Off the top of my head I can't think of why just changing sfplay/record to a buffer based model would affect this (unless you aren't running scheduler in audio interrupt, which is a pretty critical part of this patch if I remember correctly). Unfortunately i don't have the time to pick the patch apart, perform tests, and comment. I will mention a few things:
      - A good way to debug items such as frame drop and application behavior huring movie rendering is to write debug information in each frame recorded (e.g. frame number, assumed running time, any relevant parameters, etc with or jit.lcd). This might prove insightful
      - I believe the NRT model employed by this patch is only necessary for synchronizing audio/video during the rendering process. If this is not necessary, you can avoid the audio driver, scheduler, and qfaker and drive the entire system with a synthesized clock generated from qmetro->counter or something of that form.
      Hope this helps.
    • Apr 02 2006 | 3:14 pm
      Here's a little movie that demonstrates the problem. It is supposed to be rendered at 25 fps, but quicktime shows 26.09. The numbers on the bottom indicate: world-time / smtpe / dsptime. The dsptime is what the smpte and world-time is generated from, in the smpte subpatcher javascript.
      Open the movie and navigate by frame. See the 3 frames from 9402 woldtime. With the 3 squares in the middle, its obvious that a frame is skipped from the movie. (the objects are supposed to move at constant speed) The smpte code is normal (always 1frame incr) while the world and dsp time show a 2 frame increase (9402-> 9468).
      It's obvious that the objects don't move smoothly at a fixed velocity. This is always the problem in non-realtime mode, whether recording to disk or just playing, so I guess it has nothing to do with disk access.
      I have no idea what to make of these findings. Anyone a clue? Are there other things I can investigate? Maybe porting the smpte clock script to java or c?
      Best, Thijs
    • Apr 02 2006 | 3:53 pm
      I just discoverd that the jump is occuring exactly every 2 seconds, so I looked at the signal vector and it seems related. Setting the vector to 2 makes it a little smoother, 1 makes the whole ogl dissapear and bigger sizes make the animation more screwed up.
      I turned off scheduler at audio interrupt, and this lookes a lot better (maybe perfect, but that's hard to say because its playing faster than realtime). Problem is, I can't record non-realtime with audio interrupt off. Max hangs.
    • Apr 02 2006 | 6:29 pm
      In nonrealtime mode I am using qfaker in a very specific way. If done in realtime mode, or with audio interrupt off, this will cause Max to hang, as noted in the big red warnings in the patch.
      It looks like you are running into some math issues in the clock which might happen more at 25fps. I have not tried this frame rate much. I'll look at it in a bit.
    • Apr 02 2006 | 7:42 pm
      Thanks Randy for looking into it. I don't think the problem is specifically with 25fps. I've used other framerates too. All rates worked well before, until I got rid of sfrecord~. I'll just use the slow way for now.
    • Apr 02 2006 | 10:37 pm
      Hi Randy,
      I think I found the problem. 29,97fps is hard coded in some parts of the smpte_clock javascript. 29,97 is the only render_node fps setting that works, giving a 30fps output (?).
      25 fps generates a 26.09fps movie. 30 fps output a 31.58fps movie.
      So the non-realtime mode wasn't the problem after all. Glad it works.
      Best, Thijs
    • Apr 02 2006 | 11:58 pm
      if you read through the javascript, you will find that gFPS is initialized with a default value of 29.97, but this global is changed when the script receives the fps() message. If the script is not receiving the message, this will cause problems. If changing the default in the Javascript fixed your problem, it's likely that the message is not getting to the script for some reason.
    • Apr 03 2006 | 12:16 am
      Maybe I did something stupid while hacking the patch. I'll have a closer look at it tomorrow. Thanks.