nonrealtime issues part 2

    Mar 31 2006 | 4:18 pm
    Removing sfrecord~ or replacing it with record~/buffer~ solved the
    non-realtime slowdown with render_node, but I discovered that the video
    frame recordings are not accurate anymore. Frames get dropped in the
    process. In a 1minute render, the video is gradually lagging behind to
    about 2 seconds at the end.
    To me it looks like the the non-realtime mode, which is now pushing my cpu
    to 100%, is killing the whole idea behind jit.qfaker. Can someone think of a
    way to ensure that all frames are rendered and written to disk, without
    having to slow it down 30x with sfrecord~?

    • Mar 31 2006 | 8:38 pm
      Difficult to say. Off the top of my head I can't think of why just
      changing sfplay/record to a buffer based model would affect this
      (unless you aren't running scheduler in audio interrupt, which is a
      pretty critical part of this patch if I remember correctly).
      Unfortunately i don't have the time to pick the patch apart, perform
      tests, and comment. I will mention a few things:
      - A good way to debug items such as frame drop and application
      behavior huring movie rendering is to write debug information in each
      frame recorded (e.g. frame number, assumed running time, any relevant
      parameters, etc with or jit.lcd). This might prove
      - I believe the NRT model employed by this patch is only necessary
      for synchronizing audio/video during the rendering process. If this
      is not necessary, you can avoid the audio driver, scheduler, and
      qfaker and drive the entire system with a synthesized clock generated
      from qmetro->counter or something of that form.
      Hope this helps.
    • Apr 02 2006 | 3:14 pm
      Here's a little movie that demonstrates the problem. It is supposed to
      be rendered at 25 fps, but quicktime shows 26.09. The numbers on the bottom
      indicate: world-time / smtpe / dsptime. The dsptime is what the smpte and
      world-time is generated from, in the smpte subpatcher javascript.
      Open the movie and navigate by frame. See the 3 frames from 9402 woldtime.
      With the 3 squares in the middle, its obvious that a frame is skipped from
      the movie. (the objects are supposed to move at constant speed) The smpte
      code is normal (always 1frame incr) while the world and dsp time show a
      2 frame increase (9402-> 9468).
      It's obvious that the objects don't move smoothly at a fixed velocity. This
      is always the problem in non-realtime mode, whether recording to disk or
      just playing, so I guess it has nothing to do with disk access.
      I have no idea what to make of these findings. Anyone a clue? Are there
      other things I can investigate? Maybe porting the smpte clock script to java
      or c?
      Best, Thijs
    • Apr 02 2006 | 3:53 pm
      I just discoverd that the jump is occuring exactly every 2 seconds, so I
      looked at the signal vector and it seems related. Setting the vector to 2
      makes it a little smoother, 1 makes the whole ogl dissapear and bigger sizes
      make the animation more screwed up.
      I turned off scheduler at audio interrupt, and this lookes a lot better
      (maybe perfect, but that's hard to say because its playing faster than
      realtime). Problem is, I can't record non-realtime with audio interrupt off.
      Max hangs.
    • Apr 02 2006 | 6:29 pm
      In nonrealtime mode I am using qfaker in a very specific way. If
      done in realtime mode, or with audio interrupt off, this will cause
      Max to hang, as noted in the big red warnings in the patch.
      It looks like you are running into some math issues in the clock
      which might happen more at 25fps. I have not tried this frame rate
      much. I'll look at it in a bit.
    • Apr 02 2006 | 7:42 pm
      Thanks Randy for looking into it. I don't think the problem is specifically
      with 25fps. I've used other framerates too. All rates worked well before,
      until I got rid of sfrecord~. I'll just use the slow way for now.
    • Apr 02 2006 | 10:37 pm
      Hi Randy,
      I think I found the problem. 29,97fps is hard coded in some parts of the
      smpte_clock javascript. 29,97 is the only render_node fps setting that
      works, giving a 30fps output (?).
      25 fps generates a 26.09fps movie.
      30 fps output a 31.58fps movie.
      So the non-realtime mode wasn't the problem after all. Glad it works.
      Best, Thijs
    • Apr 02 2006 | 11:58 pm
      if you read through the javascript, you will find that gFPS is
      initialized with a default value of 29.97, but this global is changed
      when the script receives the fps() message. If the script is not
      receiving the message, this will cause problems. If changing the
      default in the Javascript fixed your problem, it's likely that the
      message is not getting to the script for some reason.
    • Apr 03 2006 | 12:16 am
      Maybe I did something stupid while hacking the patch. I'll have a closer
      look at it tomorrow. Thanks.