constantly report the median of last second flow of numbers

    Aug 10 2011 | 2:26 pm
    HI all,
    I'm trying to solve this problem, which looked simple to me, but I can't solve it. I have a flow of numbers coming at irregular intervals and I'd like to calculate the median for istance of the past 1 second. I know how to return the median value every second, but what i can't figure out is how i can have the median not only giving me result every second, but constantly. If something happens at a millisecond rate, I'd like to have the idea istant by istant what the last second was. And i can't solve this problem.
    I tried several obkects like [bucket] [pak] [zl median] [accum], but i can only have them reporting me the median of the (very quick) flow "every" second, not instantly (like 200 msec).
    Do you have any tip? Thanks!

    • Aug 10 2011 | 5:14 pm
      Post your patch annotated to show what happens, and what you want to have happen.
    • Aug 10 2011 | 6:17 pm
      Thanks Chris, here's my code:
    • Aug 10 2011 | 7:41 pm
      Might the mean work as well for you? Are there known temporal bound for how the data comes in? (min rate & max rate)
    • Aug 10 2011 | 7:50 pm
      Here's a stab at it;
    • Aug 10 2011 | 8:09 pm
      p e r f e c t
      simple, efficient, clear thanks a lot!
    • Aug 10 2011 | 11:06 pm
      using quickthresh is simpler:
    • Aug 10 2011 | 11:48 pm
      Terry, read the problem statement again. personal_username wants the median over a one second period, but wants it reported at 200 ms intervals, with each report covering the previous second.
    • Aug 11 2011 | 7:43 am
      if you really need a timing accuracy of 1ms t here is no other option but to go from a [metro 1] - but i wouldnt do that. just try 25 or 40 or 200 ms and see if you like the output.
    • Aug 11 2011 | 9:14 am
      Not sure if it's the question here, but I recently discover [meandev], from A.Harker ( It reports mean and deviation over time. One can set 4 different time ranges.
    • Aug 11 2011 | 11:37 am
      ok Chris-- so here is my modified patch using quickthresh (note it is no longer simple)...
    • Aug 11 2011 | 5:20 pm
      Nice, and probably more efficient than mine. I'll try to get around to benchmarking them tonight.
    • Aug 11 2011 | 7:02 pm
      And there's always our old friend bag...
    • Aug 12 2011 | 3:50 am
      I _really_ have to add bag in my bag of tricks.
    • Aug 12 2011 | 6:09 am
      but as yoann points out, [meandev] will easily do it all in one external
    • Aug 12 2011 | 8:25 am
      But does meandev calculate the median? Max already has mean.
    • Aug 12 2011 | 10:35 am
      yes i'm evidently mildly dyslexic-- looking at its help file it talks about mean of means, which I read as median of medians (a statistical procedure), so anyway chris' (topher) bag approach is probably the one to use
    • Aug 13 2011 | 8:06 pm
      wow, I didn't see your last posts guys! very very nice solution [bag], I didn't know it either, and looks like it is the best memory-efficient and simpler tool, great! I'll have a look at [meandev] now, but this solution Cristopher is really good for me! So, i assume if I want to check different time spans all I have to do is to change the pipe attribute, so if I want to track 30 secs all I have to do is this:
    • Aug 13 2011 | 11:12 pm
      To clarify meandev does not calculate the median ever, only means and deviations. The mean of means business is something that was probably useful at the time, but I forget what for (this is one of the oldest object in the package dating back over 5 years now I think and when I updated it more recently I just left all the old stuff in there).
      I could update it to do medians some time, but no promises especially as there are some good max solutions here.
      However, a word of caution:
      zl will only allow up to 256 values unless you specify otherwise.
      Below is my altered solution from christopher's. This will work such that whenever something goes in or comes out of the bag the output is updated, without needing a metro 1 (which is a lot of unnecessary calculation).It also allows lists of up to 4096 values.
    • Aug 13 2011 | 11:14 pm
      In case it wasn't clear above the idea is the output should always relate to the last second exactly in my solution (assuming I got it right - I've only quickly tested)...