to_the_sun

I have a patch which detects patterns in free-form playing, pulls out a single iteration of such a pattern and begins playing it back, looping it in a separate track. The project has been very successful up until this point. The next step however, is to set up multiple patterns looping simultaneously and in order for this to be successful I need to have their detected durations be 100% accurate so they don't stray from each other. So far they have always been off by a few milliseconds. For example, although the patch is designed to be used as a free-form jam aid, I can test it by creating a simple clip and letting it loop. The standard clip is exactly 2000 ms. My program usually detects the loop length at something like 1996 ms. This can probably be attributed to some portion of my programming, but I'd like to get as much advice about the situation as I can in order to make the best decision about where in my rather complex patch to begin tinkering. First question:

1) It is possible to measure durations accurately in M4L down to the millisecond (or better yet, sub-millisecond) isn't it?

At the very beginning of my process I use a [clocker 1.] to take these measurements, but they seem to usually be short by a ms or two. For example, if I set up a basic equally-spaced four-beat loop, the times I'm getting are usually 499 ms, sometimes 498, and rarely 500 as they should be. Here's the code:

Actually, I have plenty of other questions, but I think I'll leave it at that for now. The discrepancies concerning this clocker are highly suspect, and I'd like to hear what there is to be said about it before anything else. I have tried replacing it with a [cpuclock] without success. That resulted in wildly variable accuracy depending on buffer size and other factors; in fact it was only at all close to what it should have been when running externally in edit mode with a real small buffer. Any comments at all are appreciated!

the-best-way-to-get-sub-millisecond-accurate-timing-please-advise

Messing around with it further, I made a couple changes,

realizing that the [clocker 1.] was not actually getting me float numbers. Now I can see what's beyond the decimal point, but they seem to be fairly random intervals here... Any suggestions on a better way to go about this?

you should look at "advanced scheduler settings" in the max help. there are many salient points in this that can influence max's internal timing.

i notice a round object too this will possibly create inaccuracies in measured timings?

in your patch you use a button and a ggate. these should be replaced by a trigger and a gate object respectively

i believe cpuclock is the most accurate method of measuring timing in max, but you may not have noticed a difference due to inappropriate scheduler settings

you should look at "advanced scheduler settings" in the max help. there are many salient points in this that can influence max's internal timing.
make sure max is in overdrive too.
i notice a round object too this will possibly create inaccuracies in measured timings?
in your patch you use a button and a ggate. these should be replaced by a trigger and a gate object respectively
i believe cpuclock is the most accurate method of measuring timing in max, but you may not have noticed a difference due to inappropriate scheduler settings
hope some of these things help


The cpuclock object references the CPU clock directly and returns values with 

The cpuclock object references the CPU clock directly and returns values with microsecond precision.


Thanks for the tips. I'll keep working on it. The [round] was a recent (and unnecessary) addition, basically clutter left over from messing around with things. This was the first patch I ever started working on (back some 5 odd months ago), so back then I didn't even know there was a regular gate object or the proper use of a trigger.. Needless to say there's plenty of sloppiness to be cleaned up. And I'll read up on those scheduler settings.

Ok, I looked over the scheduler settings and changed any I thought might help, but there was no change in behavior. They now are:

Overdrive: On (Overdrive is always on in m4l devices, isn't it?)

I even disconnected the subpatch COMPILE from the rest of the possibly expensive processing that lies in the rest of the patch as a whole, just to make sure, but even with this stub of a patch I can't get accurate numbers out.

@metamax Your little patch illumines a problem for me. At first I was going to comment that the delay of 1 ms was varying by more than .5 ms both above and below. Now I come back to it and it's far worse! It's all over the place, giving me numbers as high as 22 ms! I reset all the scheduler settings to their defaults, and still. Seems to be if I close the patch I had open for editing it goes back to the more (but still not very) accurate results...

Ok, I looked over the scheduler settings and changed any I thought might help, but there was no change in behavior. They now are:
Enable Refresh: On
Event Interval: 2
Overdrive: On (Overdrive is always on in m4l devices, isn't it?)
Poll Throttle: 100
Queue Throttle: 10
Redraw Queue Throttle: 1000
Refresh Rate: 100
Scheduler Interval: 1
Scheduler Slop: 1

@metamax Your little patch illumines a problem for me. At first I was going to comment that the delay of 1 ms was varying by more than .5 ms both above and below. Now I come back to it and it's far worse! It's all over the place, giving me numbers as high as 22 ms! I reset all the scheduler settings to their defaults, and still. Seems to be if I close the patch I had open for editing it goes back to the more (but still not very) accurate results...


Any ideas about what else might be giving me such inaccurate timing? Something with my computer as a whole maybe? I'm running Windows 7, 3.3 ghz processor, 8 gb ram.

I think the timing problem is specific to M4L. Note that in Live each 3rd party plugin including M4L (audio and midi) introduces latency relative to Live's audio buffer size. For the Live set these latencies are globally compensated but the timing of events inside M4L devices will be different from the global timing.

For example, running the test from metamax with delay 1 and buffer size 512 I get variations up to 12ms which corresponds to the latency of that buffer. The accuracy would be improved by setting the buffer as small as possible, but still far away from sub-milliseconds.

I see what you're saying in regard to cpuclock and buffer size. When I use an ordinary clocker however, I don't see those kind of buffer-related variations. Would I be right to assume this is because it's using the Max scheduler, which compensates for it? Even so, my measurements are consistently short (why always short and never long?) by a ms or two. You wouldn't say this loss of ~1ms is to be expected within a device the same way you say devices "introduce latency relative to Live's audio buffer size," would you? I mean it doesn't seem affected by the buffer and it's not really even latency; I would expect latency would ADD a ms, wouldn't it?

Please forgive me being an amateur in all this, I really appreciate any help that's offered.

Broc,
I see what you're saying in regard to cpuclock and buffer size. When I use an ordinary clocker however, I don't see those kind of buffer-related variations. Would I be right to assume this is because it's using the Max scheduler, which compensates for it? Even so, my measurements are consistently short (why always short and never long?) by a ms or two. You wouldn't say this loss of ~1ms is to be expected within a device the same way you say devices "introduce latency relative to Live's audio buffer size," would you? I mean it doesn't seem affected by the buffer and it's not really even latency; I would expect latency would ADD a ms, wouldn't it?

Please forgive me being an amateur in all this, I really appreciate any help that's offered.


Using clocker is a good idea but doesn't seem reliable either. When I run the patch in Live and print the output from [clocker 1] on the Max runtime window, I get something like this

So it confirms your observation of a negative delay. You could try some more tests to find a reliable pattern but I'd suspect non-deterministic behavior.

Assuming that the variations are always very small as in my test, they could easily be eliminated with [round] on the clocker output.

PS.
Assuming that the variations are always very small as in my test, they could easily be eliminated with [round] on the clocker output.


Hmmmmmm well as long as the -0.002 wasn't 

 every ms, it wouldn't account for the full ms I'm missing.. With further investigation I have found some similarly odd behavior regarding the clocker, though. It seems that the bang arriving at the clocker is not immediately followed by output from it. If you look at my attachment you can see that the rest of the [t] is executed before the clocker spits out its first message upon being banged, which causes things to happen in the wrong order. At first I thought I had found the source of the problem, but then I set up a [delay 1] to measure this pause in the clocker's count, which would account for my missing ms (if it were a whole ms), but since the delayed bang seems to come through on time, the pause must be less than a tenth of a ms (the resolution of the clocker). Anyone see any flaws in my logic here?

Still, this is weird behavior is it not? Clockers are expected to output immediately aren't they? Or do they, unlike other objects, have to take time to sync up with Max’s internal millisecond clock or something like that? [edit]

(Attached is the exact .amxd I used to test this I case anyone should feel so obliged as to try it themselves)

Hmmmmmm well as long as the -0.002 wasn't accumulating every ms, it wouldn't account for the full ms I'm missing..  With further investigation I have found some similarly odd behavior regarding the clocker, though. It seems that the bang arriving at the clocker is not immediately followed by output from it. If you look at my attachment you can see that the rest of the [t] is executed before the clocker spits out its first message upon being banged, which causes things to happen in the wrong order. At first I thought I had found the source of the problem, but then I set up a [delay 1] to measure this pause in the clocker's count, which would account for my missing ms (if it were a whole ms), but since the delayed bang seems to come through on time, the pause must be less than a tenth of a ms (the resolution of the clocker). Anyone see any flaws in my logic here?

(Attached is the exact .amxd I used to test this I case anyone should feel so obliged as to try it themselves)


It seems that the bang arriving at the clocker is not immediately followed by output from it. If you look at my attachment you can see that the rest of the [t] is executed before the clocker spits out its first message upon being banged, which causes things to happen in the wrong order.

I suspect it is just the way your print objects are printing messages, which is dependent on the position each print object has on screen in relation to each other to determine hierarchy of order of execution.

it also takes a certain amount of time to execute the print 'request', so I wouldn't worry about the order of the print messages in the console window there

I suspect it is just the way your print objects are printing messages, which is dependent on the position each print object has on screen in relation to each other to determine hierarchy of order of execution.
 it also takes a certain amount of time to execute the print 'request', so I wouldn't worry about the order of the print messages in the console window there


what broc said. most likely a problem of the synchronisation with the host.

There shouldn't be any ambiguity in the order of the print objects. They're all off to the right and each connected to a different outlet. And as far as a delay in executing a print, one would assume that all of the print objects would be subject to the same delay, not just the print object connected to a clocker, right?

The only non-deterministic result I'm seeing here is in the exact duration of my measurement (sometimes it's 499.9, sometimes 499.2, sometimes 498.7, etc.) and this could be attributed to synchronization with the host, but the loss of ~ a ms is consistent. Now I suppose I could just throw a [+ 1] in the chain but that seems sloppy, especially if I can't figure out 

Floating Point,
There shouldn't be any ambiguity in the order of the print objects. They're all off to the right and each connected to a different outlet. And as far as a delay in executing a print, one would assume that all of the print objects would be subject to the same delay, not just the print object connected to a clocker, right?

The only non-deterministic result I'm seeing here is in the exact duration of my measurement (sometimes it's 499.9, sometimes 499.2, sometimes 498.7, etc.) and this could be attributed to synchronization with the host, but the loss of ~ a ms is consistent. Now I suppose I could just throw a [+ 1] in the chain but that seems sloppy, especially if I can't figure out why I'm losing that ms.....


sorry didn't fully understand your question-- it is strange the clocker's output occurs after the other processes have finished, but it seems to be outputting the correct readings nonetheless; one thing you could try is using a [delay 0] immediately before the clocker, to ensure all events are in the high priority thread, but I think they already are, so it will probably not change anything. maybe what you are seeing is scheduler slop-- maybe reduce it to less (if possible) otherwise you'll have to live with it

There are int arguments ($i1) in your if statements leading to truncation, eg. clocker value 1.99 will be converted to 1 (which is about 1ms difference). That's why I've suggested using [round].

Thanks for the suggestions. I tried [delay 0] and reducing scheduler slop, but there was no effect.

There are int arguments ($i1) in your if statements leading to truncation, eg. clocker value 1.99 will be converted to 1 (which is about 1ms difference). That’s why I’ve suggested using [round].

My clockers have always been putting out numbers that are clean to the level of their precision. I wasn't getting any -.002 type numbers like you said you were, but I did try using a [round] after a [clocker 0.1] (and before a [+ 1]). The deviation was still present and when I compared it to just using a [clocker 1] followed by a [+ 1], I found the latter to be slightly more accurate. I averaged the measurements taken for the first hundred beats (500.369995 vs 499.73999). So I guess I'll stick with that for now. It's sloppy but it'll have to do.

Thanks for the suggestions. I tried [delay 0] and reducing scheduler slop, but there was no effect. 

My clockers have always been putting out numbers that are clean to the level of their precision. I wasn't getting any -.002 type numbers like you said you were, but I did try using a [round] after a [clocker 0.1] (and before a [+ 1]). The deviation was still present and when I compared it to just using a [clocker 1] followed by a [+ 1], I found the latter to be slightly more accurate. I averaged the measurements taken for the first hundred beats (500.369995 vs 499.73999). So I guess I'll stick with that for now. It's sloppy but it'll have to do.


After a very successful conversion in another device from seq to seq~, i've begun to realize that probably the best way to go about things in an audio programming language like this is to encode as much data as possible in actual signals. That way one should be able to reap the benefits of sample-based accuracy, right? It shouldn't be too hard to swap out the coll for a seq~ in this case. I'm thinking the seq~ would need to be continuously phase-shifted, so that each nascent beat enters at a set point and then slides off in one direction and is finally removed when it reaches the starting point again, thereby keeping a record of the most recent X seconds, the way my coll currently does. Comments, suggestions?

Long belated success! I replaced my [clocker 1] with

And it reports the correct length of 2000 ms every time.

Long belated success!  I replaced my [clocker 1] with

And it reports the correct length of 2000 ms every time.


I'm experiencing sloppy MIDI file playback.

Is there a similar approach to get more accurate MIDI file playback? I'm sending the MIDI data directly to a soft synth, so it's not leaving Max via a MIDI port so I was expecting to get rock solid timing for my quantized MIDI files...

Is there an approach I could use similar to this one to fix that?

Is there a similar approach to get more accurate MIDI file playback?  I'm sending the MIDI data directly to a soft synth, so it's not leaving Max via a MIDI port so I was expecting to get rock solid timing for my quantized MIDI files... 

Is there an approach I could use similar to this one to fix that?


A MIDI file imported into a Live track going directly to a plug-in? You're not going to make things tighter by introducing a m4l device into the mix, if that's what you're asking. How sloppy are we talking? What's your buffer size at?

Buffer size doesn't seem to matter. I've tried it low (64) and high (1024) with the same results.

My test MIDI file is a simple 2 bar drum phrase with the only two notes quantized to be EXACTLY on beats 1 & 2 for the kick and 2 & 4 for the snare at a tempo of 120 BPM.

Recording the audio output of the audio VST~ plugin driven by the playback of the MIDI file loaded into SEQ shows the snare SOMETIMES shows up early (by a few milliseconds - somewhere around 20-30 ms) and sometimes shows up late.

I've used other sequencers and don't hear this much slop in the MIDI playback.

What can be done to make this better? Anything?

Buffer size doesn't seem to matter.  I've tried it low (64) and high (1024) with the same results.

What can be done to make this better?  Anything?

So you have a seq object sending MIDI to a vst~ object loaded with a synth, all within a max-made standalone plug-in? First thing I'd try to do is determine exactly where the timing inaccuracies are being generated. Is it in sync directly after the seq or no? If so, then try to find which other object is at fault; I assume there must be much more to the patch those two objects.

In general, I find ordinary max objects can be clunky and I trust the MSP ones, like seq~, to be more accurate if you can use them. Hope that helps.

I would love to use SEQ~ for MIDI playback, but I don't see any examples to see how this is done.

It makes sense to me that SEQ~ would be more accurate so I would like to try that.

Do you know where I can find examples of using SEQ~ to playback MIDI files?

could be tricky. Alternatively, have you looked at the 

message for the ordinary Max seq object? Maybe it could be used to have some more accurate object drive the playback.

I feel like I should ask also, is there a reason you can't just drag the midi file into a clip in Live and have it played that way?

Yeah, using it to play midi files could be tricky. Alternatively, have you looked at the tick message for the ordinary Max seq object? Maybe it could be used to have some more accurate object drive the playback.

I feel like I should ask also, is there a reason you can't just drag the midi file into a clip in Live and have it played that way?


I've looked at the 'tick' message, but it seemed a bit daunting to me to write that much code to play a midi file. But if I can't get SEQ to play any better, I may be forced to.

The reason I cannot drag a midi file into Live is because this is not a Live application.

It's a stand alone MAX app. Ableton Live is not even on the machine.

I like your trick about using the cputimer to get better accuracy. And that is the kind of accuracy I would 'like' to get... but it sure seems like a lot of work just to get a midi file to play back accurately.

Has anybody run into this MIDI 'slop' using Max before? Or is this something that people just accept as 'ok'?

Surely there has got to be a better solution than the one they're providing.

I've looked at the 'tick' message, but it seemed a bit daunting to me to write that much code to play a midi file.  But if I can't get SEQ to play any better, I may be forced to.

It's a stand alone MAX app.  Ableton Live is not even on the machine.

I like your trick about using the cputimer to get better accuracy.  And that is the kind of accuracy I would 'like' to get... but it sure seems like a lot of work just to get a midi file to play back accurately.

Has anybody run into this MIDI 'slop' using Max before?  Or is this something that people just accept as 'ok'?

Oh okay, I gotcha about the standalone part. No I don't think people generally assume sloppiness upwards of 30 ms is okay, but using the tick message shouldn't amount to much of any extra code. If I'm reading the help file correctly all you should need is something like this

And thank you for the example of the code... it would have taken me a long time to figure it out, but it was a snap with your example!

The best way to get sub-millisecond accurate timing-- Please advise