Safely Using Asynchronous Functions in Min?
Greetings! I'm trying to deal with asynchronous functions inside my MIN external. My object is not sharing any state with any other part of Max, so as far as I can tell, I don't think the Max Threading Model is a problem here.
I'll try to stay very general, but I can add more specific details if it will help.
Basically, I have a function that takes a callback as a parameter. And in certain parts of my code I need to wait for that callback to return before I proceed.
My first crude solution was to use sleep(200) to sleep for 200 milliseconds while I wait for the callback. This was a fine, though rudimentary, solution in my C++ command line prototype. But when I called sleep() in a Max external I found the whole patch would freeze for 200ms.
I thought, "Okay, I need to get this on a separate thread." So I spun up a new thread inside my object and put my sleep() on there. Same problem.
So I looked for more appropriate solutions and settled on some basic usage of futures and promises from the <future> library. This seemed like the right answer since it handled the thread generation for me, and insured I move on when I actually get the callback rather than after some arbitrary period of time.
At first I thought things were going okay, but as I started building my patch I noticed some issues. In particular, an animation driven by [line] was stuttering. When investigating this I noticed that the [line] was being banged immediately after my external containing the asynchronous functions was banged.
When I disconnected my external, the animation ran smoothly.
So my hypothesis is that Max is still hanging while my object is waiting for the callback. It's just that with the <future> library, those hangs are so tiny that I didn't notice them until I was trying to run an animation at the same time.
I did take a look at the documentation on the Max Threading Model. But as far as I can tell, that doesn't seem to be relevant here. I'm taking a message or a dictionary in, performing some asynchronous functions, and putting out a bang, message, or dictionary. But none of this needs to be strictly scheduled vis a vis Max. And none of the internal state of my object is shared with any other component of Max. If I'm understanding correctly, my object should be completely thread agnostic as far as Max threads are concerned. I think the output will be perfectly happy to be deferred.
I need to find a way to handle my internal asynchronous functions in such a way that the rest of the patch can carry on while my object waits for it's callbacks.
Apologies again if the answer is obvious here... I am definitely learning as I go when it comes to this territory. I'll be happy to provide more details about my particular implementation if that will be helpful.
Thank you for any input!
Hi there,
Some more specific detail will probably help, because I'm not sure, concretely, what you mean by 'wait' at the moment. It sounds like you might be essentially blocking the Max main thread until your async function completes, which will indeed lock up the application and defeats the purpose of delegating work to a thread.
There's a couple of strategies for dealing with asynchronous completion. In either case, the function invoking the work would then return straight away, so that Max can get on with other stuff, and the code for dealing with completion becomes another function, added into the scheduling queue some time later.
So rather than the normal synchronous pattern:
f() {
result = doStuff(input);
outlet(result);
}
it could become
f_launch() {
doStuffAsync(input, callback);
}
or
f_launch(){
my_obj.result = doStuffAsync(input, callback);
}
and then feeding the result back to Max happens in another functionf_complete() {
outlet(result);
}
And the trick is getting f_complete() to invoke back on one of Max's threads (IIRC, accessing outlets from a non-Max thread is a no-no). The two choices are basically (a) worker thread invokes a callback when it's done, and this callback should then schedule work back on one of Max's threads, (b) you poll the worker thread periodically (e.g. by scheduling a periodic job on Max's queue), and when it's done, pass the result to the outlet.
I'm not sure which of these you're pursuing, because you mention both callbacks and futures. A callback function would leave you with the challenge of safely getting your result back into the Max thread. A future could be used to poll the worker thread periodically.
Disclaimer: I've not used Min yet, but it looks like for case (a) your object could have a 'queue' and the callback from the thread would call set on that to run a function back on the main thread. For case (b) you could use a 'timer', which would check whether the std::future was valid(), and if so get() the result and output it, otherwise reschedule itself for some number of milliseconds in the future to check again.
(a) might look something like
queue sendResult { this,
MIN_FUNCTION {
myOutlet.send(this.myResult);
return {};
}
};
void myCallback(MyObject& obj, Result& result){ //called from worker thread
//a remaining challenge is to make result available in main thread without a data race.
obj.result = result;
sendResult.set();
}(b) might look something like
timer<timer_options::defer_delivery> poll { this,
MIN_FUNCTION {
if myFuture.valid() {
auto result = myFuture.get();
myOutlet.send(myResult);
} else
poll.delay(interval);
return {};
}
};
Does that help at all?
Thank you so much for your reply Owen. I will need to some time to review this and understand it.
"It sounds like you might be essentially blocking the Max main thread until your async function completes, "
This definitely seems like what is happening. Your post will definitely help me understand the problem at the very least. I will post back when I've had a chance to review thoroughly and experiment some more.
Thanks!
I spent some time with this. It clarified my thinking a bunch but I don't think I solved the problem.
Here are a few more details: I'm working with an API that has a bunch of functions that take an indeterminate about of time and make callbacks when the job is done. The callback function usually takes some parameter that includes data I care about.
I want to send my object a message that will call one of these functions, or a chain of them, and output a some result at the end (either a bang, a message, or a dict). Since the functions take an indeterminate about of time, they do not drive audio or schedule-critical material. However, I do have audio, line objects, midi, and other schedule-critical material happening in my patch. I need a figure out how to work with these functions without interfering with the the rest of the patch.
My initial problem was a basic engineering issue. How do I get my program to wait for a callback before moving on? My first crude attempt was to Sleep for an arbitrary amount of time that I was sure was enough for the function to finish. In looking for a more refined solution, I found this post that suggested setting the value of a promise in the callback. And then getting the value of an associated future later on. Getting the value of the future would cause the thread to wait until the promise was set.
Seeing the confusion this caused Owen I started wondering if I was overcomplicating things. I decided to get rid of futures/promises and simplify. If I need to wait for a callback of function A before running function B, I can just call function B in the callback of function A. This feels much better to me.
So i have:
void functionACallback(int OutNumber){
//do something important with number
functionB();
}void functionA (int InNumber, functionACallBack){
//bunch of logic that takes a indeterminate amount of time, and ultimately calles functionACallback with an int as parameter
}functionB() {
//does more stuff that sets a variableresult that I want to post out my outlet
result = "post this out the outlet";
}outlet1.send(result);In some cases I have multiple functions chained together this way before arriving at a result I want to output.
Per Owen's post, I thought they key is just to get this all off the main thread. The key is " the function invoking the work would then return straight away."
So I wrote
queue<> start startTheChain { this,
MIN_FUNCTION{
functionA(1234567890);
outlet1.send("result");
return {};
}
};message<> anything{ this, "anything", "Starts the Nakama session.",
MIN_FUNCTION {
startTheChain();
return {};
}
};But I don't think this has made any change in the behavior.
Please ignore the mention about futures above. I refactored to get rid of this. I'm just calling the next function in the chain directly in the callback now.
If this is not the right answer, I'm not sure I understand what is...
It's possible it is working. I'm not sure how to test it actually. Previously I noticed I had a problem because mine [line] animations were glitching. When I bypassed all these asyncs the animations looked the way they should. Since I did change the internal logic to get rid of futures, the animations are glitching differently now.
It's possible those glitches are being caused by something else. But I imagine there's a better way to tell if I'm inadvertently hanging my Max patch than watching to see if [line] is filling my slider smoothly.
Slightly out of my depth here, but learning to swim deeper every time. Thanks for any input!!
Hi Joe, I've not closely at this thread, but I've done quite a bit of scheduling related code in Scheme for Max. My opinion (others may disagree) is that trying to add your own threading and timing system to a Max external is a path to madness. :-) The conventional wisdom in audio app programming is to try to get away with as few threads as possible. In Max, this means the audio thread, the high priority thread (aka scheduler) and the low thread (aka main or gui). If you have Audio In Interrupt selected, the audio thread and the high thread take turns, but they still need to be isolated properly. There are just two many weird corner cases and limitations you will hit trying to intermingle some other threading system with what Max is using and expecting.
What I think you want to do is stay on the Max rails, and handle the async stuff with Max clocks. The clock system is super accurate and very robust. In Scheme for Max I'm using it extensively and I can absolutely hammer the scheduler without crashes in my load tests. One thing you might want to know - the docs do say you shouldn't create a new clock inside a high-priority thread, but I'm doing that with no issues at all. It's how the high priority thread scheduling works in S4M and I have load tested the crap out of it, it's solid. The memory allocation for a clock is tiny. You do have to be very careful about freeing clocks and other memory allocated around them, but it's not too hard.
The major thing about using the clocks is that the queueing is all handled for you. So if you put a callback on a clock, when it fires, it can do whatever you want (no limitations around not accessing inlets and outlets and so on). If you want to see how I use them in Scheme for Max, the sources is all open. I have some tricky stuff in there to handle the fact that clocks expect one pointer, so I make structs for all my clock related pointers, and pass a void pointer to that to the clock. It was a bit of work to set up, but it's working like a charm.
Now all that said, I didn't do it in Min, I used the C SDK because it was more documented when I started and I wanted things to stay Pure Data compatible. But the clock system under the hood should be the same.
hth!
Further to this, I've had no problem with clock callbacks scheduling later clock callbacks. So for example, you can put function pointer on a clock, the function runs, it does some amount of work, and then puts a next call back on to a new (or the same clock). This essentially gives you a Max safe version of code that yields.
Hi Joe,
First: Iain is quite right that getting into stuff with custom threads can be pretty tricky, and if there's a way of avoiding it then it's seriously worth considering. Custom threads can work fine (we do a lot of it in the FluCoMa tools) but one does have to be very careful indeed, and there are lots of footguns just waiting to cause havoc!
If there's a way of breaking up your big chunk of work into a series of small function calls, then the method that Iain suggests of using the queue to perform manageable chunks whilst not blocking Max from doing other stuff is definitely easier and safer. An example of that in action can be seen in the min.progress example in the min.devkit.
Basically, if your expensive function has a loop in it, then you can just think in terms of delegating each iteration of the loop into a timer function, and keeping track of the overall state in your object.
Looking at what you posted above ,queue<> start startTheChain { this,
MIN_FUNCTION{
functionA(1234567890);
outlet1.send("result");
return {};
}
};
If functionA is the expensive thing, then you're still calling this synchronously on the main thread, so it will indeed block Max; i.e., there's nothing asynchronous there (e.g. using std::thread or std::async).
The trick would be to break that functionA up if you can. However, sometimes that isn't possible. Either because you're calling into a library where you don't control the iteration, or it's a network call, etc. In those situations, you probably do need to reach for a custom thread. Happy to make a basic example with a C++ thread if you decide that's the way you need to go, but I would agree with Iain that if you can find a way to avoid it, then this will be easier in the long run!
Thank you both for the input here! I some updates:
The time consuming function is from a third party library, so I can't really break it down any further.
I tried calling that function on it's own thread.std::thread t1(theBigFunction, this, callBackFunction)
If I'm understanding correctly, doing this should allow theBigFunction to take all the time it needs over on the side, while Max's main thread carries on running Max.
To my dismay, this did not solve my problem. After taking a much closer look at my patch, I realized that the animation glitching was actually caused by a patching error, not by my external. Yes the glitching stopped when I disconnected my external. But the real issue was a misplaced patchcord downstream from the external.
So my problem is solved, even when I have theBigFunction running directly in the main thread. It appears the folks who wrote the library did a good job of organizing their function so it can return promptly.
Thanks for all the input here. I am definitely a better developer because of it.
Hi Joe, another option here would be use the node.script object to farm work out to another process. The node.script is fundamentally asynchronous, and gives you access to system calls through all the normal node libraries. I've used that in the past alongside Scheme, where by I have my Scheme calls (which technically run in the Max threads as message-time C code) send messages to node to ask for outside work and then get notified by a callback when the node stuff is done, and this means the slow bits are completely outside of the Max process. Might be worth exploring. I suppose one could do this at a C/C++ level too, but it could be a lot more work than just using node.script.