Simon Blakely

I found some externals I had written several years ago no longer worked in Max 8, so I have started redeveloping them using 

, to solve all the project-related issues I have, and to make it easier for porting to other platforms I don't have. It certainly helps set everything up and makes inlets/outlets easier.

I'm struggling with threads and thread safety.

The external accumulates audio data until it reaches a certain size, and then carries out an analysis (including a Discrete FFT) on those frames. In the past, I just let the analysis occur in the "process" function (audio thread). The 

 guide suggests that this is not the best approach.

I believe the process to follow is to accumulate the samples in the audio thread, and once I have enough samples, use a timer to schedule the analysis phase in either the high priority scheduler thread or the low priority main thread. Can I use a fifo for this (which would be threadsafe, but I have performance concerns) or can I use a vector (which I suspect will be faster) but will need to ensure that the audio thread does not overwrite the data before it has been analysed. If I have a 512 sample frame, and make the vector 1024 samples, the audio thread can write to one half, while the analysis copies the other and starts analysis. Is this sufficiently threadsafe?

I also guess I shouldn't use the scheduler thread for the analysis.

I found some externals I had written several years ago no longer worked in Max 8, so I have started redeveloping them using min-devkit, to solve all the project-related issues I have, and to make it easier for porting to other platforms I don't have. It certainly helps set everything up and makes inlets/outlets easier.

The external accumulates audio data until it reaches a certain size, and then carries out an analysis (including a Discrete FFT) on those frames. In the past, I just let the analysis occur in the "process" function (audio thread). The min-devkit guide suggests that this is not the best approach.

I believe the process to follow is to accumulate the samples in the audio thread, and once I have enough samples, use a timer to schedule the analysis phase in either the high priority scheduler thread or the low priority main thread. Can I use a fifo for this (which would be threadsafe, but I have performance concerns) or can I use a vector (which I suspect will be faster) but will need to ensure that the audio thread does not overwrite the data before it has been analysed. If I have a 512 sample frame, and make the vector 1024 samples, the audio thread can write to one half, while the analysis copies the other and starts analysis. Is this sufficiently threadsafe?

I found some externals I had written several years ago no longer worked in Max 8, so I have started redeveloping them using min-devkit, to s

min-devkit-and-threading-questions

Just to answer my own question - I went with a 

 because according to the author, the fifo implementation used in min-devkit is

So each frame is 512 samples, and using a vector, you get 64 samples per call of 

stick samples in the fifo until there are at least 512 samples.

Then use a timer (deliverer.delay) to shift to the scheduler thread, and a defer to drop to the main low-priority thread. There you can take samples from the fifo and put them into a vector of 512, and process the audio and output the analysis.

void operator()(audio_bundle input, audio_bundle output) {
        auto    in = input.samples(0);    // get vector for channel 0 (first channel)
        for (auto i = 0; i < input.frame_count(); ++i) {
            audio_fifo.try_enqueue(in[i]);
            if (currentsamples < frameSize - 1) {
                currentsamples++;
            }
            else {
                deliverer.delay(0);
                currentsamples = currentsamples - frameSize;
            }
        }
    };
    ...
    timer<> deliverer{ this,
    MIN_FUNCTION {
        deferrer.set();
        return {};
    }
    };

    queue<> deferrer{ this,
    MIN_FUNCTION {
        drain_audio_fifo_to_frame();
        return {};
    }
    };
    
    void drain_audio_fifo_to_frame() {
        int samples = 0;
        number sample;

        while (samples < frameSize) {
            audio_fifo.try_dequeue(sample);
            frame[samples] = sample;
            samples++;
        };
        ...

Just to answer my own question - I went with a fifo because according to the author, the fifo implementation used in min-devkit is 

So each frame is 512 samples, and using a vector, you get 64 samples per call of operator in the audio thread.
stick samples in the fifo until there are at least 512 samples.
Then use a timer (deliverer.delay) to shift to the scheduler thread, and a defer to drop to the main low-priority thread. There you can take samples from the fifo and put them into a vector of 512, and process the audio and output the analysis.  

void operator()(audio_bundle input, audio_bundle output) {
        auto	in = input.samples(0);	// get vector for channel 0 (first channel)
        for (auto i = 0; i < input.frame_count(); ++i) {
            audio_fifo.try_enqueue(in[i]);
            if (currentsamples < frameSize - 1) {
                currentsamples++;
            }
            else {
                deliverer.delay(0);
                currentsamples = currentsamples - frameSize;
            }
        }
    };
    ...
    timer<> deliverer{ this,
    MIN_FUNCTION {
        deferrer.set();
        return {};
    }
    };

    queue<> deferrer{ this,
    MIN_FUNCTION {
        drain_audio_fifo_to_frame();
        return {};
    }
    };
    
    void drain_audio_fifo_to_frame() {
        int samples = 0;
        number sample;

        while (samples < frameSize) {
            audio_fifo.try_dequeue(sample);
            frame[samples] = sample;
            samples++;
        };
        ...

Hi, I'm studying threading using min-sdk, and I am feeling a bit lost.

I am having a function that access an external program and collects some data that will be returned as a string. This takes time, and while waiting for it Max freezes for a couple seconds. I'd imagine that I have to do something with threading, but I don't really understand the 

. It seems that it is mostly concerned about having things arriving in time and faster. For my project I really don't care about it being fast, I know it won't, I just don't want to have Max freezing. Could you give me some help understanding this?

Hi, I'm studying threading using min-sdk, and I am feeling a bit lost.
I am having a function that access an external program and collects some data that will be returned as a string. This takes time, and while waiting for it Max freezes for a couple seconds. I'd imagine that I have to do something with threading, but I don't really understand the guide. It seems that it is mostly concerned about having things arriving in time and faster. For my project I really don't care about it being fast, I know it won't, I just don't want to have Max freezing. Could you give me some help understanding this?
Thank you for your attention.

answering myself (kind of a sad trend here):

I needed to use threadsafety, in my case, since I'm using a message to trigger this whole thing

Which will allow me to wrap the problematic part of me function with

Also, massive thanks to Timothy Place for 

answering myself (kind of a sad trend here):
I needed to use threadsafety, in my case, since I'm using a message to trigger this whole thing
message<threadsafe::yes>
Then, I needed to make sure I have a mutex m_mutex;Which will allow me to wrap the problematic part of me function with
lock lock {m_mutex};
(... le function dangereuse)
lock.unlock();I'm so happy I figure it out :D  (edit: I didn't)
Also, massive thanks to Timothy Place for this video and his work on min-sdk!

So, my problem now is the following: I need to send messages with a "del 0" or "pipe 0" and with overdrive turned on for this to work. What is the way I can force any incoming message (for examples the ones on the main thread via interacting with a message box with a mouse click) to be turned into the scheduler from within min-sdk without relying on external pipe or delay objects?

Turns out, while I was happy moving objects around while the process was going on this other thread, in reality if I needed to use any timing sensitive objects at the same, like metro, which use the scheduler thread, now they get all bogged down... So, back to square one.

Anyone can help me figure out threading in c++ with min-sdk?

Turns out, while I was happy moving objects around while the process was going on this other thread, in reality if I needed to use any timing sensitive objects at the same, like metro,  which use the scheduler thread, now they get all bogged down... So, back to square one.
Anyone can help me figure out threading in c++ with min-sdk?

a lack of replies often times indicates a complex question. threading has no simple answer. It sounds to me like your case is less concerned with Max's internal threading implementation (as described in Tim's guide better than probably most anyone can on this topic), and more a case of needing to run a task in a background thread. Since min is c++ based, your topic is probably more generally related to threading in c++ with std::thread, so here is one resource among many to browse - 

https://livebook.manning.com/book/c-plus-plus-concurrency-in-action/chapter-2/

a lack of replies often times indicates a complex question. threading has no simple answer. It sounds to me like your case is less concerned with Max's internal threading implementation (as described in Tim's guide better than probably most anyone can on this topic), and more a case of needing to run a task in a background thread. Since min is c++ based, your topic is probably more generally related to threading in c++ with std::thread, so here is one resource among many to browse - https://livebook.manning.com/book/c-plus-plus-concurrency-in-action/chapter-2/ 

So any function that you need to run that takes a long time needs to be in a function that is triggered by a timer or in a queue (lowest priority).

The low-priority function you run needs to use thread-safety to prevent it being re-entered if it is not re-entrant.

If you need to interact with high-priority objects from this low-priority function, you need to send a custom message or trigger a timer function to be processed in the timer thread to take an action. This function does not execute immediately - it either executes at the next timer slice or may be delayed further. Your low priority thread may need to set and wait on a semaphore to get the signal to proceed when the timer-level function has executed, or it may be able to continue executing regardless.

I don't know if this helps or not - maybe provide some pseudo-code that illustrates the problem you are having.

@Simon Thank you so much for getting back :)

Here is the code I'm trying to use (adapted from 

@Rob thx for the reference
@Simon Thank you so much for getting back :)
Here is the code I'm trying to use (adapted from here):

void exec(const char* cmd) {
		std::array<char, 128> buffer;
		std::string result;
		std::unique_ptr<FILE, decltype(&pclose)> pipe(popen(cmd, "r"), pclose);
		if (!pipe) {
			std::cerr<<"failed to execute code"<<std::endl;
			throw std::runtime_error("popen() failed!");
		}
		while (fgets(buffer.data(), buffer.size(), pipe.get()) != nullptr) {
			result += buffer.data();
		}
		cout<<result<<endl;
//		output.send(result);
		return;
	}

That is just a function. How are you fitting this into the min template code?

From your min template code, you would run this in the low priority thread by breaking out of the audio priority thread with a 

 function to get to the scheduler thread, then a 

But the audio thread cannot wait for the call to complete (nor the 

 should send the result as a message back into the object.

void operator()(audio_bundle input, audio_bundle output) {
        auto    in = input.samples(0);    // get vector for channel 0 (first channel)
        // call the external function
          deliverer.delay(0);
    };
    ...
    timer<> deliverer{ this,
    MIN_FUNCTION {
        deferrer.set();
        return {};
    }
    };

    queue<> deferrer{ this,
    MIN_FUNCTION {
        exec("some command");
        return {};
    }
    };
    
 void exec(const char* cmd) {
        std::array<char, 128> buffer;
        std::string result;
        std::unique_ptr<FILE, decltype(&pclose)> pipe(popen(cmd, "r"), pclose);
        if (!pipe) {
            std::cerr<<"failed to execute code"<<std::endl;
            throw std::runtime_error("popen() failed!");
        }
        while (fgets(buffer.data(), buffer.size(), pipe.get()) != nullptr) {
            result += buffer.data();
        }
        cout<<result<<endl;
//        output.send(result);
        return;
    }
        ...

From your min template code, you would run this in the low priority thread by breaking out of the audio priority thread with a timer function to get to the scheduler thread, then a queue to get to the lowest priority thread.

But the audio thread cannot wait for the call to complete (nor the timer). 
If you want a return value, exec should send the result as a message back into the object.

void operator()(audio_bundle input, audio_bundle output) {
        auto	in = input.samples(0);	// get vector for channel 0 (first channel)
        // call the external function
          deliverer.delay(0);
    };
    ...
    timer<> deliverer{ this,
    MIN_FUNCTION {
        deferrer.set();
        return {};
    }
    };

    queue<> deferrer{ this,
    MIN_FUNCTION {
        exec("some command");
        return {};
    }
    };
    
 void exec(const char* cmd) {
		std::array<char, 128> buffer;
		std::string result;
		std::unique_ptr<FILE, decltype(&pclose)> pipe(popen(cmd, "r"), pclose);
		if (!pipe) {
			std::cerr<<"failed to execute code"<<std::endl;
			throw std::runtime_error("popen() failed!");
		}
		while (fgets(buffer.data(), buffer.size(), pipe.get()) != nullptr) {
			result += buffer.data();
		}
		cout<<result<<endl;
//		output.send(result);
		return;
	}
        ...

Ok, so there is something I am still missing because it is still freezing max while it gets processed. In my case I'm not using a dsp object, so I don't need the "void operator()". Right now I am simply trying to trigger it from a "message<>"

But I was having so many problems with this that I decided to try to isolate it, just to see if I could figure this out. I will want to go back to that return string mode.

If you want a return value, exec should send the result as a message back into the object.

Yes, at first I was using "string exec() { ... return result; }". But I was having so many problems with this that I decided to try to isolate it, just to see if I could figure this out. I will want to go back to that return string mode.

So, right now my "message<>" looks like:

message<threadsafe::yes> anything { this, "anything", "run my command plz.", MIN_FUNCTION {
		std::string str = p;// this is path of my external program
		string input = args[0]; // the command to pass to my external program
		command = str+input; // put it together
		const char *cstr = command.c_str();
//		exec(cstr); // calling it directly, hangs...
//		deferrer.set(); // calling it via queue, hangs...
//		deliverer.delay(0); // calling via timer, hangs...
//		string result = exec(cstr); // eventually I want to be able to receive it back here instead of printing directly
		return {};
	}
};

Again, your function calls are events - you start something.

Then you use another mechanism (timer or message) to check for a result.

You either use a threadsafe data-structure (like a fifo) to pass the result from **exec** to your processing loop and pick it up from there using a timer, or in **exec** you create a new message with the result that is sent back to your object. Or you can use an attribute and the attribute notify mechanism.

There are many different ways to do this.

You then take appropriate action on that new message to process the result.

This uses a fifo, but is example code only - I haven't checked it throughly.

But I used fifos to move Audio data from the Audio thread to a processing thread that runs as a deferrer function, and it works well. And then I use a fifo to push Audio samples back.

     // fifo to store results
     fifo<string>    results { 10 };
        
 void operator()(audio_bundle input, audio_bundle output) {
        auto    in = input.samples(0);    // get vector for channel 0 (first channel)
        // call the external function
          deliverer.delay(0);
    };
    ...
    
    // checkresult is run every 500ms to check for external program results
    timer<> checkresult< this,
    MIN_FUNCTION {
        while (results.count() > 0) {
          do 
            <
              thisresult = results.try_dequeue();
              do something with thisresult, probably via another deferrer
            >
        }
        // check again in 500 ms
        checkresult.delay(500)
      }
    
    timer<> deliverer{ this,
    MIN_FUNCTION {
        deferrer.set();
        return {};
    }
    };

    queue<> deferrer{ this,
    MIN_FUNCTION {
        exec("some command");
        return {};
    }
    };
    
 void exec(const char* cmd) {
        std::array<char, 128> buffer;
        std::string result;
        std::unique_ptr<FILE, decltype(&pclose)> pipe(popen(cmd, "r"), pclose);
        if (!pipe) {
            std::cerr<<"failed to execute code"<<std::endl;
            throw std::runtime_error("popen() failed!");
        }
        while (fgets(buffer.data(), buffer.size(), pipe.get()) != nullptr) {
            result += buffer.data();
        }
        cout<<result<<endl;
        results.try_enqueue(result);
        return;
    }
        ...

Again, your function calls are events - you start something.
Then you use another mechanism (timer or message) to check for a result.
When you have a result, use it. 

You either use a threadsafe data-structure (like a fifo) to pass the result from **exec** to your processing loop and pick it up from there using a timer, or in **exec** you create a new message with the result that is sent back to your object. Or you can use an attribute and the attribute notify mechanism.
There are many different ways to do this.

This uses a fifo, but is example code only - I haven't checked it throughly.
But I used fifos to move Audio data from the Audio thread to a processing thread that runs as a deferrer function, and it works well. And then I use a fifo to push Audio samples back.

     // fifo to store results
     fifo<string>    results { 10 };
        
 void operator()(audio_bundle input, audio_bundle output) {
        auto	in = input.samples(0);	// get vector for channel 0 (first channel)
        // call the external function
          deliverer.delay(0);
    };
    ...
    
    // checkresult is run every 500ms to check for external program results
    timer<> checkresult< this,
    MIN_FUNCTION {
        while (results.count() > 0) {
          do 
            <
              thisresult = results.try_dequeue();
              do something with thisresult, probably via another deferrer
            >
        }
        // check again in 500 ms
        checkresult.delay(500)
      }
    
    timer<> deliverer{ this,
    MIN_FUNCTION {
        deferrer.set();
        return {};
    }
    };

    queue<> deferrer{ this,
    MIN_FUNCTION {
        exec("some command");
        return {};
    }
    };
    
 void exec(const char* cmd) {
		std::array<char, 128> buffer;
		std::string result;
		std::unique_ptr<FILE, decltype(&pclose)> pipe(popen(cmd, "r"), pclose);
		if (!pipe) {
			std::cerr<<"failed to execute code"<<std::endl;
			throw std::runtime_error("popen() failed!");
		}
		while (fgets(buffer.data(), buffer.size(), pipe.get()) != nullptr) {
			result += buffer.data();
		}
		cout<<result<<endl;
		results.try_enqueue(result);
		return;
	}
        ...

Simon, thank you so much for all the help. I feel that I am close, but I am just not getting there yet.

Do I need to use that "void operator()" if I am not using audio samples? Who starts the checkresult timer?

private:
    string command; // setting it here, so that I can easily pass it around.
    fifo<string>    results { 10 };
public:
    string dequeue; // using this to pass result to deferrer2
    
    // checkresult is run every 500ms to check for external program results
    timer<> checkresult { this, MIN_FUNCTION {
        while (results.size_approx() > 0) {
            results.try_dequeue(dequeue); // had to change this...
            //do something with thisresult, probably via another deferrer
            deferrer2.set();
        }
        // check again in 500 ms
        checkresult.delay(500);
    }};
 
     timer<> deliverer { this, MIN_FUNCTION {
        deferrer.set();
        return {};
    }};
    
    queue<> deferrer{ this, MIN_FUNCTION {
        const char *cstr = command.c_str(); // command retrieved from private
        exec(cstr);
        return {};
    }};
 
     queue<> deferrer2{ this, MIN_FUNCTION {
        output.send(dequeue);
        return {};
    }};
 
     void exec(const char* cmd) {
        output.send(dequeue);
        return {};
        std::array<char, 128> buffer;
        std::string result;
        std::unique_ptr<FILE, decltype(&pclose)> pipe(popen(cmd, "r"), pclose);
        if (!pipe) {
            std::cerr<<"failed to execute code"<<std::endl;
              throw std::runtime_error("popen() failed!");
           }
        while (fgets(buffer.data(), buffer.size(), pipe.get()) != nullptr) {
            result += buffer.data();
        }
//           cout<<result<<endl; // the idea is to queue it not to try to print it here now.
           results.try_enqueue(result);
           return;
       }

    message<threadsafe::yes> anything { this, "anything", "shell.", MIN_FUNCTION {
        std::string str = p;// path
        string input = args[0];
        command = str+input; 
//        deliverer.delay(0); // I probably need to start deliverer timer here, no?
//        checkresult.delay(500); // do I need to start checkresult timer here?
        return {};
    }};

Simon, thank you so much for all the help. I feel that I am close, but I am just not getting there yet.
Do I need to use that "void operator()" if I am not using audio samples? Who starts the  checkresult timer?
Here is what I have:

private:
	string command; // setting it here, so that I can easily pass it around.
	fifo<string>    results { 10 };
public:
	string dequeue; // using this to pass result to deferrer2
	
	// checkresult is run every 500ms to check for external program results
	timer<> checkresult { this, MIN_FUNCTION {
		while (results.size_approx() > 0) {
			results.try_dequeue(dequeue); // had to change this...
			//do something with thisresult, probably via another deferrer
			deferrer2.set();
		}
		// check again in 500 ms
		checkresult.delay(500);
	}};
 
 	timer<> deliverer { this, MIN_FUNCTION {
		deferrer.set();
		return {};
	}};
	
	queue<> deferrer{ this, MIN_FUNCTION {
		const char *cstr = command.c_str(); // command retrieved from private
		exec(cstr);
		return {};
	}};
 
 	queue<> deferrer2{ this, MIN_FUNCTION {
		output.send(dequeue);
		return {};
	}};
 
 	void exec(const char* cmd) {
		output.send(dequeue);
		return {};
		std::array<char, 128> buffer;
		std::string result;
		std::unique_ptr<FILE, decltype(&pclose)> pipe(popen(cmd, "r"), pclose);
		if (!pipe) {
			std::cerr<<"failed to execute code"<<std::endl;
		  	throw std::runtime_error("popen() failed!");
	   	}
		while (fgets(buffer.data(), buffer.size(), pipe.get()) != nullptr) {
			result += buffer.data();
		}
//	   	cout<<result<<endl; // the idea is to queue it not to try to print it here now.
	   	results.try_enqueue(result);
	   	return;
   	}

	message<threadsafe::yes> anything { this, "anything", "shell.", MIN_FUNCTION {
		std::string str = p;// path
		string input = args[0];
		command = str+input; 
//		deliverer.delay(0); // I probably need to start deliverer timer here, no?
//		checkresult.delay(500); // do I need to start checkresult timer here?
		return {};
	}};

That is pretty much looking like I'd expect it to. It looks like you are getting the idea.

I can't remember if there is an initialization function (almost certainly in the class constructor) where you can start the checkresult timer, or you can drive it off the message as you have done. You could also use a class boolean to not reschedule the timer if you have already returned a result - all sorts of different ways to manage that process.

I've been struggling with a similar problem to OCH above: I have a slow function that freezes Max for ~2-3 seconds when I run it from perform().

(it's a LibTorch call; I'm working on a fork of 

 which should query torch occasionally rather than every frame. BTW, using std::thread is 

- moving the function call to a queue<> : doesn't work, the execution still pauses until the model finishes executing

- moving it to a timer<timer_options::defer_delivery>: this defers it indefinitely, I have to turn off audio for it to get around to executing (and again it freezes Max)

as a flag to check periodically in a timer: again, doesn't keep Max from freezing.

What am I missing? The source code is a lot, but it boils down to something like:

void myObject::operator()(audio_bundle input, audio_bundle output) {

model_perform(); // slow, needs to run on the main thread!

How do I move that function call to the low-priority thread, without having it block Max and without having it wait for the audio to be off to execute? I'll deal with getting the results later, for now I just need it to run...

(it's a LibTorch call; I'm working on a fork of nn~ which should query torch occasionally rather than every frame. BTW, using std::thread is not an option here)

I've tried all the methods mentioned:
- moving the function call to a queue<> : doesn't work, the execution still pauses until the model finishes executing
- moving it to a timer<timer_options::defer_delivery>: this defers it indefinitely, I have to turn off audio for it to get around to executing (and again it freezes Max)
- using something like:fifo<bool> ready {1};
as a flag to check periodically in a timer: again, doesn't keep Max from freezing. What am I missing? The source code is a lot, but it boils down to something like:

void myObject::operator()(audio_bundle input, audio_bundle output) {
...
model_perform(); // slow, needs to run on the main thread!
}

How do I move that function call to the low-priority thread, without having it block Max and without having it wait for the audio to be off to execute? I'll deal with getting the results later, for now I just need it to run...


OK, I think I figured it out... a blocking function 

, even if it's running deferred in the main thread. Like, forget the LibTorch nonsense above, if I simply call a sleep function:

std::this_thread::sleep_for(std::chrono::milliseconds(1000));

it doesn't matter what thread we're in, Max will freeze for a second. The only way to avoid this is to use std::threads properly.

so I really wonder if @OCH got his code to work eventually... seems to me like without embedding exec() into a std::thread, it's impossible to move it to the background.

...which now still leaves me with the problem that LibTorch 

 on Windows. My first rough tests don't show any memory usage buildup though...

OK, I think I figured it out... a blocking function will always block Max, even if it's running deferred in the main thread. Like, forget the LibTorch nonsense above, if I simply call a sleep function: 

...which now still leaves me with the problem that LibTorch seems to leak on Windows. My first rough tests don't show any memory usage buildup though...

min-devkit and threading questions