Best way to implement long-running operation with C API?

    Travis Kaufman
    Apr 30 2013 | 10:24 pm
    Hey all,
    I've been working on a max external that provides an interface to cURL. The idea is that you send a message to the object (such as a bang) and the external will scrape the web data for you. Unlike JWeb, this is headless and in pure C.
    My problem is that the cURL operation is a long-running operation (network request) and it seems that even when I use defer(), defer_low(), or qelems it blocks UI events. For example, if I were to attach a bang object to my MaxCurl and then click bang, there would be a delay in the response of the object until the curling process had finished.
    So I'm wondering what the best way to overcome this problem would be, which is to make sure that any UI events that need to be executed take priority over the cURL operation. I believe (?) that MSP/Jitter have their own separate T.O.Es so I'm not so concerned about them. Is the only way to get around this to use a systhread?
    You can check out the code on github. I've been developing it using Max5 b/c I've been remiss in upgrading. If you want to test it out in a patch try putting it on your file path and typing "MaxCurl http://www.google.com". BE WARNED That if you try to pass a very large body of text into any object that displays text besides a comment (i.e. a message object, a textedit object, etc.) max will segfault, so be careful!
    Thanks so much in advance for your help!
    Best, ~Travis
    • Jeremy
      Apr 30 2013 | 11:06 pm
      Hard to know without spending some time with the code (thanks for the link, if I have a sec I'll check it out), but unless you want to use fork() and exec(), communicating via pipes with your spawned process, you'll probably want to check out the ext_systhread.h header, as you suggested.
      Best, Jeremy
    • Jeremy
      Apr 30 2013 | 11:19 pm
      OK, I found a sec. You need to read the section about the multi interface @ http://curl.haxx.se/libcurl/c/libcurl-tutorial.html -- this is exactly what that interface is for (well, one of the things). The easy interface that you're using is synchronous, so unless you work in a background thread (potentially an option, but maybe it's extra trouble that you can easily avoid by using the multi interface), it's never going to do what you want.
      The multi interface is asynchronous -- after adding your easy handles to it, you call curl_multi_perform(), which is non-blocking. So you can call it over and over again on a qelem until it's done. And then you're done. I think it should be a pretty simple rewrite and provide precisely what you're going for.
      Jeremy
    • Travis Kaufman
      May 01 2013 | 2:49 am
      Thanks so much for your quick reply, Jeremy! I'll try out curl multi and report my results.