timeout error in long N4M process


    Sep 25 2020 | 1:22 pm
    Hi,
    I'm working on a Max patch that needs to trigger a long process in a Node 4 Max script. My process end with a timeout error ("Max API Message Request timed out. id: u39633390420").
    Is there a way to fix this ? (I suppose that this timeout is a protection but is there a way to override it ?).
    I've attached the patch. You have to add a long WAV sound file (mine is 6min long) in the directory and to modify the filename in the path message.
    Regards,
    Jérémie

    • Sep 30 2020 | 9:38 am
      It looks like the solution is to change the reqTimeout parameter to something longer than 3000 in file /Applications/Max.app/Contents/Resources/C74/packages/Node\ For\ Max/source/lib/api/index.js (line 27).
      Is there a way to set this param directly from the node script (to avoid the modification of core Node 4 Max files) ?
    • Oct 05 2020 | 6:13 pm
      Hi there,
      looking at the patch and code I think the issue is actually within your application code and not necessarily the timeout itself. Maybe we need to make it a bit more clear that max.post is an asynchronous call but your for-loop is basically blocking and all the bi-directional communication is only "verified" as successful once you left the loop. With that in mind, even a bigger timeout is more of a "guess" depending on whatever the input size / length of the loop is.
      Instead you should be able to fix the error by altering your code to await the call to post. Easiest might be to use async / await here. Adding the adjusted code here to help others without a need to download (if that's problematic please let me know):
      const max = require('max-api');
      const essentia = require('essentia.js');
      const load = require('audio-loader');
      
      max.addHandler("path", async (pathname, combine, frameSize, hopSize, ratioThreshold, sampleRate, threshold) => {  
        load(pathname).then(async function(buffer){
          await max.post("loaded");
          const channel = buffer.getChannelData(0);
          const inputSignalVector = essentia.arrayToVector(channel);
          await max.post("vectorized");
          const onset = essentia.SuperFluxExtractor(inputSignalVector, combine, frameSize, hopSize, ratioThreshold, sampleRate, threshold);
          await max.post("onsets done");
          const ticks = essentia.vectorToArray(onset.onsets);
          await max.post("toArray");
          for(var c = 0; c < ticks.length; c++) {     
            await max.outlet(c, ticks[c]);
          }
        });
        await max.post("loading");
      });
    • Oct 05 2020 | 6:55 pm
      Hi Florian,
      Thanks a lot for your answer and solution ! If I understand well, I was thinking that the problem was coming from the long call to "SuperFluxExtractor", but in fact it was due to the time taken by the communication between Node and Max via the max-api and all the max.outlet() calls. With the await on max.outlet(), all calls are done, but the results arrive when they can... (without timeout).
      Am I right ?
      Regards,
      Jérémie
    • Oct 06 2020 | 11:24 am
      Hi Jérémie,
      the long call to SuperFluxExtractor shouldn't really be a problem. The timeout does not apply to the callback of a handler. So even if your code in there takes minutes to run it's not a problem and it really shouldn't be either. The issue is that your for-loop is flooding the message queue and basically the communication times out. There are probably ways for us to handle that case, buffer internally within the API, introduce more complex forms of parallel execution and messaging and with that mask the potential issue for the API user. However, given that we are dealing with interprocess communication the sentiment that basically everything is asynchronous per nature is important for the application code as well and you cannot necessarily 100% rely on messages being output / printed by the node.script object in order unless you use await with your calls to max.post and max.outlet. In your specific case it ensures that the call within the for-loop is done fully sequentially and the results / messages are sent and received as expected.
      The timeout that you tracked down in the internals of the max-api is in place in order to ensure that the communication is stable between the two processes at this point. The way we do / ensure that might change in the future but I'd refer to the user facing API and its asynchronous nature.
      Does this clear up things a bit? Let me know if u have further questions.
      Florian