Integrating Lua into a patcher

Suns_and_Zeroes's icon

Hi all,

I have a few specific questions about using the lua jitter object in a patch if anyone could help enlighten me. Ive looked all around and cant find any actual documentation on the Max API for lua. So:

1: Threading - most importantly - when a message reaches the lua object, is the lua script put in the low-priority cue, like the JS object, or does it run its code synchronously with max? To get more specific, if I send a message out of a lua obj outlet, will it execute that output in max and then continue running the script, like externals coded in C? Or will it just keep on executing itself while the Max world does its own thing?

and less involved:

2: what are the basic max-related commands, like sending something out an outlet or taking a function call in from an outlet? Cant seem to find this

Thanks for reading

Suns_and_Zeroes's icon
Brian Kirkbride's icon

I'm interested in this too. I've done some testing, which seems to indicate:

  • jit.gl.lua seems to run in high priority when receiving high priority events

  • Outlets seems to be sent immediately and then subsequent Lua code is run

I'd love confirmation of this. Here's the patch I used for testing:

Max Patch
Copy patch and select New From Clipboard in Max.

Because the Lua and JS code aren't copied to the clipboard, here are those files:

luatesting.lua
this.inlets = 1
this.outlets = 1
function dotest()
    outlet(0,123)
    print("Sent output")
end

jstesting.js
this.inlets = 1;
this.outlets = 1;
function bang() {
    outlet(0,123);
    post("Sent output\n");
}

Brian Kirkbride's icon

Oh, and I meant to reply to OP to say:

  • To output something from [jit.gl.lua] use the "outlet" function like the above code.

  • To receive input, define a function named the message you want like "dotest" above. You can also define "int" or "float" functions to handle those types of inputs.

  • For some reason, "bang" as a function name didn't handle bangs input to the Lua object. I had to send "call bang" as a message. You don't need to use "call dotest" BTW, sending a message of just "dotest" works. Not sure why it doesn't work for bang.

  • This is all analogous to the [js] object.

Suns_and_Zeroes's icon

Oh man, thank you so much!

I'll do a little testing and get back soon, but the main reason I wanted to try Lua out was to see if it performs better than the JS object as Lua is integrated more closely with C and I often run into performance issues when I pass a lot of messages over the JS-MAX barrier.

But anyway, confirmed - patch works perfectly, and thanks for the explanatory comments in there - finally a working Lua patch on the forums!

EDIT: I actually get different outputs from the Lua object based on which defer setting is used - on my machine the outputs from the JS and Lua objects are identical in nature based on which [defer] is used. What does this mean exactly?

EDIT: since it still comes out the [print R] first after going through [defer], does this mean it comes out in the low priority?

Suns_and_Zeroes's icon

Oh yes - I wanted to ask about the [delay 0] object in there- I've heard about that before. Could you please explain why this promotes it to the high priority, and I guess UI triggered events are defaulted to low priority? This seems counter-intuitive to me, but maybe its to prevent disruption of the audio processing?

Brian Kirkbride's icon

You're welcome. Regarding [delay 0], I saw this in the excellent event priority article:

There may also be instances when you want to move a low priority event to a high priority event, or make use of the scheduler for setting a specific time at which an event should execute. This can be accomplished by using the delay or pipe objects. Note that only the high priority scheduler maintains timing information, so if you wish to schedule a low priority event to execute at a specific time in the future, you will need to use the delay or pipe objects connected to the defer or deferlow objects.

Brian Kirkbride's icon

I actually get different outputs from the Lua object based on which defer setting is used - on my machine the outputs from the JS and Lua objects are identical in nature based on which [defer] is used. What does this mean exactly?

So, this is pretty tricky to explain. My thinking is that a high-priority event going into [defer] and [deferlow] should be treated identically. But a low-priority event going into [defer] executes immediately while a low-priority event going into [deferlow] will be placed at the end of the low-priority queue.

On my machine (Mac), I see the same result for [js] with [defer] and [js] with [deferlow] BUT [jit.gl.lua] does something different depending on whether I use [defer] or [deferlow]. To me this indicates that Lua can output high-priority events. It may be that having "Overdrive" on is required. I believe you need that to have separate high- and low-priority queues.

Suns_and_Zeroes's icon

You're right, overdrive was off. I now see the difference in behavior between the two objects. Thank you!

Suns_and_Zeroes's icon

I'm still curious about performance though, so I'll post a testing patch soon

Brian Kirkbride's icon

Great, I'm interested in seeing your results. There was discussion about using LuaJIT in Max, but I don't know if that ever happened. May be a slower implementation of Lua.

Suns_and_Zeroes's icon

js-vs-lua_001.maxpat
text/plain 33.61 KB

So here is a testing patch for Lua vs. JS. It's mostly focused on the computational expense of crossing over the border from the language into the MAX environment. It captures and averages how long it takes each language to send 15,000 bangs out an outlet to an object in MAX. Press the red button in the middle to do the test on both. Code for both objects below::

CODE FOR lua-speed-test.lua:

this.inlets = 1
this.outlets = 2

function testfunc()

    i = 1

    while i < 15000 do
        outlet(1, "bang")
        i = i + 1
    end

    outlet(0, "bang")
end

CODE FOR js-speed-test.js:

outlets = 2;

function testfunc(){
    i = 1;

    while (i < 15000) {
        outlet(1, "bang");
        i = i + 1;
    }

    outlet(0, "bang");

}

I suggest trying it out before reading further. The [pgate] in the top right will flip the results when flipped to the left outlet and I have no idea why. I think maybe something weird is going on in the MAX environment and maybe [timer] is not the most accurate solution.

Anyway, Lua seems about three times as fast as JS when crossing the code-MAX barrier(!), but I could be wrong about this because this changes when seemingly random other parts of the patch are changed as described above.

Spa's icon

Just some comments.

Due to the limitations of Max/jitter that is mainly single threated and the fact you can use 2 threads (low and main), I tried to find solutions since years for using not only 15% of the cpu of my computers in max.
From my point of view, the solution by order of efficiency is:

_ separate process as different applications. for example, if you've got a processing of leapmotion or kinect data, build an independant application in max and just send the result in realtime to the main max app with osc. or with syphon or spout if you want to send a processed texture.

If you need efficient number crunching in max, my recomendation is to use mxj/java. why?
Even if java seems outdated, it is :
_ far far faster that javascript
_ it can process in high priority threat
_ it can even fork a process to an other new thread (then you bring back the result in the main thread)
_ its not as 'complicate' as C object
_ it can copy and process jit matrix by vectors (group of values in each cell, like rgba), javascript can not.

If we take a process in C object in max as reference,
java will be 2 to 4 times slower.
javascript will be 100 times slower, yes 100 ...
lua is more by the slow javascript side.

For me the support of Lua is really too limited.

Try mxj with your script, you will be surprised. Do not print anything to the console or number UI, during your loop or it will kill the speed.

I use it mainly to:
process numbers in high priority thread
move datas (like skeletons points) to matrices

Javascript, for me, can and is really good if you instantiate jitter objects, anim, etc. but in fact the real processing is done in C objects by max, javascript is only scripting the structure.

Suns_and_Zeroes's icon

Thank you for that, lots of interesting stuff in there. I really like the idea of farming out certain tasks to other .exe's. I I know that Java is generally faster, however what I am more interested in is how efficiently messages can cross the barrier between code languages and MAX.

In a test I did with [mxj] and [js], it seemed like it took longer for messages to cross from Java into Max, though to be fair I was using Jruby. The speed of the language itself does not matter to me, as the speeds of all of the languages are light-speed compared with the time it takes them to cross the barrier into the Max Environment- I am looking for the implementation that does this the best without having to deal with writing externals in C. Also I don't like having to have the JVM installed and spinning for the program to work.

I am liking what I am seeing so far- Lua seems to operate in the high-priority thread and seems (based on my 'test' patch) to have an easier time getting over the barrier than JS - possibly because it's based on ANSI C. And of course the syntax is beautiful.

I've looked into LuaJIT and it appears to be incredibly fast in comparison, but I'm not equipped to port it, and the actual execution speed inside the object is not what I am looking into.

Artur Dobija's icon

Hello! Suns_And_Zeroes, thank you for opening this topic. Lua seems pretty neglected as a scripting language indeed. I'd like to ask you about your investigations after few years. Did you reached any conclusion? Or did Lua turned to be right tool for what you were working with?

Im asking cause I look for a Max scripting environment with a decent "barrier" crossing between Max and script (mainly – plenty of midi messages and possibly MPEs too).

Suns_and_Zeroes's icon

Hey there!

I recently have tried to make a patch with the lua.jit.gl object, and it seems like it is getting a bit less reliable as the new Max versions come out. I was still on Max 6 when I started this thread, and about a month ago I used in in a Max 8 patcher. I found odd anomalies with numbers going in and being stored as other integers, and other odd behavior.

As cool of a language as Lua is, I think the JavaScript object is probably the way to go, it's always been stable for me. I think it's still a quite old version of JavaScript that runs in there, for example I believe there's still no for each loop syntax available - perhaps this is different in the Node.js implentation, not sure. But I think it's running a SpiderMonkey VM from 2013 or so.

You would have to have a ton of MIDI messages coming in to make it an issue with the [js] object.

If you want to use the [lua.jit.gl] object it will most likely work for you, but if you run into what I did recently you will have to find workarounds for peculiarities of the object. I believe that object hasn't been under development for quite some time now.

Sorry if this wasn't the answer you wanted, but I've used the [js] object to react to played MIDI input and it responds in time.

Best,
Ross

Iain Duncan's icon

Hi Artur, at the risk of tooting my own horn, I would recommend you check out my Scheme for Max project, which embeds the s7 Scheme interpreter used in Common Music and Snd into Max. It includes lots of Max interface functions for dict, table, and buffer i/o, and advanced scheduler interactions. It also gives you all the hot code reloading and advanced macro options of Lisp in Max, which no other options do. Perhaps most importantly for what you're describing, Scheme for Max is the only embedded dynamic language option that allows you to run in the high priority scheduler thread and override the garbage collector, enabling it to be used for low latency timing critical sequencing. the JS and Lua options run in the low thread, meaning your timing will get off anytime that thread is given long running tasks or their GC has to do a big run. They are good for UI and graphic manipulation, but not solid timing.

You can read about S4M on all the docs linked from the git hub site or watch videos on the youtube channel to get an idea of what it's for. I built it for exactly the use cases you are describing and am able to run realtime sequencing with tons of midi messages both in stand alone Max and in Ableton Live, with timing that locks up perfectly with the master clocks of each. I've written extensive beginner friendly docs and tutorials too.

Downloads, news, etc. https://github.com/iainctduncan/scheme-for-max
Demos and tutorials: http://youtube.com/c/musicwithlisp
Beginner's book: https://iainctduncan.github.io/learn-scheme-for-max/introduction.html

Feel free to ask me any questions.
iain

Artur Dobija's icon

@SUNS_AND_ZEROES

The answer is just perfect, don't worry :) We could hope for a fix! TBH I would still be keen of experimenting with it. The "legacy" [js] object also has its peculiarities.

The [js] object should run on js ES5 (but docs still state the js 1.6 version), the NodeJS is AFAIK up to date. I use it a lot, for UI-ing mostly.

@Iain Duncan
Thank you so much for you recommendation! I already gave it a try, but I still need to learn Lisp more, haha. The docs are so impressive.

Iain Duncan's icon

Thanks Artur. Honestly, once you dive in, Scheme is not hard. It's so minimal and consistent compared to other languages! And the unusual look of the syntax becomes second nature very quickly. I have done a lot of Python and JS professionally over the last 15 years, and now I much prefer it if given my choice of languages. Matt Wright (the author of OSC and of the Kadenze Max course) wrote a great intro book called "Simply Scheme" that is available free online, and the books "Common Lisp: A Gentle Introduction To Symbolic Computing" and "How to Design Programs" are both excellent and free online too. Hope that helps!

Brian Kirkbride's icon

I would recommend you check out my Scheme for Max project, which embeds the s7 Scheme interpreter used in Common Music and Snd into Max.

This is a timely reminder to check out this awesome project. Thanks for creating it! As a Clojure user I am very interested in a Lisp executing inside Max

.Scheme for Max is the only embedded dynamic language option that allows you to run in the high priority scheduler thread

It's great to hear that Scheme for Max runs in high-priority. That's crucial for a lot of stuff I'd like to use a non-visual programming language for in a music/sound context. My experimentation (referenced earlier in this thread) had indicated that the Lua VM ran at high-priority too. I couldn't find any definitive documentation either way. Iain, did you find a place this was documented?

On my end, I'm thrilled that Scheme for Max exists because I'm much more comfortable in Lisp than Lua. Looking forward to digging into it.

Iain Duncan's icon

Thanks Brian. I could be mistaken on that, so if someone knows for sure, it would be great to know. I know that it targets the jitter and UI side of things, which generally run in the UI thread, so that was what I thought. In S4M, I give the user the option to lock the interpreter to either high or low thread, with all incoming messages deferred accordingly. This is what the JS object (and the Live API for that matter) do as well, but only to the low thread. Additionally, I have added an easy way to disable automatic garbage collection so that you can run the GC more frequently at times of your choosing, and this makes a huge difference for latency. You can even just allocate a large heap and turn it off for the duration of a piece, which works fine. It helps that the s7 author, (Bill Schottstaedt, the author of Common Lisp Music and Snd), has done a lot of work to make the GC and s7 run quickly!

Iain Duncan's icon

Also, you might be interested to know that s7 shares a surprising number of features with Clojure, though they evolved separately. Like Clojure, s7 uses keywords and Common Lisp style macros with first class environments and gensym. Clojure was also my first intro to Lisps.

Brian Kirkbride's icon

Additionally, I have added an easy way to disable automatic garbage collection so that you can run the GC more frequently at times of your choosing, and this makes a huge difference for latency. You can even just allocate a large heap and turn it off for the duration of a piece

That's fantastic as GC is the bane of timeliness in all domains. Love the idea of saying "we'll clean this mess up after the show!"

Also, you might be interested to know that s7 shares a surprising number of features with Clojure, though they evolved separately. Like Clojure, s7 uses keywords and Common Lisp style macros with first class environments and gensym. Clojure was also my first intro to Lisps.

Wow wow wow. Now I'm even more excited to dig in...just have to find the free time. Thank you Iain!

Iain Duncan's icon

You're welcome, it's a lot of fun. Locking out the GC for the duration of a piece works really well actually. I can run a very heavy sequencing environment in Live with the audio buffer at 128 if I do. There are instructions in the docs about making sure your heap size is large enough that you won't need a heap resize (also expensive) during the piece. Basically the two best low latency options are a) go for the smallest heap you can (say 8 or 16K) and run the GC lots (like every 100ms off a metro) so it gets absorbed harmlessly in your audio latency, or b) set a big enough heap, lock it out, and clean up later. I have examples in the help and docs. Though to be honest, you won't even notice the gc until your scheme code gets pretty big or you're trying to run with very low latency.