Node For Max Bundling in Collectives, Standalones and Max For Live Devices

Florian Demmer's icon

With the just released 8.0.3 update of Max 8, Node For Max now supports bundling dependencies (aka the node_modules folder) within Collectives, Standalone Applications and Max For Live Devices. In order to give you a jump start into using this feature, this is a short guide on how to do it.

Let’s assume for the following that we are working on a Project called N4M-Bundler.

/1 Creating The Project

We start off by opening Max and creating a new Project using the File Menu called N4M-Bundler. In return Max will show us the Project Window, which can be used to organize the project’s files and contents. Since we don’t want Max to try to organize contents automatically, the first thing we’ll do is disable the “Keep Project Folder Organized” option in the Project Inspector.

To open the Project Inspector use the Project Settings Menu:

Access the Project Inspector using the Project Settings Menu

In the Project Inspector disable the option "Keep Project Folder Organized":

Disable the "Keep Project Folder Organized" Option

You can now close the Project Inspector. We will now use the + Button in the bottom left corner in order to add our main patcher to the project by choosing "Add New File…" and creating a Max Patcher called n4m-bundler.main.maxpat

In order to verify what just happened we can use the Finder (Mac) / File Explorer (Windows) and look at the N4M-Bundler folder in the user’s Documents/Max 8/Projects folder. It should now contain a folder called patchers that includes our freshly created n4m-bundler.main.maxpat patcher.

In order to have a place to organize our content for node.script we will also create a folder called node_content next to the patchers folder. Our folder should look something like this:

N4M-Bundler/
├── patchers/
│ └── n4m-bundler.main.maxpat
└── node_content/

2/ Adding node_content to the Project’s Search Path

In order make the files in our node_content folder available to the project we need to add it to the project’s Search Path. For that click on the Gear Icon in the Project Window and use choose the "Search Paths" option. Max will open another window that allows you to manage the Project's Search Paths.

Using the + Icon at the bottom left of this window we can add another entry, which should give you something like the following:

Search Path Window after adding a new entry

Now click the Choose Button in that new line and add the node_content folder we created previously. After that you can rename the entry to Node Content but most importantly you have to enable the “Embedded” option. Your Project's Search Path Window should now look similar to this:

Project Search Path Window after adding the node_content folder

3/ Creating the Node.js script

We will now create the simple script for this test scenario. For that open the node_content folder in a text editor of your choice and create a file called n4m-bundler.js with the following content:

const Chance = require("chance");
const maxAPI = require("max-api")
const chance = new Chance();

const generateRandomName = async () => {
        const name = chance.name();
        await maxAPI.outlet(name);
};

setInterval(generateRandomName, 1000);

This uses a package called chance to generate a random name every second and outlet it to Max.

4/ Patching everything together and installing the chance module

We will now edit our Project’s main patcher and install the required chance package from NPM. The simple idea is to show the random name in a comment box. Start off by create a node.script instance pointing to our n4m-bundler.js file. Before we do anything else we will install the chance package by sending a message "script npm install chance" to the node.script object:

Installing the chance NPM package using a message to node.script

Once done you should find a node_modules folder and a package-lock.json in the node_content folder and can remove the message from the patcher. If you like, you can use Finder or File Explorer again to verify that your project directory looks something like this:

N4M-Bundler/
├── patchers/
│ └── n4m-bundler.main.maxpat
└── node_content/
│ └── node_modules/
│ │ └── chance/
│ │ └── *** a whole bunch of files ***
│ └── n4m-bundler.js
│ └── package-lock.json

We will now use the prepend and comment objects to show the generated, random name. And add the script start and script stop messages to control our script:

The final patch, which displays the generated names in a comment box

Clicking the script start message should start the script and show a random name in the comment box every second. You can now stop the node.script using script stop.

5/ Bundling

We have now all the pieces of the puzzle at hand and can build our project. Choose “Build Application” from the File Menumake sure the Format is set to Application and proceed. Max should now build a Standalone application containing our Patch, Script but also the chance module as a dependency.

The same would be true if we attempt to build either a Collective or a Max For Live Device.

Feel free to let us know if you have any further questions or feedback on this.

Valery_Kondakoff's icon

Hello, Florian!

Thank you for your tutorial, but it seems there is something wrong here.

1. You wrote: "We will now create the simple script for this test scenario. For that open the node_content folder in a text editor of your choice and create a file called n4m-bundler.js". But in the file listing the n4m-bundler.js file is located in the root project folder N4M-Bundler:

N4M-Bundler/
├── patchers/
│ └── n4m-bundler.main.maxpat
└── node_content/
├── node_modules/
│ └── chance/
│ └── *** a whole bunch of files ***
├── package-lock.json
└── n4m-bundler.js

2. You wrote: "We will now edit our Project’s main patcher and install the required chance package from NPM. <...> Once done you should find a node_modules folder and a package-lock.json in the node_content folder and can remove the message from the patcher." But the node_modules folder is created in the project root folder as well and not in the node_content folder:

├── patchers/
│ └── n4m-bundler.main.maxpat
└── node_content/
├── node_modules/
│ └── chance/
│ └── *** a whole bunch of files ***
├── package-lock.json
└── n4m-bundler.js

So, there is a question: why do we need this 'node_content' folder? Looks like the Node For Max bundling works well without creating the additional folder: add main patcher and js-file to the project and then point to the 'node_modules' folder to the project search path.

Can you, please, bring a little light on this issue?

Thank you once again!

Florian Demmer's icon

Hello Valery,

thanks for pointing this. And yeah sorry, it was in fact a mistake in these ASCII folder structures.

If things work for you without the node_content folder: great! However, we do recommend the usage of a separate folder (node_content in this example) for the node related content in projects. The reasons for that are mostly practical. It's easier to find/navigate the Node related content and we provide a sort of space for npm to do it's disk related work (writing/altering node_modules, creating debug logs etc) in order to avoid having it change the project's root folder. Also in more complex cases you'll be including more user authored JS files for Node For Max that are required/used for your script. This way adding a folder to the project ensures that they will be bundled as well.

Hope that makes sense? Thanks for following up and pointing out the little issue in the folder structures,

Florian

Valery_Kondakoff's icon

Hello, Florian!

Yes, now everything works as expected. Thank you for this useful tutorial!

sandcobainer~'s icon

Hey Florian,

A very detailed guide, thanks. When i run the script npm install <package-name> in my case, axios, the package is installed and the patch does what it's supposed to do, a http request.

However, my finder does not show any package.json or node_modules created. So the npm is installing the packages elsewhere. This would be a problem for bundling and distribution. Any ideas?

Florian Demmer's icon

Hey Sandeep,

glad it's working for you. In terms of the installation location - it's kinda hard to pin that down given the limited information but I can try. To make sure you don't have any naming conflicts you can send a reveal message to node.script that should show the location of the file in your Finder or File Explorer. Is there a node_modules folder next to the file that gets revealed? If you are facing a naming conflict, we highly recommend namespacing your files, fe n4mtest.index.js where n4mtest would be the project specific namespace.

Fwiw, npm install will extend the dependencies of a pre-existing package.json file but not create one (only the package-lock.json gets created). The minimum requirements are a name and version field (you can learn more about the package.json here):

{
  "name": "mypackage",
  "version": "0.1.0"
}

Hope that helps a bit with understanding the functionality of npm install? Curious to see if there is in fact an issue with multiple files having the same name in your search path or if something else is at play. Let me know and I'm happy to assist further.

Thanks,
Florian

Stepan Dyachkovskiy's icon

Hi Florian,
Thanks for this, super helpful for what I'm trying to do. I'm running into an error where the node_content folder can't be added to the application, and I'm not sure why. The Max Console reads as follows...

Adding file message.mxe64
Adding file prepend.mxe64
Adding file comment.mxe64
Adding folder node_content
Unable to add node_content!

Investigating Audio Status.maxpat...
Saving Patcher Audio Status.maxpat...
Copying External live.numbox
Copying External adstatus
Copying External umenu
Copying External metro
Copying External number
Copying External substitute
Copying External toggle
Copying External pak
Copying External speedlim
Copying External delay
Copying External button
Copying External js
Copying External bgcolor
Copying External universal
Copying External ubutton
Copying External zl
Including File interfacecolor.js
Copying External led
Copying External textbutton
Copying External panel
Copying External spray
Copying External Uzi
Copying External loadmess
Copying External !-
Copying External buddy
Copying External dspstate~
Successfully built project N4M-Bundler to application.

(emphasis mine)

There are no other errors, and the application opens just fine, but obviously can't find the n4m-bundler.js file. I'm running Windows 10, 64 bit.

I'm at a loss as to why this could be happening and I appreciate your help.

Thanks!
Stepan

Florian Demmer's icon

Thanks for the reply Stepan. Please get in touch with our support as we might have to get a copy of your project in order to be able to reproduce the issue.

Thanks in advance.
Florian

Kay Jang's icon

Hi, Florian.

The issue " STEPAN DYACHKOVSKIY " mentioned is solved?
I have the same problem.

Adding folder node_content
Unable to add node_content!

I'm also running Windows 10, 64bit.
Thanks in advance.

Florian Demmer's icon

I cannot reproduce the issue here on my end. If you still encounter issues with bundling on Windows please get in touch with our support so we can help you in a more detailed fashion.

Thanks
Florian

jgastonraoul's icon

Hi !

Just a quick reply for the others having problems to have a working build on Windows (Max 8.1.3/64 bits).
If you try to build the standalone from the project window (menu : "Manage Project > Build Collective/Application..." ) it doesn't work (you should have the message 'Unable to add node_content!' in the Max Window).
If You build using the menu "File > Build Collective/Application..." in the main patcher window, it works flawlessly.

Regards,
Jérémie

OCH's icon

Hi Florian,
Never got a chance to thank you for writing this.
Just a quick note that second picture caption has a typo: Disbale
And I would like to request this information to be added to Max built-in documentation.
(node.script help patch)

Thanks,
O

Florian Demmer's icon

@Omar thanks for the pointer, just fixed the typo. AFAIK the info is bundled with the Max application in the Working with Projects, Max For Live Devices, and Standalones. You can find the same content here:

Eric Bateman's icon

It would seem that this process is broken on MacOS Monterey. Using the node.script "open folder location" icon, it would seem that all of the javascript files are being sandboxed in a directory in the user library, and none of the node modules are being copied over.

Florian Demmer's icon

Hi Eric,
thanks for the reply. Was this working successfully before on a different version of macOS?

Would you mind getting in touch with our support with a bit more insight on your project, workflow and version of Max that shows that issue in order for us to be able to investigate further if that's a general issue or something specific?

Thanks
Florian

Eric Bateman's icon

Hello Florian,

Yes this is an internal application that we’ve been using since 2018 with previous versions of macOS.

I will reach out to the support team as you requested.

Fred Voisin's icon

Hello,

I still have the same issue Eric mentioned, Max 8.3 OSX Monterey. The nmp modules are not copied into the dir Content/Resources/C74/packages/Node\ for\ Max/source/node_modules/ , and the package.json (v. 2.0.4) does not declare them while they are into the Project's package.json. So that the app tells that the script can't start (not ready).
Is it already solved in some way ?

Source Audio's icon

why don't you copy files manully ?
At same time you could trash unneeded files from the package....

Fred Voisin's icon

Yes, sorry to have not mentioned I already did it, the result being the same , node.script tells : 'node script not ready, can't handle message...' . Same result with message 'script stop' after having sent 'script start'.
This appears as soon as I declare in the script require('sqlite3'), even if :
- I do not call at load time any sqlite function (but only require) ;
- declared [standalone @database 1] in the main patch (or not, it should not be necessary since sqlite may run apart in some node process) ;
- and copied manually all of the required modules and their dependencies.

Here is the node.script :

const path = require('path');
const Max = require('max-api');
const sqlite3 = require('sqlite3');

Max.post(`Loaded the ${path.basename(__filename)} script`);

Max.addHandler("testing", () => {
Max.post("test");
})

When starting it, node.script debug tool tells "Process Running" (in green), but the first Max.post does nothing, and the 'testing' message from Max returns 'Node script not ready can't handle message testing'.

Building from Max/File/ menu or from Project/manage does not make a difference.

A point would be that the file 'package-lock.json' which is at same level of the app's Content node_modules directory is not updated even if I consolidated the project, so that it does not mention any of sqlite3 and its dependencies. If I copy the 'package-lock.json' from the project folder, the last declaring sqlite3 package (version 5.1.6), it makes no difference.

I'm running Monterey on x86 macbookpro (Max 8.5.3).

Source Audio's icon

then your problem is sqlite related and not failed build process ?

Fred Voisin's icon

May be, not so clear. May be the build is not working properly : if the project is well defined, the generated app is not complete and requires manual intervention after build but this manual intervention does not yet fix it. And it may have something to do with sqlite3, or the OS ? so that the Cycling annoucement relating that it's better to deal with database using node is not true for me, until now the sqlite embedded in Max built works much better.

Fred Voisin's icon

So, we did a try of node.script using Max8.5.3 on Windows10 which is very strange,
see my new topic: https://cycling74.com/forums/node-script-on-windows10
The same with Max OSX Monterey is working, but not yet with a Max built app when requiring sqlite3.

Any idea ?

Florian Demmer's icon

Hi Fred, would u mind providing a minimal example project that can be used to reproduce the issue? I'm happy to look into it and see what might be the culprit but would prefer to limit the scope and take it one issue / confusion at a time as there are a lot variables at play in your reports in this thread.


One general question - are you successfully running npm install on the node.script object prior to building / bundling? I'd also be curious if you are only encountering this with the sqlite3 package or also other npm dependencies given that sqlite3 contains a native dependency and something might go wrong with node-gyp in case they can't be fetched as pre-built binaries.

tillfly's icon

Hey,
when i instantiate the node.script object with n4m-bundler.js .
the max console prints "node.script: n4m-bundler.js: No such file or directory" , although i have added the search path in the Project Search Paths.
I need to add the search path to the Max Search Path ( options->file prefs-> ... )

furthermore when I send the msg "script npm install chance" the node module is not in the folder, nor a .json. I tried to find it via "reveal" but it only points to "n4m-bundler.js".

Help appreciated :)

Dan Nigrin's icon

Hi Florian - old thread that I'm reviving, I'm trying to create a standalone that includes a Node for Max script, as well as an NPM package. I'm seen your tutorial, but I don't normally use the Project Manager and prefer to build my standalones manually - however I am failing. I started another thread (https://cycling74.com/forums/node-script-and-standalone), but will repeat the info here.

Everything works perfectly with my .maxpat file, the .js file loaded by the node.script object, and the NPM library (midi-file) all within a single folder on my Mac. Like this:

main.maxpat
my_javascript_for_node.js
package-lock.json
package.json
node_modules
midi_file
<a bunch of stuff>

No matter how I copy the above files into my standalone's Resources/C74 hierarchy, I can't get it to work - the app crashes every time, with this kind of error:

Thread 0 Crashed:: JUCE Message Thread Dispatch queue: com.apple.main-thread
0 node.script 0x10f5dc30c maxnode_protocol_connection_connect + 12
1 node.script 0x10f5de7ae node_child_process_interface_new + 526
2 node.script 0x10f5e2d82 nodescript_updatepath + 306
3 node.script 0x10f5e23ab nodescript_new + 459
<etc....>

Any suggestions for what I'm doing wrong? Thanks in advance!

Dan Nigrin's icon

Also, the formatting got mangled above - the midi-file folder, and the <bunch of stuff> are both located inside the node_modules folder.