Node For Max Bundling in Collectives, Standalones and Max For Live Devices
Florian Demmer
Jan 16 2019 | 6:26 pm
With the just released 8.0.3 update of Max 8, Node For Max now supports bundling dependencies (aka the node_modules folder) within Collectives, Standalone Applications and Max For Live Devices. In order to give you a jump start into using this feature, this is a short guide on how to do it.
Let’s assume for the following that we are working on a Project called N4M-Bundler.
/1 Creating The Project
We start off by opening Max and creating a new Project using the File Menu called N4M-Bundler. In return Max will show us the Project Window, which can be used to organize the project’s files and contents. Since we don’t want Max to try to organize contents automatically, the first thing we’ll do is disable the “Keep Project Folder Organized” option in the Project Inspector.
To open the Project Inspector use the Project Settings Menu:
In the Project Inspector disable the option "Keep Project Folder Organized":
You can now close the Project Inspector. We will now use the + Button in the bottom left corner in order to add our main patcher to the project by choosing "Add New File…" and creating a Max Patcher called n4m-bundler.main.maxpat
In order to verify what just happened we can use the Finder (Mac) / File Explorer (Windows) and look at the N4M-Bundler folder in the user’s Documents/Max 8/Projects folder. It should now contain a folder called patchers that includes our freshly created n4m-bundler.main.maxpat patcher.
In order to have a place to organize our content for node.script we will also create a folder called node_content next to the patchers folder. Our folder should look something like this:
N4M-Bundler/ ├── patchers/ │ └── n4m-bundler.main.maxpat └── node_content/
2/ Adding node_content to the Project’s Search Path
In order make the files in our node_content folder available to the project we need to add it to the project’s Search Path. For that click on the Gear Icon in the Project Window and use choose the "Search Paths" option. Max will open another window that allows you to manage the Project's Search Paths.
Using the + Icon at the bottom left of this window we can add another entry, which should give you something like the following:
Now click the Choose Button in that new line and add the node_content folder we created previously. After that you can rename the entry to Node Content but most importantly you have to enable the “Embedded” option. Your Project's Search Path Window should now look similar to this:
3/ Creating the Node.js script
We will now create the simple script for this test scenario. For that open the node_content folder in a text editor of your choice and create a file called n4m-bundler.js with the following content:
const Chance = require("chance");
const maxAPI = require("max-api")
const chance = new Chance();
const generateRandomName = async () => {
const name = chance.name();
await maxAPI.outlet(name);
};
setInterval(generateRandomName, 1000);This uses a package called chance to generate a random name every second and outlet it to Max.
4/ Patching everything together and installing the chance module
We will now edit our Project’s main patcher and install the required chance package from NPM. The simple idea is to show the random name in a comment box. Start off by create a node.script instance pointing to our n4m-bundler.js file. Before we do anything else we will install the chance package by sending a message "script npm install chance" to the node.script object:
Once done you should find a node_modules folder and a package-lock.json in the node_content folder and can remove the message from the patcher. If you like, you can use Finder or File Explorer again to verify that your project directory looks something like this:
N4M-Bundler/ ├── patchers/ │ └── n4m-bundler.main.maxpat └── node_content/ │ └── node_modules/ │ │ └── chance/ │ │ └── *** a whole bunch of files *** │ └── n4m-bundler.js │ └── package-lock.json
We will now use the prepend and comment objects to show the generated, random name. And add the script start and script stop messages to control our script:
Clicking the script start message should start the script and show a random name in the comment box every second. You can now stop the node.script using script stop.
5/ Bundling
We have now all the pieces of the puzzle at hand and can build our project. Choose “Build Application” from the File Menu, make sure the Format is set to Application and proceed. Max should now build a Standalone application containing our Patch, Script but also the chance module as a dependency.
The same would be true if we attempt to build either a Collective or a Max For Live Device.
Feel free to let us know if you have any further questions or feedback on this.
- Valery_KondakoffJan 20 2019 | 3:31 pmHello, Florian!Thank you for your tutorial, but it seems there is something wrong here.1. You wrote: "We will now create the simple script for this test scenario. For that open the node_content folder in a text editor of your choice and create a file called n4m-bundler.js". But in the file listing the n4m-bundler.js file is located in the root project folder N4M-Bundler: N4M-Bundler/ ├── patchers/ │ └── n4m-bundler.main.maxpat └── node_content/ ├── node_modules/ │ └── chance/ │ └── *** a whole bunch of files *** ├── package-lock.json └── n4m-bundler.js2. You wrote: "We will now edit our Project’s main patcher and install the required chance package from NPM. <...> Once done you should find a node_modules folder and a package-lock.json in the node_content folder and can remove the message from the patcher." But the node_modules folder is created in the project root folder as well and not in the node_content folder:├── patchers/ │ └── n4m-bundler.main.maxpat └── node_content/ ├── node_modules/ │ └── chance/ │ └── *** a whole bunch of files *** ├── package-lock.json └── n4m-bundler.jsSo, there is a question: why do we need this 'node_content' folder? Looks like the Node For Max bundling works well without creating the additional folder: add main patcher and js-file to the project and then point to the 'node_modules' folder to the project search path.Can you, please, bring a little light on this issue?Thank you once again!
- Florian DemmerJan 22 2019 | 1:53 pmHello Valery,thanks for pointing this. And yeah sorry, it was in fact a mistake in these ASCII folder structures.If things work for you without the node_content folder: great! However, we do recommend the usage of a separate folder (node_content in this example) for the node related content in projects. The reasons for that are mostly practical. It's easier to find/navigate the Node related content and we provide a sort of space for npm to do it's disk related work (writing/altering node_modules, creating debug logs etc) in order to avoid having it change the project's root folder. Also in more complex cases you'll be including more user authored JS files for Node For Max that are required/used for your script. This way adding a folder to the project ensures that they will be bundled as well.Hope that makes sense? Thanks for following up and pointing out the little issue in the folder structures,Florian
- Valery_KondakoffJan 22 2019 | 2:45 pmHello, Florian!Yes, now everything works as expected. Thank you for this useful tutorial!
- sandcobainer~Feb 14 2019 | 10:14 pmHey Florian, A very detailed guide, thanks. When i run the script npm install <package-name> in my case, axios, the package is installed and the patch does what it's supposed to do, a http request. However, my finder does not show any package.json or node_modules created. So the npm is installing the packages elsewhere. This would be a problem for bundling and distribution. Any ideas?
- Florian DemmerFeb 15 2019 | 1:17 pmHey Sandeep,glad it's working for you. In terms of the installation location - it's kinda hard to pin that down given the limited information but I can try. To make sure you don't have any naming conflicts you can send a reveal message to node.script that should show the location of the file in your Finder or File Explorer. Is there a node_modules folder next to the file that gets revealed? If you are facing a naming conflict, we highly recommend namespacing your files, fe n4mtest.index.js where n4mtest would be the project specific namespace.Fwiw, npm install will extend the dependencies of a pre-existing package.json file but not create one (only the package-lock.json gets created). The minimum requirements are a name and version field (you can learn more about the package.json here):
{ "name": "mypackage", "version": "0.1.0" }Hope that helps a bit with understanding the functionality of npm install? Curious to see if there is in fact an issue with multiple files having the same name in your search path or if something else is at play. Let me know and I'm happy to assist further.Thanks, Florian - Stepan DyachkovskiyMar 26 2019 | 11:12 pmHi Florian, Thanks for this, super helpful for what I'm trying to do. I'm running into an error where the node_content folder can't be added to the application, and I'm not sure why. The Max Console reads as follows... Adding file message.mxe64 Adding file prepend.mxe64 Adding file comment.mxe64 Adding folder node_content Unable to add node_content! Investigating Audio Status.maxpat... Saving Patcher Audio Status.maxpat... Copying External live.numbox Copying External adstatus Copying External umenu Copying External metro Copying External number Copying External substitute Copying External toggle Copying External pak Copying External speedlim Copying External delay Copying External button Copying External js Copying External bgcolor Copying External universal Copying External ubutton Copying External zl Including File interfacecolor.js Copying External led Copying External textbutton Copying External panel Copying External spray Copying External Uzi Copying External loadmess Copying External !- Copying External buddy Copying External dspstate~ Successfully built project N4M-Bundler to application.(emphasis mine) There are no other errors, and the application opens just fine, but obviously can't find the n4m-bundler.js file. I'm running Windows 10, 64 bit.I'm at a loss as to why this could be happening and I appreciate your help. Thanks! Stepan
- Florian DemmerMar 27 2019 | 8:57 pmThanks for the reply Stepan. Please get in touch with our support as we might have to get a copy of your project in order to be able to reproduce the issue.Thanks in advance. Florian
- Kay JangOct 24 2019 | 4:35 pmHi, Florian.The issue " STEPAN DYACHKOVSKIY " mentioned is solved? I have the same problem.Adding folder node_content Unable to add node_content!I'm also running Windows 10, 64bit. Thanks in advance.
- Florian DemmerOct 30 2019 | 6:55 pmI cannot reproduce the issue here on my end. If you still encounter issues with bundling on Windows please get in touch with our support so we can help you in a more detailed fashion.Thanks Florian
- jgastonraoulMay 06 2020 | 1:06 pmHi !Just a quick reply for the others having problems to have a working build on Windows (Max 8.1.3/64 bits). If you try to build the standalone from the project window (menu : "Manage Project > Build Collective/Application..." ) it doesn't work (you should have the message 'Unable to add node_content!' in the Max Window). If You build using the menu "File > Build Collective/Application..." in the main patcher window, it works flawlessly.Regards, Jérémie
- OCHSep 21 2020 | 7:27 pmHi Florian, Never got a chance to thank you for writing this. Just a quick note that second picture caption has a typo: Disbale And I would like to request this information to be added to Max built-in documentation. (node.script help patch)Thanks, O
- Florian DemmerOct 06 2020 | 9:59 am@Omar thanks for the pointer, just fixed the typo. AFAIK the info is bundled with the Max application in the Working with Projects, Max For Live Devices, and Standalones. You can find the same content here:
- Eric BatemanSep 14 2022 | 1:29 amIt would seem that this process is broken on MacOS Monterey. Using the node.script "open folder location" icon, it would seem that all of the javascript files are being sandboxed in a directory in the user library, and none of the node modules are being copied over.
- Florian DemmerSep 27 2022 | 8:51 amHi Eric, thanks for the reply. Was this working successfully before on a different version of macOS?Would you mind getting in touch with our support with a bit more insight on your project, workflow and version of Max that shows that issue in order for us to be able to investigate further if that's a general issue or something specific? Thanks Florian
- Eric BatemanSep 27 2022 | 6:10 pmHello Florian,Yes this is an internal application that we’ve been using since 2018 with previous versions of macOS.I will reach out to the support team as you requested.
- Fred VoisinMar 20 2023 | 12:38 amHello,I still have the same issue Eric mentioned, Max 8.3 OSX Monterey. The nmp modules are not copied into the dir Content/Resources/C74/packages/Node\ for\ Max/source/node_modules/ , and the package.json (v. 2.0.4) does not declare them while they are into the Project's package.json. So that the app tells that the script can't start (not ready). Is it already solved in some way ?
- Source AudioMar 20 2023 | 9:03 amwhy don't you copy files manully ? At same time you could trash unneeded files from the package....
- Fred VoisinMar 21 2023 | 10:16 amYes, sorry to have not mentioned I already did it, the result being the same , node.script tells : 'node script not ready, can't handle message...' . Same result with message 'script stop' after having sent 'script start'. This appears as soon as I declare in the script require('sqlite3'), even if : - I do not call at load time any sqlite function (but only require) ; - declared [standalone @database 1] in the main patch (or not, it should not be necessary since sqlite may run apart in some node process) ; - and copied manually all of the required modules and their dependencies. Here is the node.script : const path = require('path'); const Max = require('max-api'); const sqlite3 = require('sqlite3');Max.post(`Loaded the ${path.basename(__filename)} script`);Max.addHandler("testing", () => { Max.post("test"); })When starting it, node.script debug tool tells "Process Running" (in green), but the first Max.post does nothing, and the 'testing' message from Max returns 'Node script not ready can't handle message testing'. Building from Max/File/ menu or from Project/manage does not make a difference. A point would be that the file 'package-lock.json' which is at same level of the app's Content node_modules directory is not updated even if I consolidated the project, so that it does not mention any of sqlite3 and its dependencies. If I copy the 'package-lock.json' from the project folder, the last declaring sqlite3 package (version 5.1.6), it makes no difference.I'm running Monterey on x86 macbookpro (Max 8.5.3).
- Source AudioMar 21 2023 | 10:31 amthen your problem is sqlite related and not failed build process ?
- Fred VoisinMar 21 2023 | 11:30 amMay be, not so clear. May be the build is not working properly : if the project is well defined, the generated app is not complete and requires manual intervention after build but this manual intervention does not yet fix it. And it may have something to do with sqlite3, or the OS ? so that the Cycling annoucement relating that it's better to deal with database using node is not true for me, until now the sqlite embedded in Max built works much better.
- Fred VoisinMar 22 2023 | 1:55 pmSo, we did a try of node.script using Max8.5.3 on Windows10 which is very strange, see my new topic: https://cycling74.com/forums/node-script-on-windows10 The same with Max OSX Monterey is working, but not yet with a Max built app when requiring sqlite3. Any idea ?
- Florian DemmerMar 23 2023 | 1:56 pmHi Fred, would u mind providing a minimal example project that can be used to reproduce the issue? I'm happy to look into it and see what might be the culprit but would prefer to limit the scope and take it one issue / confusion at a time as there are a lot variables at play in your reports in this thread.One general question - are you successfully running npm install on the node.script object prior to building / bundling? I'd also be curious if you are only encountering this with the sqlite3 package or also other npm dependencies given that sqlite3 contains a native dependency and something might go wrong with node-gyp in case they can't be fetched as pre-built binaries.
- tillflyApr 20 2023 | 10:45 amHey, when i instantiate the node.script object with n4m-bundler.js . the max console prints "node.script: n4m-bundler.js: No such file or directory" , although i have added the search path in the Project Search Paths. I need to add the search path to the Max Search Path ( options->file prefs-> ... ) furthermore when I send the msg "script npm install chance" the node module is not in the folder, nor a .json. I tried to find it via "reveal" but it only points to "n4m-bundler.js".Help appreciated :)
