JS require still isn't CommonJS compatible - doesn't understand paths!


    Jun 20 2015 | 8:42 pm
    I complained about this several months ago but I just bought Max 7... so... :-)
    There are four issues with the Javascript require command in Max 7, three of which can be gotten around, but one of which can't - and makes writing production code hard.
    1. require doesn't understand paths.
    Here is where paths are discussed in the CommonJS specification. Paths are not an optional part of the spec! :-) Here are tests that include tests for absolute and relative paths.
    It is impossible to get around this restriction in any production-quality fashion. The solution "put all code in the same subdirectory" is simply a non-starter for me - I have 147 separate JS files in over a dozen subdirectories. More, I was hoping to start to introduce some well-known JS packages like lodash into my project - but the correct way to do that is with a git subtree or git submodule - i.e. as a subdirectory.
    My current Max 6 solution (use make) will continue to work, but is lame for several reasons - I have to run a make after each time I change anything, and it leaves these big ".jso" files, the result of the make with all the .js glued together, in my subdirectories - it's also not suitable for using third-party libraries that expect to use exports.
    2. The js box can't find Javascript files in a subdirectory of its current directory.
    This is actually a regression from Max 6! Before you could have a box looking like js sub/foo.js and it'd work. Now it doesn't, whether or not you use a full absolute path or a relative path.
    3. You don't get an error if require fails to find the file.
    Instead, strangely enough, require returns some sort of function that I can't identify and seems to do nothing if called. It'd be easy to detect this in your code using the JS typeof operator.
    4. Your Javascript only gets recompiled if the top .js file changes, not if any files it requires are changed.
    You could easily workaround 2, 3 or 4 if 1 worked...
    Here is a repo demonstrating these issues.

    • Jul 13 2015 | 12:47 am
      Hey Tom,
      I'm just starting to use this functionality myself, but I think that in regards to (1), this isn't an issue if the files are in Max's search path. All my files are organized as a package and in ~/Documents/Max 7/Packages/, and I have no trouble pulling a require() from different subdirectories inside that package. There might be other problems (just getting started here), but both require() and include() seem to work from inside a package.
      (Perhaps this isn't even what you're talking about, though.....)
      a
    • Jul 13 2015 | 9:01 pm
      Interesting suggestion!
      I could do that, with some changes to the code, but I'm trying to write a library that others might use (there are a couple of people who actually do already).
      As such, forcing my poor users to have my 137 .js files into the classpath in order to run it is like leaving a lot of boobytraps lying around. Some day, some .js file name of mine will collide with some other .js file somewhere, and someone will be left trying to debug it - and worse, there might not be a good solution without editing my project (which means they have to apply that edit again every time I make an update).
      The possibility of collisions isn't at all hypothetical - because the project already has ten files named test.js (and there's a good reason for it - when I create unit tests I create a file named test.js that loads the tests for that directory).
      I was also very much hoping to start using third-party CommonJS libraries - because there are a lot of them, and some of them are extremely useful and high quality. And many of these also rely on relative paths working correctly.
      For documentation purposes, this is also a bad idea. When someone sees require('controller.js'); they really have no idea where that file is. require('swirly/softstep/controller.js'); is so much clearer to read.
      I understand that I likely have a different outlook to most of you, as I've been making a living as a programmer for about 35 years, and in those 35 years I've seen every possible mode of failure you can imagine - and I've also read a lot of incomprehensible code that no one could maintain after its creator left. As such, I consider correctness and clarity to be of supreme importance.
      For example, I might be the only person in the world writing Javascript unit tests for Max/MSP! On the other hand, I have a complex setup with text score files in JSON which dynamically load handlers for MIDI as the performance goes and it has so far never crashed or failed during live performance...
      I refuse to cut corners - because cutting corners is just never worth it in the long run, because debugging is stressful and difficult. And I won't accept an "implementation" of a standard where important parts simply don't work.
      The correct way to organize a large Javascript library is to use subdirectories and relative includes. CommonJS requires that you support paths in your require statements, so if you don't, what you have is not a CommonJS implementation.
    • Jul 14 2015 | 6:42 am
      I completely understand your point of view, and agree with it....but I'm also tickled pink to have ANY sort of include capability aside from js-extensions. Hopefully we'll see some improvements in implementation in the future, as all of your points are completely valid.
      a
    • Jul 14 2015 | 12:39 pm
      yea, would be nice to have it commonjs comlient... but as I wote in an earlier discussion: The Max search path paradigm collides at some points with it. abandoning search pathes just for require doesn't seem to be a good idea. it's too deeply rooted in Max. One way would be to add a comment attributes (java style). i.e.
      /** * @CommonJs */
      could indicate to only use pathes in require() and include();
      introducing a new system of flags doesn't seem to be slick though. on top of it it would be nothing that is common in JS.
      the existing system with js-properties like autowatch, inlets and similar woud break the code outside Max. and without any extra flag there are problems with detecting the difference between require(test) and require(subfolder/test). while the second one is clear, the first one will remain ambigous.
      How does the File Object handles this case? it supports relative/absolute paths as well as Max search path... Don't remember...
      Max Js implementation is much older than common js, and that's where we are hitting a wall.
    • Jul 14 2015 | 3:37 pm
      > Max Js implementation is much older than common js, and that’s where we are hitting a wall.
      I'm not sure this is true. I'm pretty sure I remember that Cycling74 didn't actually write their own Javascript from the ground up - that would be far too much work - they used an open source one, I believe Mozilla's.
    • Jul 14 2015 | 5:16 pm
      they do, but what i meant is : when the [js/jsui] api was created, code still was pretty procedural and modual approaches in js werent at the horizon.
    • Jul 14 2015 | 9:37 pm
      An option could be using a grunt task to combine all js files into one. So you get automatic recompilation, have your seperated js files for single development and unit testing and use the single big file for deployment. But that may only help with a specific organization of code and may limit elsewhere as you pointed out with libraries... But having a grunt server runnning that rebuilds on File Change may be more convenient than a manual triggered make.... I found organizing everything in packages helpful too.
    • Jul 14 2015 | 9:57 pm
      Tom actually has its own "combiner" that concatenates js files already . The thing is to avoid that process at all due to several maintaining circumstances , especially now when CommonJs approach was considered recently in the new Max .Would love to have it implemented properly as suggested here . But if Jan is right with his observation then , that is sad honestly .
    • Jul 14 2015 | 10:01 pm
      +1
    • Jul 20 2015 | 10:35 am
      sorry to butt in, I'm a total n00b but: couldn't this issue be mitigated if a userpath/ searchpath could be specified to have unique file names based on directory path? If you check that 'unique names' box Max would then not look for files to autocomplete in that directory (as that could collide), only if the file is specified with a (relative if possible) directory reference.
      it would be a check box the advanced user understands and the beginner wouldn't touch..
      would that work?
    • Dec 15 2018 | 4:36 pm
      So it's over three years later and I tried this in Max 8 - it still doesn't work.
      This is frustrating since Max now supports Node - except that any Node package of any size will likely use relative includes, which means if I want to use the Node For Max feature, I'll need two separate JS codebases... :-/
      Any hope that this might one day work?
      Or if anyone has any workflow that isn't "throw a couple of hundred .js files into one huge flat namespace" let me know!
      ---
      I love the "unique names" box idea proposed above - it does exactly what is needed. That would work fine... But really, you shouldn't be calling this CommonJS if it doesn't do this basic thing that CommonJS environments are specified to do.
    • Oct 15 2019 | 7:51 pm
      +1 to this, can't believe common-usage relative paths still don't work with Max's require
    • Oct 15 2019 | 10:09 pm
      @Tom Swirly While relative paths still don't work, I found if I organized my code in projects then I could have retrieve a specific "MaxJSUtils.js" from the Project folder, rather any file matching that name in Max's search path. I have different versions of that file, so it was important to me to be able to select a specific one.
    • Oct 16 2019 | 7:47 am
      Oh, that's an interesting discovery! I wrote this before Projects existed, I think.
      But it doesn't solve the big issue of wanting to use Node.js packages as is.
      It's ridiculous that they've been advertising "CommonJS compatibility" for over four years, and yet have never delivered it.
    • Jan 30 2020 | 6:54 am
      +1 - especially with regards to node for max. Right now I'm already forced to run my app through the js object to access the ~buffer via JS (which another pain point on its own). This would be fine if I can easily convert my node project and use CommonJS for relative imports, unfortunately it's not that straightforward and I still need to take any one of the extra steps mentioned to make my patch work.
    • Jan 30 2020 | 9:36 am
      This was one of the reasons that I stopped using Max. I got bored with stuff like this. Note that I started using Max with version 1.0.