Husserl tutorial series (8). JavaScript: the Oddest Programming Language

Javascriptbeginner
Ernest's icon

Ernest

This tutorial is a general introduction to JavaScript issues, more similar to the first in the series. All tutorials in this series:

  1. Designing a good LFO in gen~ Codebox: https://cycling74.com/forums/gen~-codebox-tutorial-oscillators-part-one

  2. Resampling: when Average is Better: https://cycling74.com/forums/gen~-codebox-tutorial-oscillators-part-2

  3. Wavetables and Wavesets: https://cycling74.com/forums/gen~-codebox-tutorial-oscillators-part-3

  4. Anti-Aliasing Oscillators: https://cycling74.com/forums/husserl-tutorial-series-part-4-anti-aliasing-oscillators

  5. Implementing Multiphony in Max: https://cycling74.com/forums/implementing-multiphony-in-max

  6. Envelope Followers, Limiters, and Compressors: https://cycling74.com/forums/husserl-tutorial-series-part-6-envelope-followers-limiting-and-compression

  7. Repeating ADSR Envelope in gen~: https://cycling74.com/forums/husserl-tutorials-part-7-repeating-adsr-envelope-in-gen~

  8. JavaScript: the Oddest Programming Language: https://cycling74.com/forums/husserl-tutorial-series-javascript-part-one

  9. JavaScript for the UI, and JSUI:<a href="https://cycling74.com/forums/husserl-tutorial-9-javascript-for-the-ui-and-jsui"> https://cycling74.com/forums/husserl-tutorial-9-javascript-for-the-ui-and-jsui

  10. Programming pattrstorage with JavaScript: https://cycling74.com/forums/husserl-tutorial-series-programming-pattrstorage-with-javascript

  11. Applying gen to MIDI and real-world cases. https://cycling74.com/forums/husserl-tutorial-series-11-applying-gen-to-midi-and-real-world-cases

  12. Custom Voice Allocation. https://cycling74.com/forums/husserl-tutorial-series-12-custom-voice-allocation

JavaScript: the Oddest Programming Language

Aside from some functional programming languages, JavaScript is probably the strangest programming language ever designed. This is because it was designed to make it easy for non-programmers to get something working. Hence in Max:

s = "All Cats " + "are Grey"
x = 1 + .01
function s(y){ x = y ) 
post(x, s, ", by the Cure\n");

Might be expected to print " js: 1.01 All Cats are Grey, by the Cure" in the console.

  • Note 1: the "\n" adds a linefeed, which is only necessary to separate console post lines within your Javascript. When something else writes to the console, the js object will add one linefeed itself.

  • Note 2: semicolons at the end of statements aren't necessary in JavaScript if statements are on separate lines, but I've kept the habit of including them, for the occasions when I want to pack statements on the same line, to reduce scrolling while editing, and sometimes for screenshots.

But the above example won't print anything about cats, because the statement s(y){...} redeclares 's' to be the name of a function. Specifically, the s() function sets the value of the global variable 'x,' possibly from a message to the js object (the message from Max also doesn't need to be a number, for example, a message "s val" to the js object will set x to the string "val").

Not so typeless as we're led to believe

The above number and cat example illustrates the first oddity of JavaScript, which is that variables are said to be 'typeless,' so you can catenate strings with "+" signs, or add numbers to variables, and JavaScript will always work fine without complaint. In JavaScript 1.6, you can easily redefine a variable to be the name of a function. While initially it's more intuitive for a non-programmer, it's only a matter of time before a function gets unintentionally named the same as a variable, or something even worse happens to the hapless novice:

x = 3;
s = "there are" + x + "cats \n";
post(s);

which doesn't work as expected, because it mixes types. It's not the language's fault. It doesn't know whether to add numbers or to catenate strings. Those familiar with Max will understand the problem of mixing types, whereas those with less knowledge frequently give up entirely after encountering some apparently inexplicable type error.

In JavaScript, it's easy to get the obviously desired result by explicitly casting the type of a numeric variable to a string before catenating it with other strings:

x = 3 
s = "there are" + x.toString() + "cats \n"
post (s)

Non-programmers who get this far remain perplexed why this is achieved by 'x.toString()' and not 'toString(x)'. But on the whole, people just skip thinking about it very much, and just copy the things they are told to do, trying to ignore the feeling of stupidity. So it helps to understand why the available properties and methods as what they actually are in JavaScript. You can easily know what you are doing. It's not actually that complicated.

The key to understanding Javascript: almost everything is an object

Most languages have 'fields' of bits that are filled with a number, or sequence of ASCII values to represent a string, or some other data representation, such as an image or audio buffer. That is to say, the compiler handles your program by allocating chunks of memory to contain your variable values. That's not how javaScript views your script at all. It sees your script in an entirely different way.

Internal to Javascript, almost everything is an object. The entire script is an object (referred to by the keyword 'this'). Internally to the parser, the names of variables and functions are the same object, with different properties defining what they do, and with properties of names mutating, depending what the script does. When a name is used as a variable, JavaScript internally gives it a property defining its type as an integer, float, string, etc. When a name is used as a function, the function call is a property of the name which may be externally invoked as a 'method.' The only technical distinction between a property and a method, internal to the JavaScript parser, is that methods have a pointer to an instruction sequence and accept arguments. Internal properties which don't accept arguments, and which are externally visible, are merely pointers to variable values, with additional internal properties declaring their read/write and other security permissions.

The active properties for any programmer-defined name in a script change dynamically, whenever statements do different things with them. So in the above example, 'x' first has a property defining it as an integer, and 'toString()' is a method that acts on the object named 'x" that transforms it from an object assigned an integer property to an object assigned a string property. When the language designers added arrays, they added a new object type called 'Array' which inherits the 'toString() property from variables, so you can similarly transform an array called 'myArray[]' to a string with myArray.toString(). To JavaScript, arrays contain a list of objects, and the items in the array can be a mix of anything--numbers, strings, other objects, other arrays, whatever--and internally, JavaScript doesn't care. The flexibility of its object-based design and the expansive possibilities of inheritance are simultaneously the greatest benefit and biggest curse of the language.

On occasion, inheritance creates properties and methods that don't appear to do much useful. But they are still there, so JavaScript has quite a lot of properties and methods in its corners you'll never want to use. This means, the important thing in learning scripting is finding the language subset most useful to you and ignoring the rest of it. Just as non-native speakers can still communicate quite a bit in English despite having tiny vocabularies, you'll find you can do an enormous amount after only learning a little JavaScript. And these days, there is copious help on the Web, so when you get stuck, you can simply type in a question in your browser and be amazed how many thousands of other people have already appreciated the answer you're looking for. So saying, there are some specific things I've had to think about a lot when programming for Max, so here they are.

Scoping out the namespace: explicit declarations

Unlike languages like C, you mostly are not required to 'declare' your names at the beginning of a routine, at least in Max's 1.6 version of JavaScript. In more recent versions of JavaScript, declarations may still be anywhere in your code, but declarations are always required the first time you use a variable, even for loop iterators. So things like this have started to pop up alot:

for (var i =0; i < myArray.length; i +=1) {....

In Max's version of JavaScript, you don't need to include the 'var' declaration--but more recent JavaScript versions require it. If you look inside Google's JQuery libraries totaling 900KB, you'll see they're so crammed with acronyms, it's no surprise Google has insisted on such new restrictions. But it remains debatable why such restrictions have to be enforced on everyone, which has really slowed the adoption of the new version. On the one hand, some people are really annoyed about the more recent restrictions (that Max doesn't have yet), because we are accustomed to being able to copy things around, but....

//initialize a buffer's first channel with a number sequence
// on first bang from external metro
// then check if its contents have changed:

var myBuf = new Buffer("myBuf");// binds js myBuf with Max "myBuf"

var semaphore = 1;   // can't be inside bang(),
                     // or it will be reset to 1
            `        // every time bang() is called

function bang(){
    if(semaphore){
        for(var i=0;i<myBuffer.length;i+=1){
            myBuffer.poke(1, i, i);
            semaphore = 0;
         }
    }else{         
//ok....I'll simply copy the loop statement...
         for(var i=0;i<myBuffer.length;i+=1){ // < BUT It DOESNT WORK!
            if (myBuffer.peek(1, i) !=i) 
                post ("buffer index ", i, "has changed\n");
         }
    }
}

  • Note: this example uses "i +=1" (like in gen~) instead of the more common "i++". JavaScript supports "++" too, but it will only permit loop step sizes other than 1 with a "+=" operator, for example, "i+=2" -- or even "i+=n" if you've already defined 'n'.

  • Note 2: on the topic of operators, JavaScript also includes bitwise comparison and bit-shift operators. Bit-compare is useful for masking bits, for example in MIDI data; and bit-shift provides multiplies and divides by factors of two that are much faster than standard multiplication and division.

just copying the loop statement causes a compiler error, even in Max's JavaScript 1.6 version, because the loop iteration variable 'i' has already been declared. This new restriction requiring such declarations in later js versions makes a good reason to declare everything at the top of the global namespace, (or the top of a function, for local variables)....or not to declare the variable at all, which will work fine in Max now, but not in future JavaScript versions. But really! Wasn't javaScript meant to be easier than that? On the other hand, such explicit declarations at least mean you can't redefine a variable name as a function name by accident.

  • Note: functions can themselves contain definitions of their own local functions, because, functions are just another type of object. IN that case, the local function is basically not accessible except by the function which contains it.

Many have rightly observed that enforcing variable declarations, and other similar restrictions, are obsolescing the original reason for making almost everything into objects. So there's also speculation that JavaScript might be soon joining prior attempts to widen software programming's appeal, such as BPL and CP/M, in the dreaded graveyard. Notwithstanding, there's trillions of users accessing billions of pages with Javascript snippets, JQuery libraries, etc, so it's not as if another Bill Gates will wipe JavaScript out by Christmas.

Scoping the namespace: global versus local definitions

In all scripts, you have to be most careful with any names you use at the top level of a script, outside any functions, although it's much simpler in Max than in Web browsers. By convention, all such 'globals' are declared before functions, but you can have them anywhere you want. Once the script is compiled, any variables or other objects in this 'global namespace' retain their values across all invocations of the script. By contrast, inside functions, the values of variables only persist for the duration of the function call. This means you can make values 'static' just by putting them at the top level, outside any function.

var myGlobal = 3;
function changeVariables(x){
    myGlobal += x ; // the new value is retained 
                    // in the global namespace
    y += x; // y is a local function variable and undefined, 
            // causing an error in some parsers
}

Object-oriented programmers object to the simplicity of the global namespace, particularly because in browsers it is 'truly global,' meaning that ALL scripts share ALL names in the global namespace. Originally this was believed to be a useful feature, but in the last decades, different script writers have found their scripts being ruined by other script writers using the same names for things. This is not a problem in Max, which restricts the 'global namespace' to be within each 'js' object. So you don't need to be concerned about another script somewhere else polluting your data.

Max Tip: initialize your global variables

You will save yourself a lot of time by initializing all your global variables to a useful state, even if they are immediately set to something else during normal operation. This is because every time you save your script, to check it compiles properly for example, any messages your patch had sent to set variables in the JavaScript object will be forgotten, and all the globals will be back to their initial state.

Property Traversal: the slowest task in JavaScript

Each name you add to a namespace increases the length of the list of the global or local members that the program has to search through, every time it encounters any name of anything. As JavaScript internally creates a lot of hidden properties and methods on anything you declare, the internal list that your program has to search through quickly gets very long.

Keeping the namespace small is the single most important thing you can do in your code design to keep a script fast. Now by 'keeping the namespace small' I'm not saying a few dozen objects are bad, that's fine. But when you start adding things with dots in them, such as the extremely useful call 'this.patcher.getnamed("myobject").message(x), then JavaScript has to traverse the object lists that the dotted declaration creates all over again, each time it does anything, because all the objects chained together by the dots become part of the namespace. That includes, in this very useful declaration, every single property of every single top-level object in your JavaScript, including all of JavaScript's built-in objects and Cycling74's extensions (which is what 'this' refers to), and ALSO, every single property of every single object in your patch (because of 'this.patcher'), until it finds the script name 'myobject' from the target object's inspector window, to which to send a message.

That really can slow your script down while audio or video is running, because the higher-priority thread keeps suspending the JavaScript for time-critical tasks. If your design has a lot of audio or video data, the JavaScript gets knocked out the primary and secondary caches too. This is not to say Cycling74 did anything wrong in its implementation. The most CPU-intensive part of JavaScript has always been property-list traversal. There is a much longer and slower activity in any JavaScript program called 'garbage collection,' which refers to a slow and long process of freeing memory from bits of lists scattered around that might not be needed by anything. This was always part of the language's design, because it was reasoned people don't really care how long it takes to clean up the trash made to display a Web page, or how hard the CPU has to work at any point in time, as long as the page displays as fast as possible, as soon as the content is downloaded.

The good news about 'this.patcher.getnamed("myobject").message(x) is, if the script name isn't found at all during execution, you'll get an error reported in the console that the object name is undefined. After writing a function containing such calls to external objects, you do have to bear in mind that you can't just compile it to see it works properly. You have to call the function too, before all possible errors can be reported in the console.

Sending messages to Max objects: a real-world example from Husserl3

Husserl has about 200 UI objects, so I make it easy for myself to access them all like this:

var scriptNames =[ "chan", 
"whl","e1a","e1d","e1s","e2a","e2d","limlvl","e2s","e3a","e3d",// 1~10
"e3s","f1c","f1p","f1q","f1s","f1t","f1cs","f2cs","f1ps","f1qs",//11~20
//... (and so on)
"xx4", "xx5",xx6" ]; //141~143
var scriptIds    = new Array(160);
var debug       = 1;
var sinit       = 0;
function iDinit(){
  var p = this.patcher;
  for(i = 0; i < scriptNames.length; i++){
    scriptIds[i] = p.getnamed(scriptNames[i]);
    if (debug) post("init", i, scriptNames[i], scriptIds[i], '\n');
  }
  sinit = 1;
}

Conventionally one puts the largest structures first, so I declare two arrays, one to hold all the script IDs, and one to hold the script names.

This example shows the two most common ways to declare arrays. Purists would prefer multidimensional arrays (myArray[][]), but for technical reasons it's a real pain declaring and filling multidimensional arrays in JavaScript, and it doesn't really make that much difference just to have a few single-dimensional arrays if your second dimension is small.

It may be easiest to avoid multidimensional arrays in javascript, but arrays themselves are good things, because the name of the array is passed as a pointer between functions, which makes it easy to pass around large amounts of data efficiently.

  • Note: I earlier mentioned the Array.toString() method. Well when you write a post() statement to send something to the console, you separate individual items with commas. You can also put constants and single-dimensional array names in a post statement:

    var text = ["this", "is", "some", "text", "in", "an", "array"];
    post(1, text);

  • Max's post() method simply regards the comma-separated list declared in its argument as an array, and converts it into a string for display. So this example prints "1 this is some text in an array" to the console. The same applies for outlet statements:

    outlets = 1;
    var text = ["this", "is", "some", "text", "in", "an", "array"];
    outlet (0, "this gets sent out the outlet and ", text);

  • The outlet() method's first argument, '0,' selects the zero-indexed first outlet, and the following two arguments are catenated to send the message "this gets sent out the outlet and this is some text in an array." Be careful to declare the number of outlets first, or the compiler will generate an error.

If you do have a lot of data collections, a simple way to avoid the hassles of declaring and filling multidimensional arrays is to make an array of arrays, which can be useful to pass collections of unrelated bits in and out of functions all at once, particularly for callback functions as I will discuss further in a later tutorial. Functionally, an array of arrays appears exactly the same as what people think of as a multidimensional arrays after it's made, but it's built in a different way:

array1 = ["cat as string", catAsVariable, referenceToCatObject];
array2 = [5, 4, 3, 2, 1];
arrayofArrays = [array1, array2];

One can then access the items inside the array of arrays via sequential indices, so in your code it looks exactly like a multidimensional array:

arrayofArrays[1][0]

returns '5'. And technically, an array of arrays is more powerful than a 'standard' multidimensional array anyway, because you can also change an entire dimension very easily:

newEmptyarrayForThreeCats = new Array(3);
arrayofArrays[1] = newEmptyarrayForThreeCats;

Not that I've needed to get so sophisticated myself very frequently. In my Husserl3 example, I just declare two separate single-dimensional arrays, because in normal usage the array of script names isn't needed at all. The iDinit() function fills the scriptIds[] array with the internal numbers that Max internally creates as object reference IDs for all the objects in the patch. Subsequently, one can send a message to the objects like this:

 scriptIds[1].message(0); //send message to reset the mod wheel to zero
 
 scriptIds[1].message("set", 0); // sets mod wheel to zero without 
                                 // causing it to issue a message

Well that's easy and quite efficient if you're just manipulating displayed values. Now fancy dancy programmers will tell you that it's better to use 'named indices,' (called 'associative arrays' in JavaScript) and 'structures.' Very fancy dancy. But they are much slower. The parser has to iterate through each character of the string name and search for it, whereas for a numerically indexed array, the index number provides a direct pointer to the array value...something that the x86 instruction set is specifically designed to find VERY quickly, for historical reasons. Way back in 12-Mhz 80286 days, Excel spreadsheets were the big money maker, and people wanted to look up numbers in rows and columns as quickly as possible. So numeric indices are several orders of magnitude faster.

Having said that, if you need REALLY low-latency response, JavaScript is not for you, because it runs on the low-priority thread. Max's JavaScript doesn't work very well for, say, a note sequencer, because it will slow down whenever something significant happens on the Max message queue, or in A/V processing. If the low-priority event queue can't keep up, it just starts dropping messages.

JavaScript is great in Max for drawing to display objects. It saves you a lot of wires and messages. (In Husserl3, the audio routines store values in a "buffer~" for display, and a "qmetro" sends bangs into JavaScript for it to peek the "buffer~" and draw the results...which I plan to describe in the next tutorial).

While the above loop to fill the scriptIds[] array could be at the top level, outside a function, I've found that unreliable, because sometimes the JavaScript tries to initialize the array before Max has finished initializing the rest of the patch, causing 'object undefined' errors in the console. So I put the initialization inside a function that sets a 'semaphore' variable, sinit, when it's done. When a function needs to send a message to a Max object, it first checks if the 'sinit' semaphore has been set, and if not, calls the initialization routine:

sinit = 0

function iDinit(){
  var p = this.patcher;
  for(i = 0; i < scriptNames.length; i++){
    scriptIds[i] = p.getnamed(scriptNames[i]);
    if (debug) post("init", i, scriptNames[i], scriptIds[i], '\n');
  }
  sinit = 1;
}
function bang(){
  if (sinit == 0) iDinit();
  //...

I've found it important to include a 'debug' post message inside the loop. If an object name isn't found, there's an error message, but you can't tell which name wasn't found, if you don't make a message inside the loop that indicates which loop iteration failed. I also enable debug posts with a debug variable, because I keep adding post statements back for debugging all the time. With a debug variable I can leave them in and disable them when I don't want the console filled with messages. Eventually I delete all the post statements then regret it later.

The reason to go through all the above is that the iDinit() function keeps a lot of names and property searches out of the global namespace. All the 'dots' are eliminated from operation after initialization. The references to the display objects are all stuffed into a big array. If you don't do that, you will find the display updates very slowly from the JavaScript thread.

----------------------------------------------------------------

Appendix A: Reserved names and prototypes

Then there's the second, less complex oddity of JavaScript: reserved object names and hidden prototypes. For example, there is a 'Window' object, which is often the first thing JavaScript programmers encounter. All the methods and properties on the Window object are reserved. That is because the 'Window' object has a predefined prototype internally. That design 'feature' really annoyed object-oriented programmers, who think they should be able to add whatever they like to anything. So when people complained about it, the language designers exposed the method to prototype custom objects.

Object prototyping is actually how they built the language in the first place: when prototyping, one is using the language's own language constructor to construct new objects for it. Generally speaking, there's little utility to going through the effort of prototyping objects unless you are making a javaScript-based product, but it is there if you want it.

Thus, when Cycling74 wanted to define its own properties and methods on the application window, it wasn't possible to bind them to the Window object, and so Cycling 74 made a prototype for a new object called the 'Wind' object for its own application window. So for us, the properties and methods on the 'Wind' object are reserved to do exactly what Cycling74 wants.

Then programmers complained they couldn't add their own properties to the top-level objects. So Cycling74 exposed attribute declarations on the Javascript object itself.

Appendix B: Max's js object 'attributes'

Attributes are one of the first things you might encounter in the Max help, but they are a bit of a red herring when you are starting out. Generally, Cycling74's attributes are only necessary if you want to access the properties from other objects. Instead, you can use your own 'property setter' (as in the first example above) to set any global variable with a message to the js object from your patch:

myGlobalValue = 0;
setMyGlobalValue(x){ myGlobalValue = x; )

So the patch can send a "setMyGlobalValue 1" message to the JavaScript object, after which any function in your script can access its value via the global variable. A 'getter()' method isn't necessary at all by this technique.

The Max 'attributes' on the js object do have two benefits. First, you can remove the variable from the global namespace. Second, other objects in your patch can easily access the attribute. However, if you're not wanting to access the attribute value from another object outside the js object, the attribute merely becomes another property that must be traversed in the object member list, as detailed above. So beyond external access, it's more of a programming convenience to get and set Max's attributes in the js object, and doesn't really improve script performance.

Conclusion

The next tutorials will describe how to get things from gen~ via a shared buffer; sending buffers to multisliders and plot~ objects; drawing with the Max jsui object; and lastly, callbacks for repetitive and time-delayed functions, which most people find the most difficult thing to do. Also they will introduce the document object model (DOM) which includes all the reserved names and explains the dots properly.

But as mentioned above, in the majority of cases, you don't need to know more than a half dozen of Cycling74's extensions to JavaScript, and it's mostly just finding the ones you want. Thank the stars for that, because JavaScript is indeed a very odd language. As I've probably already turned your head into a pretzel, this tutorial has reached an early mind-bending end.

----------------------

With special thanks to Ginger Edighoffer, for her patience in teaching me the innards of JavaScript, and to my manager Sheree Warburton, who let me practice despite it not strictly being part of my job description :)