How I am handling a medium-sized Max/js project

Since I just posted a request, I thought it only fair to add some actual content as a balance.

I do a live show and I want a way to "sequence" an entire show and control it from my instrument without ever having to touch the mouse during the whole show. I wanted a way I could hook together sets from songs without having to build new max patches for each set. I wanted a much better way to manage all the many sounds I have from my instrument.

To do this, I've been working on a moderately large Javascript program for Max - right now it's 1423 lines of Javascript, just over half of which is unit tests, as well as a half dozen data files (but I'm expecting a lot more data files (in JSON) once it's all working). I'm expecting the code code to grow to about 2000 lines with somewhat under half the code test code.

I wanted to share my experiences with doing "rapid development" this way and of course get feedback.

I don't think there was ever a point where I didn't expect to have to write tests, so I've been using some sort of hand-rolled, cheap unit test system since I started this. In code like this where I'm trying to bang out a lot of experimental stuff and make massive changes to the data structures all the time I simply have to have complete tests for everything - I have 45 tests right now.

I know rapid development and heavy unit testing seem to be contradictory, but I've banged around the data structures pretty hard by now in many different ways and for the last week or so I've been attempting to delete as many of them as possible and replace them with either lists or dictionaries (associative arrays).

And debugging is really hard... so might as well try to avoid it as much as possible!

Here's how it works.

I've divided up my code into many small Javascript files each containing something like one class or a few related functions - files with names like groups.js or midi.js.

I use the C preprocessor to put these together properly in exactly the same way you'd use include files in C (pretty simply if you don't know how that works).

So if my code is in command.js and includes other files, the preprocessor puts them together into a file called command.jso.

This is all accomplished with a simple Makefile and gcc.

Many developers have make and gcc installed or you can get free from here: http://developer.apple.com/tools/gcc_overview.html (and you should - everyone should have gcc for just this reason...)

There's a test subdirectory - with lots of JS test code and a tiny Max patch called tests.maxpat which just has a js box with tests.jso and a single message button "compile, test".

So how do I write non-trivial tests?

To pontificate a little, Javascript is a very advanced scripting language. I like Python a better (if nothing else, I don't have to do this bogus "#include" stuff in Python!) but all the Python features that allow me to do rapid development are there or at least possible.

Advanced languages such as these let you write code some significant factor more rapidly than older-style languages. I've written some number of 10^5 lines of C and C++ and barely 10^4 lines of Python/Javascript and yet I believe I'm as much as 3 times as productive in those languages as in C/C++ (and to a less pronounced state, Java).

So I make heavy use of mocks, where I create a "fake" version of say this.patcher that I use for testing which has methods that record what happens to them. At the end of running the test, I can look at what commands the mock patcher had to execute, and see if they are what I want.

And you have no idea how many bugs I've found this way before I ever played a note!

Here's how I mock "patcher" for example - though I expect this is somewhat overkill, I wanted to learn a little more about Javascript's capabilities.

Early on, I define:

var _patcher = this.patcher;

and always use _patcher in my code and never this.patcher.

When I run a test, I replace _patcher with a mock class called TestPatcher

_patcher = new TestPatcher();

that has all the methods on this.patcher I use, but instead of doing anything, simply records what happened in an internal array, "commands" and returns a mock object.

function TestPatcher(commands) {
  var maxobj_index = 0;
  var commands = [];
  var maxobjs = [];

  function push_command(name, args) {
    var command = [name];
    for (var i = 0; i < args.length; ++i)
      command.push(args[i].name || args[i]);
    commands.push(command);
  };

  function TestObject(type) {
    maxobjs.push(this);
    this.name = 'm-' + type + '-' + (maxobj_index++);
    this.message = function(_) {
      push_command(this.name + '()', arrayfromargs(arguments));
    };
  };

  function mock(name) {
    return function(_) {
      var args = arrayfromargs(arguments);
      push_command(name, args);
      return new TestObject(args[2]);
    };
  };

  var methods = ['connect', 'disconnect', 'newdefault', 'remove'];
  for (var i = 0; i < methods.length; ++i) {
    var method = methods[i];
    this[method] = mock(method);
  }
  this.commands = commands;
  this.box = 'js';
};

and now we're here, here's my Makefile:

PREPROCESS_CORE=gcc -E -P -C -w -x c
PREPROCESS=$(PREPROCESS_CORE) $(DEFINES)

all: command.jso test/tests.jso midi_main.jso

%.jso: %.js
  $(PREPROCESS) $< -o $@

clean:
  rm *.jso

test/tests.jso: command.jso test/*.js
command.jso: *.js

As this advances, I'll write up more of it - I hope this has some value to readers here in this limited state and don't hesitate to send me email with questions!