Andrew Benson

Using Socket.IO to send and receive realtime data in Max .

Extracting dictionary data from JSON to a Jitter Matrix

Combining server, client, and web interface

Adding mobile interactions to a Max project

Connecting a Max patch to interactive data is a fundamental interest many of us share. This recipe provides a quick dive into Socket.IO, a full-featured realtime data package for Node that can handle everything from chat to image and data transfer. Here, we create a simple drawing interaction that has both a web and Max front end that can communicate together. This could serve as the first step in an interactive art installation or crowd-interaction performance experience when combined with the rich media available in Max.

ExpressJS : quick and easy web server package for node

Socket.IO : Node package for real-time communication over a network

dict.iter: unroll a dictionary into a series of smaller dictionaries

Socket.IO uses a web server to route communicate between different client processes. For this recipe we have a NodeJS script that runs this drawing_server in addition to a client drawing_friends patch/JS combo. The drawing_server also uses ExpressJS to serve up a public webpage that communicates via Socket.IO as well.

While Max most often works best using simple messages and lists, NodeJS tends to rely heavily on JSON data. In this patcher, we pass out the Socket.IO JSON data as a Max Dictionary and use dict.iter to unravel the data from each of the connected peers. 

To visualize the incoming peer data, we first pack the coordinates into a Jitter Matrix and send that along to jit.gl.multiple. Note that this patch is only rendering the most recent peer locations, but a little texture feedback is used to accumulate gestures.

To get started setting up the server, I worked from the Socket.IO Getting Started Tutorial which shows how to set up a simple messaging app. From there, it was simply a matter of determining how to format the communication between Max and the web page. The important thing to do is to make sure you create message handler functions on each end for any messages you’d like to use.

Similarly, much of the front-end site (inside /public folder) borrows code provided by Socket.IO, with some additional HTML5 Canvas handling. All of the physical interactions are handled by JQuery for the sake of ease.

For each client, we track the location (x, y) and whether they are “active”. On the webpage, this is done using canvas coordinates and in Max we use mouse coordinates from the jit.world window. These values are packed into a JSON object and broadcast as a message to Socket.IO.

In this recipe, the server is running locally, on your computer, and the website can only be accessed on your computer or local machines in your network, depending on your security settings. One way to make it accessible from any Internet-connected device is to use Heroku or a similar cloud service to host the drawing_server code and public website. 

Learn More: See all the articles in this series  

node00.png

node-recipe-00-socket-drawings

Node Recipe 00: Socket Drawings

Using ReactJS to generate a dynamic web front-end for Max.

When creating an art or commercial installation, it’s often helpful to create an admin page to schedule events and check status of the system. This recipe provides a template for running an audiovisual installation that responds to scheduled events and provides a form to create new events through a web front end. To accomplish all of this, we use a bunch of libraries that will be very familiar to JavaScript and NodeJS programmers, but might be new to Max users. Express provides routing services for the Node server, while React provides a reactive user interface in the browser. We also use Axios to simplify the HTTP communication between the web page and server.

ReactJS : dynamic front-end framework built around modular components

Axios: simplifies HTTP and JSON communication between front-end page and server

To get the React site working properly, you can either build an optimized production site or run the React development server. If you just want to get running right away, I recommend running the “script npm run build” message, which will compile all the React source files and make a page that our Max server can load. After that is built, and you’ve started the node.script, you can launch a browser to “localhost:5000” on your local machine.  While editing the React source files, you’ll probably want to run in development mode, which can be done by opening a terminal to the root folder of this recipe and typing in “npm start”. This will automatically launch a browser to a page at “localhost:3000” that updates when the React files change. Note that this React dev server communicates via proxy with our Max server on port 5000.

The node.script object outputs a variety of messages using “Max.outlet()” calls. These are passed around by using route or select.

We break down the different “scheduledevent” messages coming from node.script to provide triggers to the mock installation we’ve got running below. The web interface provides a menu to define these when creating a new event.

To create the React front end, I started off by bootstrapping the project using the create-react-app NPM library. From there it was mostly a matter of setting up the different components (EventEntry, EventEntries, EventForm, Stats) and how they were to interact with the API I created using Node and Express.

The React front-end communicates with our Max-driven Node server (server.js) through ExpressJS on the server and Axios on the front end. The server is set up to respond to the form inputs by storing new entries into a file. To see how that works see the functions starting with “app.get” or “app.post” in server.js and the React-side calls inside of App.jsx.

To store our event entries, we use a file-based storage system, which is largely implemented inside of services/events.js. It’s important to note that all functions that interact with the file system or call APIs make use of the “async/await” feature of NodeJS to run asyncronous code.

On the Node side of things, we set up Timeouts to trigger scheduled events, as well as sending occasional “report” messages that will trigger a status update from the Max patch.

Learn More: See all the articles in this series 

Screen Shot 2019-03-01 at 11.11.12 AM.png

In this recipe we put Node to work as a portal for us to control and build a schedule for an audiovisual installation patch via a dynamic web page. To accomplish this we lean on some popular web libraries and the built-in features of Node for Max to communicate with a running patcher.

node-recipe-01-installation-calendar

Node Recipe 01: Installation Calendar

Using an agent simulation library to generate complex behavior

Passing dictionaries between Max and Node

This time out, we're keeping things simple. Unlike previous recipes, we’re only using one NPM library (flocc), and making full use of its features to create a simple simulated crowd visualization. 

Simple simulations like this can be super fascinating as emergent patterns, and complex behaviors arise from a set of simple rules. Starting with a simple agent/flocking simulation, we’ve expanded it with an additional “solitude” behavior that changes each agent’s attraction to other agents based on how much time they’ve been near or apart from others. 

Consider what you would do with the output of such a simulation. How would you extend the behavior?

flocc : general purpose NPM simulation library

dict : pass a dictionary of state information between Max and Node

While our simulation is running, it sends out a dictionary for each clock tick that includes information about the simulation. This data was formatted to make it especially easy to pack into a Jitter Matrix using jit.fill after using jit.unpack to extract it.

Since there are a number of settings to configure for the simulation state, you can use a dictionary to avoid making a whole list of Max Handlers in the JS code. Inside of Javascript, we use Object.assign() to set the state that we’ve changed without erasing the existing contents.

To manipulate the simulation, try changing the values of the simulation settings. The settings break down like this:

Maxspeed: The maximum speed each agent can travel

Maxforce: The maximum speed change that can occur in each clock tick.

Awareness_range: sets the search radius that defines which other agents each agent responds to.

Goal: How much force to devote to traveling toward the target

Solitude: Determines how quickly avoidance behavior switches to attraction. Larger values mean greater time between those changes

Near: determines the radius to search when doing solitude testing

Too_close: sets the radius for general collision-avoidance behavior

The flocc library defines an “environment” which is the world in which the simulation will run. Once that is created, you can populate the environment with “agents” – individual entities that have internal state that exist in the world. Those agents can also have “rules” applied to them - functions that are called for each clock tick of the environment.

Behavior of each agent is determined by these rules. In our code, most of the actual behaviors are defined in the simbehavior.js file.

It should be noted that each agent in the simulation stores its internal state in a Javascript object (similar to a Max Dictionary). A lot of the work of creating a simulation like this is determining what will be considered a global state for the simulation as a whole and what states are associated with individual agents. 

The existing simulation behavior could easily be extended by adding new functions to the simbehavior.js file and calling them from inside the motion() function.

This time out, our Node For Max recipe is going to keep things simple. We’ll be using only one NPM library (flocc), and making full use of its features to create a simple simulated crowd visualization that should spark your imagination.

Search Results