Nick Suda

Love Has No Labels is an ad campaign by the Ad Council which addresses our inherent internal biases, diversity, inclusion, and love across classifications. The original video has accumulated over 100 million views between Facebook, YouTube and mirrors combined.

Conceptually, the project required creating a "live X-Ray machine," where all an audience could see was non-descript skeleton figures embracing, kissing, playfully interacting, and so on. Technologically, this implied a live motion capture rig that required little setup, so that actors could easily enter and exit the stage and still have the tech appear relatively invisible.

In using an Xsens IMU (cameraless) mocap setup, the potential for accumulating error and drift in the coordinate system of our digital avatars was a very real concern. This is compounded by using two suits in constant, close proximity relationship to each other.  Therefore, we researched how we could re-adjust the position of our skeleton avatars in realtime.

We discovered that Translation parameters (as well as pretty much anything else you'd want to do) are exposed openly in Maya (our CG platform) by the Maya Embedded Language, or MEL.

Furthermore, some of the skeletons' most important gestures - such as opening and closing jaws for laughing, kissing, and talking, and opening and closing hands for waving, holding hands - are not articulated by the motion capture suits. So, our animator exposed meta-parameters for articulating these parts with keyframes indexed from 0 to 10.

We then created a Max patch that opened up a TCP connection to the Maya rendering computer using a stock mxj TCP object. Over this socket, we took in CCs from a MIDI controller and wrapped them into coherent MEL command strings on their way out.

With eyes on camera feeds inside and outside of the stage box, avatar translation was placed on rotary knobs, and fists and jaws lived on the vertical faders of a Livid Ohm RGB for the most performative articulation. This effectively became a secondary puppeteering stage for our realtime mocap rig.

love-no-labels

Love Has No Labels

Chris Mizerak

OSCRouter is an easy-to-use UDP/TCP packet router for Mac and Windows, with optional OSC (Open Sound Control) specific options.  Full source code available under open source license (MIT).

oscrouter

OSCRouter

Ben Nevile

In order to maximize the potential of the built-in networking classes that are new with version 4.5 of Max, it's helpful to understand a few things about the way that modern network protocols operate. This article is intended to help you get off the ground by answering some basic networking questions.

Q: What are the differences between the net.maxhole, net.udp.*, net.multi.* and net.tcp.* classes?


Which of the above classes to choose depends on your application. All of them are based on either UDP or TCP, the two dominant protocols that computers use to communicate in today's networks. To communicate with the internet these protocols both use what is called the IP (Internet Protocol) layer, which is why you sometimes see acronyms like TCP/IP.

UDP (User Datagram Protocol) is the simpler of the two protocols. To send data via UDP, all that's required is a destination port and IP address. When using UDP your data is wrapped into a packet with a header that, among other things, contains the packet's destination address. UDP does not provide any feedback on the status of the packet after it has been sent, so if for some reason the packet can't be delivered to the destination - the internet goes down, someone trips over a network cable, the receiving computer isn't listening on the proper port, etc - the application that sent the packet will never know. Furthermore, UDP provides no guarantee that the packets you send to another host will arrive in the order that they were sent. If there is a single, well-defined route between the computers the probability of packets arriving out-of-order is close to zero. When data is being transmitted over the internet, however, different packets can take different routes to the same destination, and consequently the probability of UDP packets arriving out-of-order is high. The higher your rate of transmission, the higher your probability that packets will arrive out-of-order.

Like UDP, TCP (Transmission Control Protocol) sends data in packets. However, extra information is embedded in the packet headers to help the protocol track the packets it has sent. When a computer receives a TCP packet it sends back a small ACK (acknowledgement) message which includes a packet ID. This allows TCP to track which packets have been received successfully by the remote computer. If the sender does not receive an ACK within a certain time the packet is re-sent. Because each packet is labelled chronologically, the TCP implementation on the receiving computer is able to sort the incoming packets into the proper order even if they were received out-of-order.

These differences between the protocols are evident in an examination of the Max objects provided with v4.5 for use with the mxj Java object. net.udp.send has an inlet to take data to send to the specified address and port, but no outlets that report back on the status of the packets. net.tcp.send on the other hand has three outlets that report on the status of the data you've input to be sent: one outlet reports what data was sent successfully, another reports what data was not sent successfully, and a third outlet reports the number of packets that are still "in limbo" - that is, no acknowledgement of their successful transmission has been received, but the algorithm hasn't given up on them yet.

To make the back-and-forth communication between TCP hosts possible, the protocol requires that a connection be made between the two computers prior to sending any data. No such connection is required with UDP. Having to make this connection causes a delay before any data can be sent. The time delay is roughly equivalent to the RTT (round trip time) that it takes a packet to reach the destination and come back. For applications sending a lot of data the cost of this connection is mortgaged over the time it takes to send all of the data, and therefore is usually negligible.  Currently with net.tcp.send and net.tcp.recv, the TCP connection is created and destroyed for each new piece of input data to be sent, similar to HTTP. This results in less efficient transfer of data, but allows for one net.tcp.recv object to receive data from multiple clients. In a future version the object may allow a TCP connection to remain intact between data transfers. Some examples of common applications that use TCP are SMTP (email), Telnet (remote login), FTP (file transfer) and HTTP (the www). Some examples of applications that use UDP are DHCP (bootstrap IP protocol), NTP (network time protocol) and Traceroute.

net.maxhole was built in order to make simple network commmunication as easy as possible. Simply instantiating net.maxhole objects in the default form allows instant networked communication without having to think about IP addresses, ports, or protocols. net.maxhole uses multicasting, which is just a UDP transmission to a special kind of IP address that every computer will receive. So whereas with regular UDP you specify a single receiver and the packet is intelligently routed to just that one machine, with a multicast address the routers send the packet to every computer. The range of a multicast packet is controlled by a special piece of data called the time to live that exists in every IP packet header: every time the packet is received by a router, the time to live is decreased by one, and when the time to live reaches zero, the packet is destroyed. For net.maxhole the time to live is 1, so it will only be passed through a single router to other machines in your local network. For more adventurous multicasting, the net.multi.send implementation of the multicasting protocol allows you to set the time to live via the "maxhops" attribute. An example of a system that uses multicasting is Apple's Rendezvous service-discovery technology (which other organizations call ZeroConf.)

In summary, if your networking demands are modest the easiest object to use is net.maxhole. It should be appropriate for most small local network situations you can throw at it. If communication latency and throughput is of utmost importance, and you don't want to multicast to all the machines on the network, then UDP is worth investigating. If reliability is important, then using TCP is recommended.

Q: How can I send and receive data through a router's firewall?

When thinking about networking it is useful to think in terms of clients and servers. A server process sits and waits for a connection on a port, and a client process initiates the communication to the server. In terms of the Max networking classes, net.tcp.recv is a server, and net.tcp.send is a client. A connection between a client and a server has four important pieces of data: the server's IP, the server's port, the client's IP and the client's port.

TCP and UDP both define a group of well-known ports to identify well-known services: FTP is 21, HTTP is 80, etc. Well-known ports tend to live in the range between 0 and 1024. Clients, however, use ephemeral, or short-lived ports. These port numbers - which typically range between 49152 and 65535, depending on the OS - are automatically assigned to a client by the UDP and TCP code that lives in the operating system's kernel. UDP and TCP guarantee the uniqueness of the ephemeral ports they assign, which prevents conflicts in communication between different client processes. This scheme is what allows your computer to have more than one active FTP session going, or be able to download web pages from more than one site at the same time.

A server that wants to simultaneously handle more than one process also needs to use its own ephemeral ports to create connections so that it can keep listening for connection requests on its well-known port. For example, if you make a FTP request to 142.34.56.222:21 from 145.35.52.65:57474, the server may request that the connection be made on port number 50423 instead of 21. ie, the final connection would be between 142.34.56.222:50423 and 145.35.52.65:57474.

Local area network routers add more complication. To the outside world all the computers behind the router have the same IP address, so the router must distribute all incoming packets based on the port. The complication is that each computer controls its own ephemeral ports, and so there is the potential for conflict. For instance, if two computers behind the same firewall both request FTP sessions with outside sources, there is nothing preventing both of the computers from assigning port 50000 to the request.

The solution, which is called NAT (Network Address Translation), ensures that there is no conflict between ports by changing the outgoing port numbers without any of the computers knowing about it. When a router sees a new outgoing request it changes the ephemeral port number for the client computer to a new port number from a table maintained by the router. The remote server returns the request to the port that the router set up, and then before passing the data back to the client computer the router changes the port back to the number that the client is expecting.

Most routers have a built-in firewall that stops incoming data before it reaches computers on the local area network. Usually the router provides a method of configuring the firewall so that an administrator can decide which ports should be open and closed, and which computers in the LAN should receive the data sent to the open ports. When initiating a TCP connection from a client within the LAN to a server outside the firewall it should not be necessary to explicitly configure the firewall since the router will adjust the firewall's rules to appropriately pass data. UDP, however, is a connectionless protocol, so to receive or send UDP packets through a router the port must be explicitly opened by an administrator.

max-networking-basics

Max Networking Basics

Arvid Tomayko

A Max object written in java (using mxj) to communicate back and forth over a single persistent TCP/IP socket with a TCP/IP server, something the net.tcp objects that ship with Max can't do. Opens a socket, then sends and receives info without closing it. Closes the socket only when you tell it to, or when the server disconnects (ie from inactivity). 

Useful for connecting to simple hardware network devices like EZ Web Lynx and other microcontrollers/DSPs that don't easily support something nicer like OSC, or any TCP Protocol that sends data back on the same socket.

2013-11-09 - New in 0.6: correct reporting of Nagle's Alogrithm (noDelay) use, attributes are output as messages rather than single symbols in response to get commands, outlet response for Nagle's Algorithm (noDelay)

Tested on MaxOS X 10.5, 10.6, 10.8 and Windows XP, Max/MSP 5.1.x and 6.1.5

Developed by Arvid Tomayko-Peters for Your Heaven, LLC™

Thanks to David Reilly for his TimedSocket class.


tcpclient

tcpClient

So you want to move data between two or more computers in real time. It could be just a little data, such as individual Max messages sparsely distributed in time, or a lot of data, such as a stream of matrices or one or more audio channels. How do you do it?

Networking is a sometimes confusing world. There are many networking options built into Max, and this article will endeavour to make clear which option is best for your application.

We could start our discussion with a whole lot of information about protocols, ports, addresses, and all sorts of other computery details that would make the nerds among us squeal with delight. But let's assume for a moment that unlike me, you have better things to do with your time than worry about implementation details. Maybe you just want a quick and easy way to be able to send information from one computer to another.

The net.maxhole Java class was built in order to make simple network commmunication as easy as possible. Simply instantiating net.maxhole objects in the default form allows instant networked communication without having to worry about any other details.

In the image above, an integer has been sent into the left-hand maxhole, and it has come out of the outlet of both maxholes. This illustrates that every maxhole is both a sender and a receiver of data: any data sent into a maxhole is repeated out by all other maxholes. This includes net.maxhole objects in the same patch, on the same computer, or on different computers. Notice that net.maxhole is not given a name as an argument, as you would do with send and receive to set up distinct message paths. Accomplishing this type of filtering is easy with a post-maxhole route object.

How does it work? The simplified explanation is that every maxhole is always waiting for messages from another maxhole. When a Max message is sent into maxhole's inlet, it broadcasts the data to a networking address that all the other maxholes are watching. When the other maxholes see the message being broadcast, they send this message out their outlets.

So if you're following along at home, you may have opened up the net.maxhole help patch on two different computers. If it's working, great! Stop reading this article and get on with your life.

If it's not working, and you're sure both computers are connected to the same network, you probably have a problem with firewalls. Every "packet" of information spread through a network has an address and a port number. A firewall is a gate that only lets certain addresses and ports pass. Firewalls are implemented at many different points in a modern networking architecture.

The above diagram shows a basic networking setup: two computers are connected to a wireless router, which is in turn connected to the internet. The router has a firewall for its connection to the outside world, and a different firewall for the internal connections. Each computer has a firewall built into the networking component of its operating system. Not illustrated is the fact that it's also possible that the network hardware in the computer, such as the ethernet card or the wireless antenna, has its own firewall.

net.maxhole sends data out on port 7474. So if computers A and B want to use maxholes to talk to one another, they must ensure that all the firewalls in between have port 7474 open. With most personal, local area network routers it should not be a problem for the multicasting address that net.maxhole uses to pass through the firewall unimpeded. However, large institutional firewalls are often more selective, so it may not be practical to use net.maxhole to communicate with machines elsewhere on the internet.

So for one reason or another, you may have discovered that net.maxhole isn't appropriate for your application. Perhaps you want to communicate beyond the boundaries of your local network, but you don't have permission to open ports in your organization's firewall. Let's drop a level deeper into network world and learn some details.

UDP and TCP are the two dominant protocols that computers use to communicate in today's networks. To communicate with the internet they both use what is called the IP (Internet Protocol) layer, which is why you sometimes see acronyms like TCP/IP. UDP (User Datagram Protocol) is the simpler of the two protocols. To send some data via UDP, all that is required is a destination port and IP address. When using UDP your data is wrapped into a "packet" with a header that indicates the destination address and some other information, and then sent out through your network interface.

Let's examine the help files for net.udp.send and net.udp.recv:

net.maxhole sends UDP packets to a special multicasting address that broadcasts one message to multiple machines.  In the case of the net.udp.send object, you specify the destination address and port that incoming messages should be sent to - so you're no longer broadcasting, you're sending data to just one machine. Accordingly, the port for net.udp.recv must be the same as the sender.

Like net.maxhole, net.udp.send and net.udp.recv are Java classes that run within the mxj object.  It's also possible to use the udpsend and udpreceive objects to send Max messages as UDP packets. These objects are written in C and do not depend on Java. They also feature a special mode that emulates CNMAT's otudp external, a popular networking object originally developed at the University of California, Berkeley, for pre-OS X Apple computers.

You may have noticed in the help patch that the net.udp.send object is initialized with an IP address of 127.0.0.1.  This is the special local loopback address. Any data sent to that address is not sent out on the networking hardware, but rather is fed back into the network hardware's input buffer. It's a convenient address to use for the help patches since it allows the net.udp.recv help patch to receive data from the net.udp.send patch.

This may lead you to another question: how can you find out the IP address of the machine you want to send data to? We'll talk more about this in a later section.

UDP does not provide any feedback on the status of the packet after it has been sent. So if for some reason the packet can't be delivered to the destination - the internet goes down, someone trips over a network cable, the receiving computer isn't listening on the proper port, etc - the application that sent the packet will never know. Furthermore, UDP provides no guarantee that the packets you send to another host will arrive in the order that they were sent. If there is a single, static route between the computers the probability of packets arriving out-of-order is close to zero. When data is being transmitted on the internet, however, different packets can take different routes to the same destination, and consequently the probability of UDP packets arriving out-of-order is high. The faster you send data, the higher the chance that packets will arrive out-of-order.

Like UDP, TCP (Transmission Control Protocol) sends data in packets. However, extra information is embedded in the packet headers to help the protocol track the packets it has sent. When a computer receives a TCP packet it sends back a small ACK (acknowledgement) packet to the compuer that sent the packet. This allows the TCP to track which packets have been received successfully by the remote computer. If an ACK is not received within a certain time of sending a packet, the packet is re-sent. Because each packet is labelled chronologically, the TCP on the receiving computer is able to sort the incoming packets into the proper order even if they were received out-of-order.

To make the back-and-forth communication between TCP hosts possible, the protocol requires that a relationship be formed between the two computers prior to sending any data. No such connection is required with UDP. Having to make this connection causes a delay before any data can be sent. The time delay is roughly equivalent to the RTT (round trip time) that it takes a packet to reach the destination and come back. For applications sending a lot of data the cost of this connection is spread over the time it takes to send all of the data, and therefore is usually negligible. Some examples of common applications that use TCP are SMTP (email), Telnet (remote login), FTP (file transfer) and HTTP (the interweb). Some examples of applications that use UDP are DHCP (bootstrap IP protocol), NTP (network time protocol) and Traceroute.

The difference between the protocols is evident in an examination of the net.tcp.send and net.tcp.recv help files:

net.udp.send has an inlet to take data to send to the specified address and port, but no outlets that report back on the status of the packets. net.tcp.send on the other hand has three outlets that report on the status of the data you've input to be sent: one outlet reports what data was sent successfully, another reports what data was not sent successfully, and a third outlet reports the number of packets that are still "in limbo" - that is, they have not been successfully transmitted but the algorithm hasn't given up on them yet. For some applications this type of feedback is very important. The tradeoff is that this feedback increases the communication overhead of the connection, and therefore UDP is slightly more efficient.

So far all the examples we've shown are useful only for transmitting Max messages. What if you want to send media data? Can you simply insert a jitter matrix patch cord into a net.maxhole object? You can, but what would be transmitted to the other maxholes would be a message such as "jit_matrix u911000001". Jitter patch cords pass references to matrices by name; in the case above, u911000001 is the name of a matrix on the sending computer. A matrix with this name may or may not exist on the receiving computer, and even if it does exist the data could be completely different!

Part of the Jitter 1.5 upgrade were the jit.net.send and jit.net.recv objects, whose help files are shown below:

The jit.net.* objects access the input matrices and use TCP to transmit the data. (Why TCP? In our internal testing, we found no speed advantage by sending the data with UDP, and using TCP ensures that frames do not arrive out-of-order. This is a good example of the cost of a TCP connection being small compared with the amount of data being sent.) Like the net.tcp.* objects, an IP address and a port number are attributes of the objects.  The matrices that are input into a jit.net.send object are transmitted in order to its connected jit.net.recv object. The rate at which matrices can be transmitted depends on the size of the matrices and the speed and reliability of the network connection between the two objects.

Since the jit.catch~ and jit.release~ objects can transcode between the signal and matrix worlds, the jit.net.* objects can be used to send audio over the network.  The audio-over-network example patch, which is included in the standard Jitter distribution, is pictured below.

A frequent point of confusion with these networking objects is that it's not always straightforward to find out the IP address of the computer that you're using. The net.local class was designed to help with this:

Sending a bang to net.local causes it to populate a umenu object connected to net.local's right outlet.  From this menu one can select and send the name of one of the computer's networking interfaces to net.local, which causes the IP addresses of that interface to be send to a umenu connected to the left outlet. The result is a cross-platform method of determining the proper IP address. One could imagine a strategy where net.maxhole is used to communicate the IP addresses of computers to one another, and then one of the more specific networking protocols is used to transmit the data.

So it turns out there are a bunch of ways to move data between computers.  If your networking demands are modest, the easiest object to use is net.maxhole. It should cover almost any small local network situation you can throw at it. If communication latency and throughput is of utmost importance, and you don't want to multicast to all the machines on the network, then the net.udp.* objects are worth investigating. If reliability is important, the net.tcp.* objects may be the right choice, and the jit.net.* objects are what you use to transmit media data.  To determine the IP address of a computer, you can use the net.local class.

Networking is a sometimes confusing world. There are many networking options built into Max, and this article will endeavour to make clear which option is best for your application. 

Search Results