Jitter Net Message Format? Jit.net.send to Arduino

Hi all,

I'm currently trying to find the fastest method of getting relatively low-res video data (12,075 pixels, RGB) at decent fps (30-60 ideally) from Max to an Arduino via ethernet. An idea I've had during this process has been to skip creating a message format using a bunch of intermediary max/jit objects by instead using jit.net.send to directly send the data to the arduino as if it were another device with Max running.

I've found that jit.net.send won't send anything unless it detects a jit.net.recv on the other end of the connection, so I can't just blindly send a matrix into it and expect it to show up by checking for TCP data on the arduino. Is there any info available, does anyone know the details, or Cycling74 can you share, how jit.net communicates so I might be able to get jit.net communication happening with a non-Max-capable device?

So far I've found that the two objects seem to perform some kind of handshake upon connection and upon each matrix send. The initial connection without a matrix attached looks like:
1. host (jit.net.send) sends TCP header-only 66 byte packet with SYN flag
2. client (jit.net.recv) returns TCP header-only 66 byte packet with SYN, ACK flags
3. host sends TCP header-only 54 byte packet with ACK flag
This is in line with what I understand is required connection establishment process for TCP, except that the expected rolls are reversed (i.e. jit.net.recv is apparently the server, not the client) (https://en.wikipedia.org/wiki/Transmission_Control_Protocol#Connection_establishment)

After the connection is established, data transmission looks something like:
1. jit.net.send sends as many packets as required to transmit the matrix, broken into packets containing up to 1460 data bytes, plus a 54 byte ACK header. The final packet required (and the first, if the matrix fits into 1 packet) also has the PSH flag set. For the first packet sent after receiving a reply from jit.net.recv has a Seq number equal to the last received ACK number, and any data-containing packet following this required for the same matrix has a Seq number equal to the Seq of the previously sent packet + the data size of the previously sent packet.
2. jit.net.recv returns ACK header-only packets for some of the data-containing packets, if there was more than one packet required to send the matrix. In very brief testing with a matrix requiring 9 packets, only 3 or 4 ACKs were sent back with numbers corresponding to the sequences of the data-containing packets.
3. jit.net.recv returns a PSH, ACK packet, with Seq number equal to the ACK of the packet it was sent, and ACK number equal to the Seq of the packet it was sent + the length of the packet it was sent. However, this seems to get a bit jumbled when sending matrices larger than 1 packet in size.

I writing this, I've realised I need to get the Arduino doing the handshake before I can progress further, so that's task number 1 right now...

Lastly, I also found that the matrix data, in packet/s containing it, is structured with some kind of custom "header" type info, followed by the matrix data in rows as raw bytes, and trailing zeros following each row to make up any gap to an even multiple of 16. When the data has to span more than one packet, the packets after the first do not have this custom header, i.e. they're just a continuation of the matrix data, and the size of the custom header seems to be 296 bytes (note 1). This was great to find, as it should make getting the matrix data easy once the connection is established, but I wonder if that custom "header" type stuff can give any extra information which might be useful, e.g. for keeping track of the beginning and end (and size) of that frame of matrix data. If anyone has this info and can share it, that'd be fantastic!

Cheers for any help!

Note/edit 1: Turns out this applies for a 1 plane matrix, 2 dimension matrix, but I've since found this number increases with various increases in plane count and dimension count