Parsing/Tokenizing Data Streaming From Serial Object

Apr 12, 2010 at 11:24pm

Parsing/Tokenizing Data Streaming From Serial Object

Max Community,

I’ll preface the following by saying that I’m not a Max programmer, but I am a C programmer. So, I’m writing a patch in which the serial object is receiving packets from an Arduino microcontroller formulated in either of the following two strings:

Ax300y350z400d360rn
Bx400y500z600d0rn

The first packet represents a string sent by radio A (hence the letter A at the beginning of the string), while the second packet represents radio B. In both cases, the characters x, y, z, d, r (carriage return), and n (newline) are always included in the string/packet. Moreover, x always appears as the 2nd character, y always as the 6th, and z always as the 10th. (This occurs because three characters always appear between x and y, three characters always appear between y and z, and three characters always appear between z and d. The carriage return (r) and newline (n) characters, however, don’t always appear in the same place, since the string between d and r is sometimes made up of one char, sometimes of two, and sometimes of three. (d is for direction in degrees, so, sometimes the direction is 1 degree north, or 359 degrees from north.)

Now that you’ve read the background on the string formulation, I can ask: How can I parse/tokenize data coming in from Max’s serial object according to delimeters? In this case, the delimeters are A (B in the other case), x, y, z, d, and r (I can ignore the n character). Put another way, how can I extract the numerical values appearing between the non-numeric characters?

Thank you,
Roy

PS I’m using Max 5.0.7

#49716
Apr 12, 2010 at 11:41pm

A regexp object or two could probably do this if you were up on the regexp syntax, which I struggle with each time I have to use it.

You could also iterate through the string with iter and atoi and parse the hard way.

#178573
Apr 13, 2010 at 12:25am

Try [regexp ([AB])x(\d{3})y(\d{3})z(\d{3})d(\d{1,3})] and then use a [route A B] on the second outlet. You should now have your lists of numbers. This expression assumes that the numbers will all be positive, so no minus signs to watch out for. I hope it helps.

lh

#178574
Apr 13, 2010 at 12:40am

another regexp solution:

– Pasted Max Patch, click to expand. –
#178575
Apr 13, 2010 at 3:03pm

Christ, Luke, and Emmanuel: thanks so very much for your speedy replies.

I’m currently experimenting with Luke’s suggestion to use regular expressions, about which I know very little. I researched the syntax Chris mentioned in the RE (regular expression) above, but I can’t find anything explaining the last component in the RE:

...d(\d{1,3})

I’m guessing it means one, two, or three digits?

#178576
Apr 13, 2010 at 5:29pm

Yes:

\d means a digit
{ } is a quantifier to determine how many times the “thing” before appears. in this case from 1 to 3.

http://www.perlmonks.org/?node_id=967

#178577
Apr 13, 2010 at 6:05pm

Emmanuel: thanks for clearing that up. I knew I was close.

So, I included the regexp object (with the syntax that Luke provided above) into my patch. To test it, I included a few test messages that resemble what’s coming into the regexp object’s inlet via the serial object. (See the small attached Max patch.) When I click on the test messages, the patch works fine, sending the data I need out of the regexp‘s backreferences outlet. However, when I run the patch with live serial data, all of the data streams out of the regexp‘s unmatched inlet. What am I doing wrong?

#178578
Apr 13, 2010 at 6:40pm

I think that the problem is that regexp is expecting to chew on messages, and what you’re presenting it with is a bunch of ascii bytes. Either collect the bytes into packets, or build a state machine to parse the bytes as they come in. This edit of your patch is untested, but is in the ballpark for gathering things into a message:

– Pasted Max Patch, click to expand. –
#178579
Apr 13, 2010 at 6:52pm

Chris: thanks for the advice, and for editing my patch. I don’t, however, know what to make of the strings that you and Emmanuel have included in your posts. That is, the data between

– Pasted Max Patch, click to expand. –
#178580
Apr 13, 2010 at 7:07pm

Those are patches, pasted as text:

http://cycling74.com/forums/topic.php?id=25890

#178581
Apr 13, 2010 at 7:12pm

Totally got it, Emmanuel. Now I can see your original patch from yesterday.

#178582
Apr 13, 2010 at 7:38pm

btw, I think Chris meant to invert itoa location. The idea is that you group the ASCII values together, and use the carriage rerturn to know when the “word” is finished. Then you can convert it to a string and parse it with regexp. Some prefer to do all the parsing on the ASCII numbers… YMMV.

– Pasted Max Patch, click to expand. –
#178583
Apr 13, 2010 at 8:19pm

Toldja it was untested. :-)

#178584
Apr 14, 2010 at 10:38pm

Thank you all for your help, especially Emmanuel and Chris. Everything works fine. Once I finish commenting the patch, I’ll post it here for reference.

Best regards,
Roy

#178585
Nov 25, 2010 at 3:49am

Hi there,

I’m getting analog sensor values from the serial port. An ‘a’ and a ‘b’ character is appended before each value. Therefor this is what I get from the serial port ( after fromsymbol/itoa) printed:

print: b0
print: a0
print: b0
print: a0
print: b0
print: a3
print: b0
print: a94
print: b0
print: a198
print: b0
print: a230
print: b0
print: a263
print: b0
print: a300
print: b0
print: a344
print: b0
print: a394
print: b0
print: a418
print: b0
print: a444
print: b0
print: a488
print: b0
print: a508
print: b0
print: a535
print: b0
print: a550
print: b0
print: a564
print: b0
print: a572
print: b0
print: a585
print: b0
print: a595
print: b0
print: a606
print: b0
print: a616
print: b0
print: a626
print: b0
print: a632
print: b0
print: a637
print: b0
print: a643
print: b0
print: a645
print: b0
print: a649
print: b0
print: a658
print: b0
print: a662
print: b0
print: a667
print: b0
print: a670
print: b0
print: a672
print: b0
print: a676
print: b0

I would like to parse the two inputs values separately. How can I use the characters to parse values from he list? Looking forward to your kind help!

#178586
Nov 25, 2010 at 9:37am

you can use [spell] or [regexp] as mentioned. try [spell] first and use a [select] to distinguish between the “a” and “b” ASCII numerical values.

#178587
Jan 16, 2011 at 8:46pm

Hi there,
I’m attempting to use the serial object to extract the data received by usb from an xbee and interpret it. I can see the data in x-ctu however i can’t access it through max/msp at the moment
Does anyone know the best way of doing this?
Kind Regards
Miles

#178588

You must be logged in to reply to this topic.