Forums > MaxMSP

### Parsing/Tokenizing Data Streaming From Serial Object

Apr 12 2010 | 11:24 pm

Max Community,

I’ll preface the following by saying that I’m not a Max programmer, but I am a C programmer. So, I’m writing a patch in which the `serial` object is receiving packets from an Arduino microcontroller formulated in either of the following two strings:

`Ax300y350z400d360rn`
`Bx400y500z600d0rn`

The first packet represents a string sent by radio A (hence the letter A at the beginning of the string), while the second packet represents radio B. In both cases, the characters `x`, `y`, `z`, `d`, `r` (carriage return), and `n` (newline) are always included in the string/packet. Moreover, `x` always appears as the 2nd character, `y` always as the 6th, and `z` always as the 10th. (This occurs because three characters always appear between `x` and `y`, three characters always appear between `y` and `z`, and three characters always appear between `z` and `d`. The carriage return (`r`) and newline (`n`) characters, however, don’t always appear in the same place, since the string between `d` and `r` is sometimes made up of one char, sometimes of two, and sometimes of three. (`d` is for direction in degrees, so, sometimes the direction is 1 degree north, or 359 degrees from north.)

Now that you’ve read the background on the string formulation, I can ask: How can I parse/tokenize data coming in from Max’s `serial` object according to delimeters? In this case, the delimeters are `A` (`B` in the other case), `x`, `y`, `z`, `d`, and `r` (I can ignore the `n` character). Put another way, how can I extract the numerical values appearing between the non-numeric characters?

Thank you,
Roy

PS I’m using Max 5.0.7

Apr 12 2010 | 11:41 pm

A regexp object or two could probably do this if you were up on the regexp syntax, which I struggle with each time I have to use it.

You could also iterate through the string with iter and atoi and parse the hard way.

Apr 13 2010 | 12:25 am

Try [regexp ([AB])x(\d{3})y(\d{3})z(\d{3})d(\d{1,3})] and then use a [route A B] on the second outlet. You should now have your lists of numbers. This expression assumes that the numbers will all be positive, so no minus signs to watch out for. I hope it helps.

lh

Apr 13 2010 | 12:40 am

another regexp solution:

``````
-- Pasted Max  Patch, click to expand. --

```
```
Apr 13 2010 | 3:03 pm

Christ, Luke, and Emmanuel: thanks so very much for your speedy replies.

I’m currently experimenting with Luke’s suggestion to use regular expressions, about which I know very little. I researched the syntax Chris mentioned in the RE (regular expression) above, but I can’t find anything explaining the last component in the RE:

`...d(\d{1,3})`

I’m guessing it means one, two, or three digits?

Apr 13 2010 | 5:29 pm

Yes:

`\d` means a digit
`{ }` is a quantifier to determine how many times the "thing" before appears. in this case from 1 to 3.

http://www.perlmonks.org/?node_id=967

Apr 13 2010 | 6:05 pm

Emmanuel: thanks for clearing that up. I knew I was close.

So, I included the `regexp` object (with the syntax that Luke provided above) into my patch. To test it, I included a few test messages that resemble what’s coming into the `regexp` object’s inlet via the `serial` object. (See the small attached Max patch.) When I click on the test messages, the patch works fine, sending the data I need out of the `regexp`‘s `backreferences` outlet. However, when I run the patch with live serial data, all of the data streams out of the `regexp`‘s `unmatched` inlet. What am I doing wrong?

###### Attachments:
1. Serial_Object_That_Tokenizes_Data-f.maxpat
Apr 13 2010 | 6:40 pm

I think that the problem is that regexp is expecting to chew on messages, and what you’re presenting it with is a bunch of ascii bytes. Either collect the bytes into packets, or build a state machine to parse the bytes as they come in. This edit of your patch is untested, but is in the ballpark for gathering things into a message:

``````
-- Pasted Max  Patch, click to expand. --

```
```
Apr 13 2010 | 6:52 pm

Chris: thanks for the advice, and for editing my patch. I don’t, however, know what to make of the strings that you and Emmanuel have included in your posts. That is, the data between

``` -- Pasted Max Patch, click to expand. -- ```

Apr 13 2010 | 7:07 pm

Those are patches, pasted as text:

http://cycling74.com/forums/topic.php?id=25890

Apr 13 2010 | 7:12 pm

Totally got it, Emmanuel. Now I can see your original patch from yesterday.

Apr 13 2010 | 7:38 pm

btw, I think Chris meant to invert itoa location. The idea is that you group the ASCII values together, and use the carriage rerturn to know when the "word" is finished. Then you can convert it to a string and parse it with regexp. Some prefer to do all the parsing on the ASCII numbers… YMMV.

``````
-- Pasted Max  Patch, click to expand. --

```
```
Apr 13 2010 | 8:19 pm

Toldja it was untested. :-)

Apr 14 2010 | 10:38 pm

Thank you all for your help, especially Emmanuel and Chris. Everything works fine. Once I finish commenting the patch, I’ll post it here for reference.

Best regards,
Roy

Nov 25 2010 | 3:49 am

Hi there,

I’m getting analog sensor values from the serial port. An ‘a’ and a ‘b’ character is appended before each value. Therefor this is what I get from the serial port ( after fromsymbol/itoa) printed:

print: b0
print: a0
print: b0
print: a0
print: b0
print: a3
print: b0
print: a94
print: b0
print: a198
print: b0
print: a230
print: b0
print: a263
print: b0
print: a300
print: b0
print: a344
print: b0
print: a394
print: b0
print: a418
print: b0
print: a444
print: b0
print: a488
print: b0
print: a508
print: b0
print: a535
print: b0
print: a550
print: b0
print: a564
print: b0
print: a572
print: b0
print: a585
print: b0
print: a595
print: b0
print: a606
print: b0
print: a616
print: b0
print: a626
print: b0
print: a632
print: b0
print: a637
print: b0
print: a643
print: b0
print: a645
print: b0
print: a649
print: b0
print: a658
print: b0
print: a662
print: b0
print: a667
print: b0
print: a670
print: b0
print: a672
print: b0
print: a676
print: b0

I would like to parse the two inputs values separately. How can I use the characters to parse values from he list? Looking forward to your kind help!

Nov 25 2010 | 9:37 am

you can use [spell] or [regexp] as mentioned. try [spell] first and use a [select] to distinguish between the "a" and "b" ASCII numerical values.

Jan 16 2011 | 8:46 pm

Hi there,
I’m attempting to use the serial object to extract the data received by usb from an xbee and interpret it. I can see the data in x-ctu however i can’t access it through max/msp at the moment
Does anyone know the best way of doing this?
Kind Regards
Miles

Viewing 17 posts - 1 through 17 (of 17 total)

Forums > MaxMSP