unpacking a packet of parsed data.
Hi, I need some help unpacking a packet of parsed data coming from an arduino.
I'm trying to collect data from an eeg device and use this data to control parameters of a synth built in max.
The data comes in through the serial device, and should be in a packet of eleven values. For instance
200, 0, 0, 103589, 698790,247658,278765,172456,567898,678543,456763
The values are updated and sent as a new packet once per second.
I need a way of unpacking, and separating the data, so that each of the eleven values can be fed into a separate number box.
I don't really know enough about what I'm doing. I worked out from reading the max tutorials that the data comming in is integer and it needs to be a different format to work in max?
If attached a basic patch I've tried to construct based on the max tutorial for unpacking serial data. I had it working a little bit when I spent a while playing with different setups and removing objects etc, but it was only collecting four values, not the eleven values that it should be reading.
for instance, it will generate numbers into the boxes
44, 50, 44, 55, 49, 48, 44, 50 ... etc
it also only seems to be updating once every three seconds, even though the metro is set to 100
I'm new to all this, and as it stands its no longer working. The serial monitor in the arduino software is reading the packet data as it should, so its just a case of max interpreting this data into something usable that I'm struggling with.
Thank you very much for any help you can provide.
We need more info, specifically, what number is used to start or stop the packet (if there is one). That way you'll know when to bang out the items in zl.group. Also, what values are actually sent? I thought all values over serial were 0-255, so if there are bigger numbers, they're combined.
I updated the patch a bit and put some hints in there. First things first, though, we need to know what's actually being sent. Check the Max window for details in the new patch.
Hey Seejay,
Thank you so much for your help so far. I've been playing around a bit more, and based on what you helped me with in this thread, as well as the other thread about filtering out all the 32's coming through, I've built a somewhat modified patch that I think is getting closer, but maybe I've just complicated it and actually getting worse?
I looked more into the arduino side of it, because I'm actually using a library that does all the parsing built specifically for the brain reading device. It sends out the data as ascII... which then must get converted to come over serial. In the patch I updated, I use the ItoA object, and print this to the max window, and the figures that come up are exactly a match to what gets printed in the arduino serial monitor. You can see an image attached. Basically all of these 11 groups of numbers are variable. So for instance the first 200 that you see, this is the signal strength coming from the device, which varies from 200 to 0. The next two zeros that you see are variable between 0-100 and concentration and meditation levels of the person wearing the device - they are both 0 right now because the device is sitting on the table. The next 8 are 6-7 digit variables, that represent the various alpha, theta etc brainwave readings. So I'm looking for a way to break these 11 variable values, and patch them into their own number boxes, so I can use them individually to control parameters. For instance the first reading, signal strength, I just want going into a number box so I can see that its working, whereas the second reading, concentration I want to control a volume slider... etc
I think I'm getting closer to achieving this maybe, given that since you helped me in the other post, I've figured out that each space will return a 32, and each comma will return a 44, I'm just trying to work out how I tell the unpack object to combine all of the numbers together between each 44, i.e between each comma, seeing how the data is separated by commas?
Looks like there are spaces between each number as well (no problem, just need to know that). Can you post a patch in which you add a comment box that has a complete "set" of the data (all numbers, plus commas, plus spaces before and after until the next set)? Actually, maybe two complete sets, to see the "gap". Just paste it from the Max window. This would really help work it out from here, because I obviously don't have the raw serial data to work with ;) (though I wish I did...love EEG stuff...)
If there's any way to insert a different special character on the Arduino end of things, at the end of each set of data, that would help a lot. Maybe a colon. That would be the magic character which starts the parsing/unpacking process.
I updated the arduino sketch, it prints a colon ; at the end of each packet now.
I also modified the max patch, and included two lines of the serial data in separate message boxes, I thought maybe you could play around with sending them through the patch that way, since you don't have the actual serial data coming in!
Thank you again for your help!
Took a look, didn't get too far because again, it's still not quite the same as having the actual serial "stream". Did you modify the serial data that you put in the message boxes (i.e. remove spaces or anything)?
A couple thoughts: ditch the commas, the spaces between the values will be fine as a separator (ASCII 32). Also don't use a semicolon (;) for the line end, both commas and semicolons are special characters in Max and working with them can be squirrely. Use a full colon (:) instead.
If the values truly do come out of [serial] just as you've shown here, you're nearly there. You shouldn't need [itoa] then (BTW what was it for?) [zl group] to collect the values, then [sel :] (or the ASCII equivalent if you're watching for numbers rather than characters) will bang out the group. then you're off to [unpack] and good to go.
if your datastream starts with the number 200 and sends out packets every second , it is easy :
data ->[iter] -> [zl group 11] -> [route 200 ] and you've got your 10 remaining items list, see patch below:
Hi, sorry I've been busy with work and other projects, and haven't had time to sit down with this in a while.
Thank you both for your help again. Seejay, the message boxes where not modified, it is exactly as the serial data comes into max. Unfortunatly most of the arduino code is comming from an external library, so removing the spaces and commas requires modifying the library, which is again beyond my ability in programming.
I tried instead adding a 27 and a colon at the beginning of each packet. Like I said the 200 is a variable, and will begin to change once the headset is being worn, so having 200 as the route selector wont work once its being used.
I modified my patch according to Karrrlo's help. It seems to be almost there, I've been tweeking and trying different approaches for a few days now, but for some reason the unpack just doesn't work if I have the serial data comming in. If I have the exact some data put through the patch by the message boxes, it unpacks, but if I run it through the serial instead it doesn't want to unpack?
I've attached some screen shots and my patch below so you can get more of an idea of what I'm talking about.
Thank you both again very much for your help!
a bit tricky to tell from the Max window. The third image has some clues. try the [serial] with and without [iter] and post a screenshot similar to the third one here maybe? [iter] may or may not be needed...
Looks like the values are going OK through [itoa], then becoming lists of single digits. I think you'd need to collect everything that's between each comma into one [zl group] and when the comma hits, bang out the list, then use [regexp] to remove spaces, then a [fromsymbol] to ensure it becomes a recognized number. Each of these will go to a separate number box, and use the "end-of-packet" value to restart which number box you send the incoming current value to.
Not sure though...more screenshots of experimentation could help...?
I tried placing the print object at different parts of the patch and took some screen shots. I placed the [sel 32] right after the serial data comes in to try to filter out the spaces.
The first two screenshots are before the [iter] and after. This object doesn't seem to make much of a difference. Adding the [sel 32] at the start has added the error checksum message, this used to be an occasional "packet to long" error message, know it is quite regular "checksum error".
Screenshot 3 is further down the max window with the print object in the same location.
Screenshot 4 is with print placed after the [zl group] object.
I also placed the print object after the [route 27], and no data seems to be output from this object, i.e the max window did not print anything.
I wasn't to sure where about you where meaning to place the [regexp] and [from symbol] withing the signal flow?
Thank you again so much for your help!
Dear Egor.
I also am interested in the EEG technology and, after searching here and there and also landing to this site,I managed to get data into MAX from Arduino and splitting individual values into integer boxes to be used inside Max for your synth.
Hope this will help You even if more than one year has passed.
Please let me know if You find some issues.
Regards
Houseman
Hey Housman,
Thanks very much for your help... and yes it has been a year but I am still working on the project.
What eeg device are you using? I've switched to the open eeg project.... Only two channels, but more accurate then the toy I was using.
Your patch is extremely helpful!
Thanks again!
Hi Egor, happy to know the patch is useful for You.
I have opened a tool page on cycling74 since I want to investigate a little bit more and embed in Max also the blink feature and the RAW data.
I want also to implement in Max a feature that controls the data flow and discard wrong packets.
I will keep on working and update the news:
https://cycling74.com/tools/eeg-tgam-thinkgear-brain-wave-unpacking-data-from-arduino-into-max-msp/
I am currently using the devices Mattel Mindflex, NeuroSky headset, they all use the same ASIC TGAM chip inside.
But as You told I am afraid they lack a little bit about reliability even if I know that it can be not as a switch ON/OFF feature.
Sometimes Meditation and Attention seems to be a little bit blurring and even if I concentrate the level goes down or backwards the level increases even doing nothing..
How do you feel with Open EEG? What do you mean with more accurate? More stable signal? more features? Please tell me
Best regards and see You