unpacking a packet of parsed data.
Hi, I need some help unpacking a packet of parsed data coming from an arduino.
I’m trying to collect data from an eeg device and use this data to control parameters of a synth built in max.
The data comes in through the serial device, and should be in a packet of eleven values. For instance
200, 0, 0, 103589, 698790,247658,278765,172456,567898,678543,456763
The values are updated and sent as a new packet once per second.
I need a way of unpacking, and separating the data, so that each of the eleven values can be fed into a separate number box.
I don’t really know enough about what I’m doing. I worked out from reading the max tutorials that the data comming in is integer and it needs to be a different format to work in max?
If attached a basic patch I’ve tried to construct based on the max tutorial for unpacking serial data. I had it working a little bit when I spent a while playing with different setups and removing objects etc, but it was only collecting four values, not the eleven values that it should be reading.
for instance, it will generate numbers into the boxes
44, 50, 44, 55, 49, 48, 44, 50 … etc
it also only seems to be updating once every three seconds, even though the metro is set to 100
I’m new to all this, and as it stands its no longer working. The serial monitor in the arduino software is reading the packet data as it should, so its just a case of max interpreting this data into something usable that I’m struggling with.
Thank you very much for any help you can provide.
We need more info, specifically, what number is used to start or stop the packet (if there is one). That way you’ll know when to bang out the items in zl.group. Also, what values are actually sent? I thought all values over serial were 0-255, so if there are bigger numbers, they’re combined.
I updated the patch a bit and put some hints in there. First things first, though, we need to know what’s actually being sent. Check the Max window for details in the new patch.
----------begin_max5_patcher---------- 2052.3oc6cstaihCE92sOEV4OqVorYw1XtLZ0JsO.6SvLiF4PbRYJAX4xz1Y zLO6qu.IgzPflBTSCsRNoNT6iOmO+cN9Di4G2dyrkQOxRmA9.3ifat4G2dyM xpDUbSweeyrszG8BnoxKa1VVZJcCa1b0msbiWTPTBR0DFKbrrmezKvEFfOW8 xa4UGlu0OLfkI6YTQk9aBiRXdA9d2Kp1nn5XZl2c9ga9B+yxTsOx0bgAxz00 j2r172OGfbLWXwqQVkS0tJJOqrufkMZBKkElQy7iBOrytye0JVkZxXOlUYnY L+fhCGT9qj5wnke8OLgkpw0QgYgzsL4G8OI9zfxOQIUYOEyTs7rY6ZoMIzU9 bw6P4PzPqodriqK0+6x5fvEDWBzw9DivCTcX6ENDCCCKgZpxfYuoj5c+ljn7 P43wPT4Ou8VQw7KCJITgx5+yc+LJPY1N60UFPgZxzdAxU7iN.xvt5JHiHUVt t8D7xKZ6VgfeL75on7eKH.7.MLCjcmeJHKB7oYY4grOMCv+vDv2nA4rTv5nD v5D1+kyB874+c1czL.MgAnwwIQwbEX1NvafeHyiKsYGBhpfCgE1nhBi5PfvS i.g0i.McPREo4BBmgiCAwbznAdN.iqE7UZ1VlubY.KNxuvByahWJpjahXIhw +ya2T+UrCE91CfWmrY4KBk6zLJ+TvVkb1Vfr5pEihszjM9R0.47fbSG7Bat4 .h3xNrunRCYOv0BOCpm5QCX.C.hP3bSF.AM0rV.Nqkdz5ESOV.NgXS0K3EPC EYIx3MmZz9UQM1QzfGIqmAXTTakwIlX4XgE+a172frObD2gjlQgAOA7dxKf8 KvC9b5ysz64Dg4q7ilMvrcJGFlt1GS2cF.06N5NKMktiXHmmSPjA1qtDaB3d yW4udMKgIctSCKwrfO76uQ3TKrbgGWo3Tht5V10Yma4BSz.BV+KPJiwwmLvJ ZFE3GJe++RejysFtJ5gAFphH7UEKVuB26Qw5juRgqlZJbEoLKPE6ZOfTyh1r If8phMDd9XC8ti4cOaU6LbmBjpboThOKK6rHHioIb6cFK4KrP5QVyCgHnZiL Th9+bCFRj49QQKi2aYTBetQEEGY9AEWZrd0rngMGrr1KBHbA4PQYaKReB1ZW 5Szf0HfG+qQvQl2KS4hDfcNdINgC74NzDZjuH7m0mrHmA7X5nvPnWhmq9F8. uLGJi8kQVCTQFL9.COrbj3BLVwsHC06sOqrPq9kVQz2i.Fk7sKYIcBf3tiR5 e8Fg1ibrkccQrWDimgbxR7qzm1ROx6Kg0hxPcZ3Iuf.hiR1RypIzF3kjNLQP OyAbLS3l8vSOZHWaxqphXd2Qx4YLwbMK++NnRX+bit+17spEPEFEx96Y6uZY lNkW8BiW9bD9nJNOKJ74nhNZxC21mxxiqTE8wZFNsJduc5zivfNpujKjnDY9 9fAGhzPFbj8DCt1yfqPNWML3HmIF7IFb8iAmfc0QFb2IFbcmAu.4b0vfiMlX vmXv0OF7huMSciAmLwfq6L3Ea.nqmXvslXvmXv0PFbDVGYvwSL3ZOCtB4b8v faNwfOwfqeL3X0TOciAGNwfq6L3EHmqGFbzDC9DCt9wfibrzw8hxTdv0dF7B jy0CC9TdvmXvafA+6AKDit3AdyJCUKGtX+lZZ0.INZfHwunTgO+8HQtF4uWs 0j0M+8lS960d+8Jjy0ydOEO4uexe+nXpIjbcskTfSISYZp43Xpo6UlSS3zLy oYliCml1WWyLmRe0zLy1jPheMzIuhTYeb5pGYj3R9BHR4CZ9GeMk8pgFr3Rz PrhyDVoArRdXLWD.Fm52ANAnk4Ju33qgX1DFZf.Quh6z4N3kINqdzAmsFxYY NwYUhUXeeE06zfBdDYv4UJ6PbA1.KB3intifMIkkc14CSTn3nRt54Tsk07ck 1npimfHOg48z2EXUieUclpchNBdzxkbEkt3pcTz50OWxjg1pJN5x6JvygcK7 nPp2UV4T6tmNlqNygvWFMICDGEDvwJxifO0oXDupjrA9L3yVk5JEgEA2vwa1 6sCdOntd7M2dX+6xCcOS68nx2tybOqW0Qt2Kl4ZfOu83JfjHwwy8.Gojx1Rj 6eEKyN21dQAJcIahkp4K68zAtWcOfLjm3digmMFEK+iXoU2vB55yCid0WScO g.TA7rB3ZY7Fkb.0Na01tAFngZezc4YFXb9jAP1jxmeIG8bfRZwJCLbOpJMJ OwqbjWtyCA6krUrzL+vclmOt+QcD.tOW6OyXcB9ycEt6ab+TQbBmYVPak51H zvJWyauLKdT9znPi0NoFOFkZXaDZrlApEGN0sSpasptVNytSpaEpVyfGNiPY 1lKONMIyFchPu0ekLoDEtTHk2+g3dA.IFXtMLvPN8w.yzVca4nVarETk2Frc uMLsaZXZogfNqlDZSMTnIMIzHMTnMaRnMzPgF1XbX18BoTwdRxPNo0xQlaGD t+nnvMML0QGFnlDZcbZ.rIgFpgBciyBzs4tVsPnIZVrlliv31NJlrZX1c0Lo F01v50oEPgZSb8HacSUaOFU0sg9.QzMUMYLppaCqGR2n8P3wnptMYfAoagef fiQUcabKB0N2hccdmGfflfiQW4vVESstktDn0XTU2Fmh5VPSvw3RAfswmHT2 RmFrMjd5FQMrM9DczMgtM9DM0QgF1BgFNx99Ag86pWTey8z33uwRRK5Yo7Na K8qp10Z9spa2O0eJ62YIru4Wd8l2JZsed6+SxS.Fg -----------end_max5_patcher-----------
Thank you so much for your help so far. I’ve been playing around a bit more, and based on what you helped me with in this thread, as well as the other thread about filtering out all the 32′s coming through, I’ve built a somewhat modified patch that I think is getting closer, but maybe I’ve just complicated it and actually getting worse?
I looked more into the arduino side of it, because I’m actually using a library that does all the parsing built specifically for the brain reading device. It sends out the data as ascII… which then must get converted to come over serial. In the patch I updated, I use the ItoA object, and print this to the max window, and the figures that come up are exactly a match to what gets printed in the arduino serial monitor. You can see an image attached. Basically all of these 11 groups of numbers are variable. So for instance the first 200 that you see, this is the signal strength coming from the device, which varies from 200 to 0. The next two zeros that you see are variable between 0-100 and concentration and meditation levels of the person wearing the device – they are both 0 right now because the device is sitting on the table. The next 8 are 6-7 digit variables, that represent the various alpha, theta etc brainwave readings. So I’m looking for a way to break these 11 variable values, and patch them into their own number boxes, so I can use them individually to control parameters. For instance the first reading, signal strength, I just want going into a number box so I can see that its working, whereas the second reading, concentration I want to control a volume slider… etc
I think I’m getting closer to achieving this maybe, given that since you helped me in the other post, I’ve figured out that each space will return a 32, and each comma will return a 44, I’m just trying to work out how I tell the unpack object to combine all of the numbers together between each 44, i.e between each comma, seeing how the data is separated by commas?
Looks like there are spaces between each number as well (no problem, just need to know that). Can you post a patch in which you add a comment box that has a complete "set" of the data (all numbers, plus commas, plus spaces before and after until the next set)? Actually, maybe two complete sets, to see the "gap". Just paste it from the Max window. This would really help work it out from here, because I obviously don’t have the raw serial data to work with ;) (though I wish I did…love EEG stuff…)
If there’s any way to insert a different special character on the Arduino end of things, at the end of each set of data, that would help a lot. Maybe a colon. That would be the magic character which starts the parsing/unpacking process.
I updated the arduino sketch, it prints a colon ; at the end of each packet now.
I also modified the max patch, and included two lines of the serial data in separate message boxes, I thought maybe you could play around with sending them through the patch that way, since you don’t have the actual serial data coming in!
Thank you again for your help!
Took a look, didn’t get too far because again, it’s still not quite the same as having the actual serial "stream". Did you modify the serial data that you put in the message boxes (i.e. remove spaces or anything)?
A couple thoughts: ditch the commas, the spaces between the values will be fine as a separator (ASCII 32). Also don’t use a semicolon (;) for the line end, both commas and semicolons are special characters in Max and working with them can be squirrely. Use a full colon (:) instead.
If the values truly do come out of [serial] just as you’ve shown here, you’re nearly there. You shouldn’t need [itoa] then (BTW what was it for?) [zl group] to collect the values, then [sel :] (or the ASCII equivalent if you’re watching for numbers rather than characters) will bang out the group. then you’re off to [unpack] and good to go.
if your datastream starts with the number 200 and sends out packets every second , it is easy :
data ->[iter] -> [zl group 11] -> [route 200 ] and you’ve got your 10 remaining items list, see patch below:
----------begin_max5_patcher---------- 801.3oc4XtsbZCCDF9Z3oPiu1MiNK4dWeN5zIiATINEjY7gTZxj28JsxjRSH VdRAOI0CLHzAV+6O1csz9v7YIKJ2apSPeF8UzrYOLe1LXH+.y55OKYa99kax qgkkr0TWmu1jjFlqwruAFmhwonvaBlIzYoHYlVk45S4JoP6ZUZ2WbyqnbgLE IjJclabWifyRQtAUR1AKuKuY4ME10WWYV1DTHCStJXeeCmE5ouBi9V2OpXEH lxE29Il7fk9doswlu0.S8kph7MGlorsYioo4W6LgKPRxSVx1tsv5lDtqoGYo 5h6gUSntq9SqMXIXwD+fONet+izARUq4mNM+Bnd+Fz5px1cHBoGrHo.Gj.Nj Y9Oo3SCEp9s.kzyCXnmOvTzXphSDBGXgpWhHNetIjKsaxqE7gQ3tnttfttXt tPttHtt.tt3smEtsovZVV1ZACl85nkB9WbNzHBblJNMaI5ORgfsaWzqSEKCb jDX3VOi0mWEY.dU6xqbi6bju1XyWrAVA90nRg6OFWX3hb65ygyG8h.Ho.7Mz CAP7oHfXgGbIGBfXSQ.Ed.eW1kH.hNAADM35voCAPjoHfXPRZ9fRRimh.BCI oYCIIc1DjODEfD1PxQqmh7gConYCIEsZJxmPlG1PxPK+OjOm97Zs1c4K+AB+ xWwQIMruaFt2ywweCG0niW+KMmiCBhOebtxYWiiQ3nGNlhCmcSg6iprOXkKX .Umho0DsxWc.FkHH96cAkRcsZIg3KOES4FwcPOFmPb+.khp0oHM2cf53kgI7 3CNU2WsonuCNWLXN3X+Oq9evsle7+ly0ksUKOHlCEXC8mauUl5lBadSQo8nE 4KuxQK5lhUqL1iShspn1mWCXyoCDFpd701JpdXimbDCAO5QSOtDjnrHxgHFU 4niIG9nJGUL4vFU4HiIG5nJGQL4PFU4viIG7nJGVD4jMppgFQMiaVGRD0nFU 0DKkrbzTCa.pg+tRM9B5ezemWzGedI2bQXuO461cmoptylfRb6l71xJeWYJz svF5B6nOoxbWwg0Ca7JIuxsMvF2d.aqBahauVlL2ecdb9uQI8qap -----------end_max5_patcher-----------
Hi, sorry I’ve been busy with work and other projects, and haven’t had time to sit down with this in a while.
Thank you both for your help again. Seejay, the message boxes where not modified, it is exactly as the serial data comes into max. Unfortunatly most of the arduino code is comming from an external library, so removing the spaces and commas requires modifying the library, which is again beyond my ability in programming.
I tried instead adding a 27 and a colon at the beginning of each packet. Like I said the 200 is a variable, and will begin to change once the headset is being worn, so having 200 as the route selector wont work once its being used.
I modified my patch according to Karrrlo’s help. It seems to be almost there, I’ve been tweeking and trying different approaches for a few days now, but for some reason the unpack just doesn’t work if I have the serial data comming in. If I have the exact some data put through the patch by the message boxes, it unpacks, but if I run it through the serial instead it doesn’t want to unpack?
I’ve attached some screen shots and my patch below so you can get more of an idea of what I’m talking about.
Thank you both again very much for your help!
a bit tricky to tell from the Max window. The third image has some clues. try the [serial] with and without [iter] and post a screenshot similar to the third one here maybe? [iter] may or may not be needed…
Looks like the values are going OK through [itoa], then becoming lists of single digits. I think you’d need to collect everything that’s between each comma into one [zl group] and when the comma hits, bang out the list, then use [regexp] to remove spaces, then a [fromsymbol] to ensure it becomes a recognized number. Each of these will go to a separate number box, and use the "end-of-packet" value to restart which number box you send the incoming current value to.
Not sure though…more screenshots of experimentation could help…?
I tried placing the print object at different parts of the patch and took some screen shots. I placed the [sel 32] right after the serial data comes in to try to filter out the spaces.
The first two screenshots are before the [iter] and after. This object doesn’t seem to make much of a difference. Adding the [sel 32] at the start has added the error checksum message, this used to be an occasional "packet to long" error message, know it is quite regular "checksum error".
Screenshot 3 is further down the max window with the print object in the same location.
Screenshot 4 is with print placed after the [zl group] object.
I also placed the print object after the [route 27], and no data seems to be output from this object, i.e the max window did not print anything.
I wasn’t to sure where about you where meaning to place the [regexp] and [from symbol] withing the signal flow?
Thank you again so much for your help!
I also am interested in the EEG technology and, after searching here and there and also landing to this site,I managed to get data into MAX from Arduino and splitting individual values into integer boxes to be used inside Max for your synth.
Hope this will help You even if more than one year has passed.
Please let me know if You find some issues.
Thanks very much for your help… and yes it has been a year but I am still working on the project.
What eeg device are you using? I’ve switched to the open eeg project…. Only two channels, but more accurate then the toy I was using.
Your patch is extremely helpful!
Hi Egor, happy to know the patch is useful for You.
I have opened a tool page on cycling74 since I want to investigate a little bit more and embed in Max also the blink feature and the RAW data.
I want also to implement in Max a feature that controls the data flow and discard wrong packets.
I will keep on working and update the news:
I am currently using the devices Mattel Mindflex, NeuroSky headset, they all use the same ASIC TGAM chip inside.
But as You told I am afraid they lack a little bit about reliability even if I know that it can be not as a switch ON/OFF feature.
Sometimes Meditation and Attention seems to be a little bit blurring and even if I concentrate the level goes down or backwards the level increases even doing nothing..
How do you feel with Open EEG? What do you mean with more accurate? More stable signal? more features? Please tell me
Best regards and see You