Evangelos Angelakis

I am not very experienced in Max, so any help is appreciated.

For a part of my PhD project I would like to do the following: I need to record data streams from the Kinect Azure sensor and store them in a file for offline synchronization with data streams from other sensors (recorded through their own software) and subsequent -offline- statistical analysis.

I am using the object dp.kinect3 to get the data streams from kinect to Max and I will need to record 4 numbers (x,y,z and orientation) from each of 6-7 skeleton joints, and will probably use the default sample rate for Kinect (30Hz).

Any thoughts on how to do this, or which objects you recommend as more suitable (i.e coll, mubu)?

Hi everyone!
I am not very experienced in Max, so any help is appreciated.

For a part of my PhD project I would like to do the following: I need to record data streams from the Kinect Azure sensor and store them in a file for offline synchronization with data streams from other sensors (recorded through their own software) and subsequent -offline- statistical analysis.

Hi everyone!
I am not very experienced in Max, so any help is appreciated.

For a part of my PhD project I would like to do the following: ... 

seeking-advise-on-data-streams-recordingstorage

that other senors and sofware will probably dictate capturing form,

including timestamp format, in order to synchronise all sensor values

which max object and form is most suitable would also depend on amount

of captured data, or better say capturing length.

if all kinect sensors share same line, 30Hz make 120000 lines per hour.

unless you capture only when sensor values change.

For really long captures, I would rather use shell object

and write values directly into text file.

" >> ~/Desktop/021222_113325KinectLog.txt

which will insert line by line into text file on Desktop 

that other senors and sofware will probably dictate capturing form,
including timestamp format, in order to synchronise all sensor values
for analysis.
which max object and form is most suitable would also depend on amount
of captured data, or better say capturing length.
if all kinect sensors share same line, 30Hz make 120000 lines per hour.

For really long captures, I would rather use shell object
and write values directly into text file.
for example:
echo "timestamp values" >> ~/Desktop/021222_113325KinectLog.txt

which will insert line by line into text file on Desktop (space or tab separated)


'Source Audio', the rest of the sensors will end up as .wav signal, which I will later transcribe to whatever format I get from Kinect/Max, I seem to be flexible there.

Kinect will be capturing continuously (not just when values change) and each experiment session should be about 30-40 minutes.

Thank you all so much for your replies!
'Source Audio', the rest of the sensors will end up as .wav signal, which I will later transcribe to whatever format I get from Kinect/Max, I seem to be flexible there.

Kinect will be capturing continuously (not just when values change) and each experiment session should be about 30-40 minutes.
Any suggestions given these details?

Please post details about wished capture form, like if you need a time stamp on every line, if you want to put all sensors into same line or not,

if you want to separately capture each skeleton joints etc etc.

without that infos ... it is difficult to suggest something.

You can of course also capture values into buffer~ and save as wave files

Please post details about wished capture form, like if you need a time stamp on every line, if you want to put all sensors into same line or not,
if you want to separately capture each skeleton joints etc etc.

I am sending the details which you suggested:

I hope I am including everything needed for a more informed suggestion.

I am sending the details which you suggested: 

Txt will be then fed to Matlab for processing

I can collect everything in one text line per sample. 

There will be one Kinect Azure sensor only.

Capturing will be continuous (not just when values change)

All joints captured together in one text line

 4 numbers per skeleton joint (x,y,z,orientation)

Timestamp needed only in first and last line 

I hope I am including everything needed for a more informed suggestion.
Really grateful for the help!




I forgot to ask about max or win, because that makes a difference

for writting directly to text file, which is very simple on Mac.

max date object can not output sub second time format.

If that is needed, one would have to run per second triggered ms counter for that and form timestamp string - Date_Time (HH:MM:SS:sub)

if no subdivsions are needed, it would make things easier.

other option is to use timestamp when creating capture file.

In any case decision to collect data first into max text object and save a file

when finished is one option, to use shell object and feed input directly into text file a 2nd option.

The reason I thought about shell object in first place is that it is more secure, every entry will get written into text file,but in case of max text object, a crash would wipe everything collected so far.

here is first example capturing int max text object

I will post another one with shell object laer, if you use Mac.

I forgot to ask about max or win, because that makes a difference
for writting directly to text file, which is very simple on Mac.

as first about timestamp:
max date object can not output sub second time format.
If that is needed, one would have to run per second triggered ms counter for that and form timestamp string - Date_Time (HH:MM:SS:sub)
if no subdivsions are needed, it would make things easier.
other option is to use timestamp when creating capture file.
In any case decision to collect data first into max text object and save a file
when finished is one option, to use shell object and feed input directly into text file a 2nd option.

The reason I thought about shell object in first place is that it is more secure, every entry will get written into text file,but in case of max text object, a crash would wipe everything collected so far.
------
now to the concrete examples :
here is first example capturing int max text object

I will post another one with shell object laer,  if you use Mac.






Wow! Thanks!
I will try that later tonight.

I run on Windows, so no need for Mac...
Really grateful!


Just finished with a bit of testing and it works! Thanks!

Can't figure out why the timestamps' differences (end-start) in seconds do not seem to correspond to the number of data lines divided by sampling rate (30), but I'll figure it out... 

I'll also try to implement the sub-second time stamp that you mentioned.

Btw, I switched the [metro 33,333] to [metro 30 hz] . Should be more precise, right?

Also, the [dp.kinect3] object help patches use the [qmetro], do you think that this could make the patch drop frames and affect the number of text lines? I need these to be fixed at 30 lines per second, in order to make subsequent synchronization with other data streams possible.

I have yet to test with the actual number of joints and orientation, but I am confident!

Just finished with a bit of testing and it works! Thanks!
Can't figure out why the timestamps' differences (end-start) in seconds do not seem to correspond to the number of data lines divided by sampling rate (30), but I'll figure it out... 

I'll also try to implement the sub-second time stamp that you mentioned.


Also, the [dp.kinect3] object help patches use the [qmetro], do you think that this could make the patch drop frames and affect the number of text lines? I need these to be fixed at 30 lines per second, in order to make subsequent synchronization with other data streams possible.

I have yet to test with the actual number of joints and orientation, but I am confident!

date outputs seconds only, so in theory you can be offset by max 29 messages from date output.

I don't know anything about that kinect object, but I would remove qmetro.

The idea in my patch is to first collect ALL values into a list,

and then output collected list using external metro - that 33.333 or 30 Hz as you prefer.

That also means that precise timestamp could differ from "real time" by 1 metro clock, or 33.333333333 ms.

I don't really know how to get that sub second timestamp using date object when triggered only once, because my idea was to poll date output all the time, not only Start and End.

Date also has ticks output, but that will give you time in ticks elapsed since system start, not absolute real time.

date outputs seconds only, so in theory you can be offset by max 29 messages from date output.
I don't know anything about that kinect object, but I would remove qmetro.
The idea in my patch is to first collect ALL values into a list,
and then output collected list using external metro - that 33.333 or 30 Hz as you prefer.
That also means that precise timestamp could differ from "real time" by 1 metro clock, or 33.333333333 ms.

I don't really know how to get that sub second timestamp using date object when triggered only once, because my idea was to poll date output all the time, not only Start and End.
Date also has ticks output, but that will give you time in ticks elapsed since system start, not absolute real time.



in fact on window cmd TIME can output seconds fraction

image.png

and also echo text to file works just as well as on Mac to write logs directly to a text file.

One can also use java, which I dislike ...

for example now mxj could be reduced to needed timestamp

Here is max patch that does all using shell object

path for shell must have proper format, that's why regexp etc.

I even think in last shell object in that respect, but this is 100% secure path form. Name of text file can be also combined with time stamp etc, but that is your decision, how to name capture files.

Echo command creats file itself, no need to create it manually before capturing.

https://github.com/jeremybernstein/shell/releases

One can also use java, which I dislike ...
for example now mxj   could be reduced to needed timestamp


I even think  in last shell object in that respect, but this is 100% secure path form.  Name of text file can be also combined with time stamp etc, but that is your decision, how to name capture files.
Echo command creats file itself, no need to create it manually before capturing.
and path to shell object

https://github.com/jeremybernstein/shell/releases



That's great! Both are elegant solutions.

Sorry to say that still some assistance is needed...

In the first patch Capture Start timestamp in the file is not the correct. It seems to always be 4-5 sec after the Capture End timestamp of the previous capture session, even if these are hours apart from each other. Capture End timestamp seems to be correct...

I tried to run the second patch (after installing shell) but I cannot get it to create a file. If I understand correctly it is supposed to do this automatically, but I must be doing something wrong... I tried various tricks that I could think of, but haven't cracked it yet...

That's great! Both are elegant solutions.
Sorry to say that still some assistance is needed...

In the first patch Capture Start timestamp in the file is not the correct. It seems to always be 4-5 sec after the Capture End timestamp of the previous capture session, even if these are hours apart from each other. Capture End timestamp seems to be correct...

wrong capture start time is definitelly due to timing.

Date object should be triggered at loadbang

it needs few times to do so to update correctly.

here is fixed version, see reversed order

Now to shell - can you post your file path,

And if there are any errors in the console ?

you can insert print at shell output to check for any errors.

Are there any restrictions to read / write on your PC ?

P.S. I have to download that latest shell version and test it

I actually use older one from 2017, maybe there are some differences.

Even if you keep using text object for capture,

shell can at least help you get timestamp with sub - seconds

Not that it can't be done with js, but one has to install java runtime to use js in Max , and I do not do so on my PC

and in general avoid java because that forces users to also

wrong capture start time is definitelly due to timing.
Date object should be triggered at loadbang
it needs few times to do so to update correctly.

here is fixed version, see reversed order
time, date


-----
Now to shell - can you post your file path,
how do you enter file path ?
And if there are any errors in the console ?
you can insert print at shell output to check for any errors.
Are there any restrictions to read / write on your PC ?

P.S.  I have to download that latest shell version and test it
I actually use older one from 2017, maybe there are some differences.

Even if you keep using text object for capture,
shell can at least help you get timestamp with sub - seconds
Not that it can't be done with js, but one has to install java runtime to use js in Max , and I do not do so on my PC
and in general avoid java because that forces users to also
install it.




Teams meeting recordings will be stored in OneDrive or SharePoint, depending on the type of meeting. Note that a recording will expire and be automatically deleted after a set period of time. The amount of time it's available is set by your admin, but you can change or remove the expiration date of any given recording

Teams meeting recordings will be stored in OneDrive or SharePoint, depending on the type of meeting. Note that a recording will expire and be automatically deleted after a set period of time. The amount of time it's available is set by your admin, but you can change or remove the expiration date of any given recording. 

Shell version did not want to write files in root or desktop, but gave no error messages (although I tested to see and directories with no permission, or ones that did not exist gave error messages...) Anyway, a new folder on the root directory was a place where it is writing correctly.

Shell version was saving capturing time, but Only the fractions of the seconds. I added two more list items ($2 $3 and $3 $4 respectively) in capture End and Capture Start messages.

Now both options work!
Text fixed version you sent is working

Shell version did not want to write files in root or desktop, but gave no error messages (although I tested to see and directories with  no permission, or ones that did not exist gave error messages...) Anyway, a new folder on the root directory was a place where it is writing correctly.

Glad it works now, strangely , I got one list output from shell for timestamp

Max text object or direct capture to file ?

I removed any kind of protection from the windows system, read-write, firewall etc and have full root rights even on system itself.

That's why I overlooked that write problem.

Glad it works now, strangely , I got one list output from shell for timestamp
maybe because of older version I used.
Will look into it.

What do you find better at the end ?
Max text object or direct capture to file ?

P.S. 
I removed any kind of protection from the windows system, read-write, firewall etc and have full root rights even on system itself.
That's why I overlooked that write problem.



Thank you for the time, knowhow and work you put into it!

I will be conducting various tests to see which combination will work best for the project, as there are also other parameters to consider (Sensor sample rate, Nyquist, GPU load, synchronization with other sensors etc).

I also managed to get a timestamp from the kinect sensor itself, although it's in unix time, so it will have to be translated...

You have been most helpful and have my sincere thanks!

Yes, I too am glad they both work!
Thank you for the time, knowhow and work you put into it!

You have been most helpful and have my sincere thanks!
E. 

I am returning to this chain as I have new data and would again be grateful for your help @Source Audio

I have decided to go with the Shell Object option for my experiments and I have implemented that bit into patches that I am using.

However, I have run into a glitch and could use some assistance.

When I review the test data from my sensors capture I realized that the date lines were too few.

I seems that the [shell] patch does not record in 30Hz for some reason...

I went back to your original [shell] patch and made only the few corrections we had discussed and then added a [random] number generator to feed the data recorder at 30Hz. No other input or change was done.

I saw that the number of lines of text recorded divided by the recording duration gave me a data recording frequency of about 10 Hz, regardless of how many numbers were fed for each data line (I tried examples from 1 to 80 numbers per line of data).

Could you please take a look at the result?

Any suggestions on how to get 30 lines/second?

Hi!
I am returning to this chain as I have new data and would again be grateful for your help @Source Audio
I have decided to go with the Shell Object option for my experiments and I have implemented that bit into patches that I am using.

However, I have run into a glitch and could use some assistance.
When I review the test data from my sensors capture I realized that the date lines were too few.
I seems that the [shell] patch does not record in 30Hz for some reason...
I went back to your original [shell] patch and made only the few corrections we had discussed and then added a [random] number generator to feed the data recorder at 30Hz. No other input or change was done.

I am attaching a test patch.
Could you please take a look at the result?
Any suggestions on how to get 30 lines/second?

You have some problem with that patch - it can't be loaded.

It could be that there is some problem with fast updating - reading to disk, I don't know.

In any case, you can allways use max based txt file....

You have some problem with that patch - it can't be loaded.
It could be that there is some problem with fast updating - reading to disk, I don't know.
In any case, you can allways use max based txt file....


I can't download the patch from my previous post either... sending again just in case you see something wrong with it, because I really liked this idea of yours much better than the text native object.

You are right - there are missing lines in text file.

I run a test with indexed 10000 lines several times.

There were allways missing lines, sometimes few after a 1000 or so,

but all tests had +- 50 lines out of 10000 missing.

I did that on Mac - have no time to digg windows system for that.

I can't say what is causing dropouts, I've run tests without touching anything in max or computer, to avoid any interrupts due to gui

When I get more time I will look more deep into it,

You are right - there are missing lines in text file.
I run a test with indexed 10000 lines several times.
There were allways missing lines, sometimes few after a 1000 or so,
but all tests had +- 50 lines out of 10000 missing.
I did that on Mac - have no time to digg windows system for that.

I can't say what is causing dropouts, I've run tests without touching anything in max or computer, to avoid any interrupts due to gui
or mouse actions.

When I get more time I will look more deep into it,
maybe there are alternatives. 




My tests gave about 9 lines of text per second, so i.e. in a 100-second test I got about 900 lines instead of 3000, which the 30Hz freq should output. 

So missing lines were about 2100 in 3000!

My tests were also without touching anything and using a DELL G5 (Intel Core i7 10750H, 16GB with SSD and a RTX 2070). This laptop is for dedicated use for my PhD experiments and doesn't even have Office installed (in order to keep the SSD free and the system unburdened by applications), so I doubt that there were serious interruptions.

Great! Thanks!

I had many more missing lines!
My tests gave about 9 lines of text per second, so i.e. in a 100-second test I got about 900 lines instead of 3000, which the 30Hz freq should output. 
So missing lines were about 2100 in 3000!

I am sure it has to do with disk access, if shell can not execute

adding a line to file and rewritting it, it simply continues

That is way too many missing lines.
I am sure it has to do with disk access, if shell can not execute
adding a line to file and rewritting it, it simply continues
without warnning



if you pack data into jitter matrixes you can read and write them blockwise (frame by frame).

something similar happens when you use audio signals to store stuff.

I just made few tests on windows machine.

Shell and any kind of script that writes into text file is very slow.

no matter if echo, type or then powershell.

It is definitely not possible, at least with the shell object.

it is simple to test , you run a couner that writes lines,

and insert a counter at shell done bang outlet.

Shell and any kind of script that writes into text file is very slow.
no matter if echo, type or then powershell.

it is simple to test , you run a couner that writes lines,
and insert a counter at shell done bang outlet.

Ok! Thank you for putting time and thought into it. Really appreciated!

Also thanks @Roman Thilenius, I will have to look into jitter matrixes to see how this can be implemented (I am not very experienced Max user yet)

@Source Audio, that leaves us with the [text] solution you recommended, at least for now.

I will run tests to see if I get dropped data in one single transfer from [text] to the .txt file.

I will need to record continuous 40-45 minutes (per session) of data from 10 (it turns out) Kinect skeleton joints with orientation data. This means 45mins*60sec*30fps*80(floats/per line)=6.480.000 floats

Ok! Thank you for putting time and thought into it. Really appreciated!

Also thanks @Roman Thilenius, I will have to look into jitter matrixes to see how this can be implemented (I am not very experienced Max user yet)

@Source Audio, that leaves us with the [text] solution you recommended, at least for now.
I will run tests to see if I get dropped data in one single transfer from [text] to the .txt file.

I will need to record continuous 40-45 minutes (per session) of data from 10 (it turns out) Kinect skeleton joints with orientation data. This means 45mins*60sec*30fps*80(floats/per line)=6.480.000 floats

what range will floats have, and which precision ?

I would run counter that appends full lines and simply test.

On my mac appending 108000 lines with 80 floats into text took 3 seconds.

I would run counter that appends full lines and simply test.
You can even uzi that lines.
On my mac appending 108000 lines with 80 floats into text took 3 seconds.
file size 94.9 MB

Float range will be around from -1.5 to 3.0 (in meters) with a maximum of 4 floating points precision 

Tested using a different patch I made, but used same number of lines (10800). I set value range from -1.5 to 3.0 and a 4 floating points precision, plus an extra index bang number for every line of 80 floats. File size 6.214 KB, which I think makes sense (10800*80*7=6.048.000 Bytes plus about another 53.000 Bytes for the extra 10800 index numbers, and we're getting there)

I am trying to explore other options, like the MUBU external... we'll see

Text saving was successful for me too.
Tested using a different patch I made, but used same number of lines (10800). I set value range  from -1.5 to 3.0 and a 4 floating points precision, plus an extra index bang number for every line of 80 floats. File size 6.214 KB, which  I think makes sense (10800*80*7=6.048.000 Bytes plus about another 53.000 Bytes for the extra 10800 index numbers, and we're getting there)

I am trying to explore other options, like the MUBU external... we'll see

Thanks for the input!
E.

Seeking advise on Data Streams Recording/Storage