Jeff Thompson

Hi all, looking for some suggestions (and apologies for the long description):

I'm trying to manually write audio sample values from a text file back into a wav file - imagine a really long text file full of values from -1/1 . The sample values were extracted from various recordings using a Processing sketch I wrote and saved into text files; those files were then numerically sorted.

The Processing sketch was really fast, using for loops to iterate the array of samples as fast as possible. However, now that I am trying to put the samples back together, I'm running into some problems, mostly in terms of speed.

While realtime, this sounds quite different than when I manually write sample values to a buffer. I'm assuming this is because a wav frame holds more than just a sample value. Audacity's import raw function sounds identical.

2. Write to peek~/buffer~ from jit.textfile

This works great and sounds more like I'm intending. However, since peek~ and buffer~ don't have a "bang when sample is written" feature, I have to put a 1ms delay before triggering the next sample. This means that a 10 second clip takes a few minutes (44,100 x 10sec = approx 7.35 minutes). For the amount of samples I'd like to work with, this won't work since it is so far from being real-time.

So: any suggestions for a quicker way to do this?

I've spent some time looking through this resource on how wav files work:

http://www.sonicspot.com/guide/wavefiles.html

... but it's really confusing. I'm glad to really learn how to manually write wav files if need be, but I suppose I would prefer something a little more familiar using MSP.

I'm trying to manually write audio sample values from a text file back into a wav file - imagine a really long text file full of values from -1/1 .  The sample values were extracted from various recordings using a Processing sketch I wrote and saved into text files; those files were then numerically sorted.

The Processing sketch was really fast, using for loops to iterate the array of samples as fast as possible.  However, now that I am trying to put the samples back together, I'm running into some problems, mostly in terms of speed.

1. "openraw" in sfplay~
While realtime, this sounds quite different than when I manually write sample values to a buffer.  I'm assuming this is because a wav frame holds more than just a sample value.  Audacity's import raw function sounds identical.

2. Write to peek~/buffer~ from jit.textfile
This works great and sounds more like I'm intending.  However, since peek~ and buffer~ don't have a "bang when sample is written" feature, I have to put a 1ms delay before triggering the next sample.  This means that a 10 second clip takes a few minutes (44,100 x 10sec = approx 7.35 minutes).  For the amount of samples I'd like to work with, this won't work since it is so far from being real-time.

So: any suggestions for a quicker way to do this? 

... but it's really confusing.  I'm glad to really learn how to manually write wav files if need be, but I suppose I would prefer something a little more familiar using MSP.

manually-writing-sample-values-to-buffer-faster

What's the format of the file you saved from P5?

There are chances you have an ASCII text file, so using 

 will create a sound file with the ASCII representation of your floats, which is not what you want.

Can't you change your P5 sketch to directly write a binary file of floats?

To avoid your 1ms delay, can't you use an 

1.
What's the format of the file you saved from P5?
There are chances you have an ASCII text file, so using openraw will create a sound file with the ASCII representation of your floats, which is not what you want.
Can't you change your P5 sketch to directly write a binary file of floats?

2.
To avoid your 1ms delay, can't you use an uzi to read/peek~ your file?


It's a text file... I'm assuming ASCII? In any case, it comes into Max as floats. I know the WAV file is written as hex values, but it seems Max would have tidied that up for users. I do wish the sfplay~ reference had more info about what exactly happens when the raw file comes in.

It's my understanding that it basically reads it like a WAV file without a header, meaning it assumes some of the samples are the extra data that comes with the sample values in WAV format. Pure conjecture, and I could be wrong.

Uzi causes a stack overflow at the Jitter objects - I'm guessing messages coming in before the previous one is finished.

It's a text file... I'm assuming ASCII?  In any case, it comes into Max as floats.  I know the WAV file is written as hex values, but it seems Max would have tidied that up for users.  I do wish the sfplay~ reference had more info about what exactly happens when the raw file comes in.

It's my understanding that it basically reads it like a WAV file without a header, meaning it assumes some of the samples are the extra data that comes with the sample values in WAV format.  Pure conjecture, and I could be wrong.

I would like to know more specifically about how sfplay~ opens raw data. I read in this post: 

https://cycling74.com/forums/how-does-sfplay-sonify-non-audio-files

 that the object's messages of 'samptype, offset, srate and srchans' will change how it's rendered, but is there anything else we should know about importing information. And as an ancillary question, is there anything similar for buffer~ ?

I would like to know more specifically about how sfplay~ opens raw data. I read in this post: https://cycling74.com/forums/how-does-sfplay-sonify-non-audio-files that the object's messages of 'samptype, offset, srate and srchans' will change how it's rendered, but is there anything else we should know about importing information. And as an ancillary question, is there anything similar for buffer~ ?


Sound file are binary datas, generally starting with a header (containing informations about bit depth, sample rate, number of channels, and many other things like midi note, markers, regions, loops, etc.).

Those binary datas can be 8 bits integers, 16bits integers, 64 bits floats, etc.

 number of bytes (this offset could represent a header), then will create a sound with its sample length determined by 

, will interleave those samples depending on the 

srcchans, and will use as the sampling rate.

So if you choose int8, each sample will be create with 8 bits.

Jeff, I'm quite conviced you have an ASCII file, so each character representing your floats will be considered as 8bits of datas:

And those 4 numbers (probably followed by a carriage return, i.e. ASCII 13) will be used with openraw, not 0.51.

So openraw will first skip the offset number of bytes (this offset could represent a header), then will create a sound with its sample length determined by samptype, will interleave those samples depending on the srcchans, and will use  as the sampling rate.

48, 46, 53, 49 (see atoi if you're not familiar with ASCII)

And those 4  numbers (probably followed by a carriage return, i.e. ASCII 13) will be used with openraw, not 0.51.

try a [deferlow] object after the uzi to slow things down and see if that avoids the stack overflow (just a guess)

try a [deferlow] object after the uzi to slow things down  and see if that avoids the stack overflow (just a guess)
T


Thanks for the more detailed thoughts on the openraw. If it's helpful for Kenneth, my experiments suggest that sfplay~ assumes the file is stereo and does some unexpected panning with non-audio files. Again, not exactly sure why but can be quite nice.

You may well be correct that the data is read as ASCII as far as spflay~ is concerned. Would the sample values need to be in hex, like they are written in a normal .wav file?

Also, the site I linked to above suggests that "a 16-bit sample can range from -32,768 to +32,767". Would this mean the resulting file should not have values between -1/1?

Thanks, I'll give that a shot today. That's one of those objects that I rarely use and just hadn't thought of.

@Patrick:
Thanks for the more detailed thoughts on the openraw.  If it's helpful for Kenneth, my experiments suggest that sfplay~ assumes the file is stereo and does some unexpected panning with non-audio files.  Again, not exactly sure why but can be quite nice. 

You may well be correct that the data is read as ASCII as far as spflay~ is concerned.  Would the sample values need to be in hex, like they are written in a normal .wav file?

Also, the site I linked to above suggests that "a 16-bit sample can range from -32,768 to +32,767".  Would this mean the resulting file should not have values between -1/1?

@Terry:
Thanks, I'll give that a shot today.  That's one of those objects that I rarely use and just hadn't thought of.


So adding [deferlow] after the "current index" outlet of the [uzi] and before calling the line from [jit.textfile] seems to work great. It's still definitely not real-time, but WAY faster than before.

Here's the basic code, if useful for others. I've also uploaded a text file of samples so you don't have to make your own.

So adding [deferlow] after the "current index" outlet of the [uzi] and before calling the line from [jit.textfile] seems to work great.  It's still definitely not real-time, but WAY faster than before.

Here's the basic code, if useful for others.  I've also uploaded a text file of samples so you don't have to make your own.


You don't need a counter, uzi does this already.

The following patch works without a deferlow. BTW, the first datas in your attachment are below -1. I tried only with the first 44100 lines.

I just had a look in Processing: it doesn't seem to be trivial to write floats as binary datas in a file.

You don't need a counter, uzi does this already.
The following patch works without a deferlow. BTW, the first datas in your attachment are below -1. I tried only with the first 44100 lines.
I just had a look in Processing: it doesn't seem to be  trivial to write floats as binary datas in a file.


Ah, you're about the counter! Nice tidying up!

I've found that jit.textfile seems to work a little better with VERY large files (hundreds of MB or more, which is what I'm dealing with). Might take some more experimentation to see if it really makes any difference.

As for binary data... yeah. I'm poking around FileOutputStream but it is quite confusing. Supposedly the Ess library can write to file from a float array, but I'm not having any luck there. If I can get the Max solution to be close to or less than real-time, then that option should work great.

Ah, you're about the counter!  Nice tidying up!

I've found that jit.textfile seems to work a little better with VERY large files (hundreds of MB or more, which is what I'm dealing with).  Might take some more experimentation to see if it really makes any difference.

As for binary data... yeah.  I'm poking around FileOutputStream but it is quite confusing.  Supposedly the Ess library can write to file from a float array, but I'm not having any luck there.  If I can get the Max solution to be close to or less than real-time, then that option should work great.


Manually writing sample values to buffer... faster