Text object - characters per line limit

Kenneth Newby's icon

Hello all. Is there any way to overcome the character limit per line of the text object? Or any workaround?

Thanks in advance.

Kenneth.

Kenneth Newby's icon

Obviously not a high priority. It's frustrating in that the text object can read the file and displays the correct line-lengths in its display but doesn't have the heft to actually output that content.

Inconsistent.

jvkr's icon

Are you sure it is the text object, and not the object dealing with the output? What does a [ zl.len @zlmaxsize 9999] report?

Kenneth Newby's icon

Yes it's the text object. I have a text file with a line that [ zl.len @zlmaxsize 9999 ] reports as 508, but the text object outputs the line as truncated to a length of 411.

Source Audio's icon

In max 6 & 7 text outputs 4096 chars per line,
which seems to be the per line limit.

Maybe there is something in that text line which truncates
the output.

Kenneth Newby's icon

Here's a simple patch that demonstrates the problem. Read the longLine.txt file into the text object. Then send the <line 1> message to the text object. Compare the output with the contents and you'll see that the text object is truncating the line.

text test.maxpat
Max Patch

longLine.txt
text/plain 4.39 KB

Kenneth Newby's icon

. . . and there are no special characters in the line. It's a simple string of floats.

Source Audio's icon

That text file has 4500 chars inside.
First line has 4499 chars, second is empty.
float like 0.123456 counts 8 chars.
Selection shows 4096 chars and that's where
output of text object ends :



Kenneth Newby's icon

That is, indeed, the place where the truncation occurs. I'll work up a scheme to break my source text files into smaller lines and stitch them together to get the longer data. I'm working with SPEAR files and they can have these very long lines.

It still strikes me as odd that the text object itself can contain a longer line than it can output. That was a source of my confusion.

Thanks for that clarification!

Kenneth Newby's icon

So I guess that's a feature request / enhancement for the text object: please raise the maximum number of chars per line.

Source Audio's icon

I don't know if that is acceptable for Your needs,
but coll objects does not have that limit per line.
All one would need to do is to insert :
1, float string here;
for proper coll formating
and maybe more lines per coll file
2, next float string here;
3, next float string here;

Kenneth Newby's icon

Yes, indeed, and I'm actually reading the text into a dictionary as I can store the partial data in a more convenient hierarchical form. The problem is getting the text into the patcher in the first place, which is why the text object is useful here. I could certainly load a dictionary or coll by hand but SPEAR analysis files can be quite large. I fiddled for a bit with the cnmat sdif objects but they didn't seem to like the sdif files that SPEAR exported.

Source Audio's icon

I don't really see the problem, that is plain text files with just few lines
that need to get removed, and 3 chars inserted to make coll file..
One could even make drag&drop script to do so

Kenneth Newby's icon

Perhaps I'm being a bit thick here, but the bottleneck to doing this is the text object itself. If it won't output the complete lines then I'm looking at parsing a text file in some other way?

jvkr's icon

Maybe javascript offers some options. There is the jsfiletester-example in the examples folder.

Source Audio's icon

I don't know how You want to use the Data in SPEAR files.
That is the main point.
If You want to be able to output complete line, than coll would be
ok, just one needs to number the lines and add semicolon at end.
here is unix script to do so :
awk '{printf("%d, %s\n", NR,$0";")}' path to input file > path to output file
for example:
awk '{printf("%d, %s\n", NR,$0";")}' ~/Desktop/SPEAR/file-1.txt > ~/Desktop/SPEAR/file-1Coll.txt
-----------------
old file:
aaa
bbb
ccc

new file:
1, aaa;
2, bbb;
3, ccc;


If You try to load large text file into javascript, save Your work before
as it is probably going to freeze max.

Roman Thilenius's icon

when you feel the need to put more than one page of text into one line, you should normally consider using signal buffers or jitter matrices (but the latter have a size limit somewhere, too.)

of course if the data is already in a text file...

>> I'll work up a scheme to break my source text files into smaller lines and stitch them together

that would be my approach, too.

otoh, you could as well try to do this with a text editor outside max.

Kenneth Newby's icon

The Unix script approach works like a charm. I converted that script with the appropriate escape characters to be able to be used with the shell object. For those with similar issues here's the script:

awk '{printf(\"%d\, %s\\n\"\, NR\,$0\"\; \" )}' ~/Desktop/SPEAR/file-1.txt > ~/Desktop/SPEAR/file-1Coll.txt

Just pop that into a message box and send to the shell object. This script, of course, assumes there's a folder named SPEAR on the Desktop that contains a file named file-1.txt Rename these for your own purposes. I'll be making a simple drag and drop solution that I'll post here shortly that will simplify the process even more.

Thanks for the help!

Kenneth Newby's icon

Oh, and what I'm up to is I'm looking to do some orchestral synthesis (aka audio mosaicing) using SPEAR analysis files as source material. The first step has been reading in the files and being able to play them back as simple additive synthesis—which simply reproduces what SPEAR can do with its analysis. That's all working nicely in Max with the ability to independently alter time and pitch, spectrum stretch/compress, amplitudes, etc. Next step is massaging these into scores for virtual instruments.

Kenneth Newby's icon

Here's a drag and drop solution that uses the dropfile and shell objects to load a coll with the contents of a SPEAR analysis text file. It assumes a directory named SPEAR on the Desktop containing the analysis files. (caveat: no spaces in filenames allowed.)

formatTextFileForColl.maxpat
Max Patch

Source Audio's icon

That's good news.
One could also remove the 5 leading lines using
sed -i '' '1,5d;' ~/Desktop/SPEAR/file-1.txt
or
sed -i '' '/par/, /data/d' ~/Desktop/SPEAR/file-1.txt

to remove lines matching :
par-text-frame-format
point-type index frequency amplitude
partials-count
frame-count
frame-data

before numbering the lines, but it has to be done
outside of max.
At least in max6 shell is not allowed to delete the temp file
after removing the lines, so one gets left with 2 files.
A simple droplet that would perform that would be easy to do.
Here is a bit modified patch that has different path handling.
Spaces in file name are no problem.

Max Patch
Copy patch and select New From Clipboard in Max.


Kenneth Newby's icon

Lovely! I've not spent enough time with regular expressions to be able to manipulate the path/filename in this elegant way. You're inspiring me to dig a little deeper into this.

In the meantime, again, thanks for the constructive interaction on this. Here's an update on where I'm at with the project... a screenshot of the display of the partial analysis of the song of a Thrush. Cursor follows playback. Partials can be selected and highlighted in red, soloed, muted, etc.

Max is playing this nicely. Now that I've got access to each partial individually it should soon be possible to move beyond resynthesis with sinusoidal oscillators to trying out some mappings using virtual instruments.

Source Audio's icon

Looks great .

Do you know/remamber Gary Lee Nelson's SPEAR externals ?

http://sites.music.columbia.edu/cmc/courses/g6610/fall2005/week13/index.html

http://sites.music.columbia.edu/cmc/courses/g6610/fall2005/week11/index.html

Kenneth Newby's icon

Yes, those Columbia archives were what pointed me to SPEAR in the first place. “We stand on the shoulders of giants.” ;-)