mugloch

This is probably me doing something wrong, any advice appreciated...

I've set the 'Text' object's precision attribute to 8, but the floating point numbers in the txt file are still being truncated to 6 decimal points, which is frustrating because it's GPS coords and all the activity is happening in the final few digits of the float.

bee1-25-6-15 v2.txt

float-precision-error.maxpat

Hi,
This is probably me doing something wrong, any advice appreciated...
I've set the 'Text' object's precision attribute to 8, but the floa

'text'-object-float-precision-error

I just noticed I'm using 32bit MAX, will that limit my float precision?

Even in the 'Text' object help-patch, the tab on floating point precision won't let me type numbers with more than 6 decimal places.

I just noticed I'm using 32bit MAX, will that limit my float precision?  

with 32 bits you only have a precision of 7-8 digits, and what you see in some GUI objects is a "corrected" value, which can differ from the true one. you can turn display correction off globally.

i have no experience with 64 bits in max, so i can not properly make comparisons here (in case that is different for 64 bit numbers in max.)

i have no experience  with 64 bits in max, so i can not properly make comparisons here (in case that is different for 64 bit numbers in max.)

Thanks Roman, do you know how I turn it off globally? I can't find anything in preferences or patcher info.

'text' object float precision error?