float accuracy

Feb 8, 2010 at 4:47pm

float accuracy

It seems that Max by default only treats float numbers accurately up to 7 significant digits. The flonum box does show up to 16 decimals, but I haven’t found a way to accurately adjust anything but the first 7 decimals (or less if there is anything but 0 in front of the decimal point). Is there any way to increase this accuracy?

Feb 8, 2010 at 8:59pm

Number of Decimal Places attribute in the Inspector?

That’s not the whole story though, to send it to various places where it would auto-truncate (coll or text) you need to send it through

sprintf “%.10f”

where %.10 represents 10 digits of precision.
Thanks to Luke Hall for this one :)

Feb 8, 2010 at 10:17pm

Thanks for the reply! But I’m sorry, that’s not the problem. In the inspector I can only define how many decimals are shown. But from the 8th decimal on, I cannot adjust the number accurately.

The main problem is, however, Max doesn’t seem to take in account more than 7 significant digits in it’s calculations.

Try connecting a flonum object through a + 100. object to another flonum object. Set Number of Decimal Places to 12 in both, then adjust the number in the first. The second flonum box will only follow the first flonum box accurately to four decimals (7 significant digits). At least this is what happens on both of my computers…

Is this just a built in limitation that I can’t do anything about?

Feb 8, 2010 at 10:27pm

I think what you’re referring to is the limitations of single-precision 32-bit floating point numbers. It is a “limitation” of Max and MSP, but 32-bit float is pretty standard, and I would think it is unlikely to cause you any problems. Here is a better explanation than I could give of what I’m talking about:


Feb 8, 2010 at 10:36pm

Thanks Tim! That explains it. I knew the basic floating point was limited, but didn’t realise it was that limited… Actually, it did already cause me some problems, but I then decided to rework my plans and redesign my patch to turn the problems into advantages instead. :)

Feb 9, 2010 at 8:55am

and if this is of any help, try the hr objects by Mr Clayton (on cycling’s share page) as they provide 60-ish bit precision.


Aug 30, 2012 at 9:18pm

thanks Tim, your link helped me to understand some strangeness i recently came across with floats in max. O.


You must be logged in to reply to this topic.