float accuracy

    Feb 08 2010 | 4:47 pm
    It seems that Max by default only treats float numbers accurately up to 7 significant digits. The flonum box does show up to 16 decimals, but I haven't found a way to accurately adjust anything but the first 7 decimals (or less if there is anything but 0 in front of the decimal point). Is there any way to increase this accuracy?

    • Feb 08 2010 | 8:59 pm
      Number of Decimal Places attribute in the Inspector?
      That's not the whole story though, to send it to various places where it would auto-truncate (coll or text) you need to send it through
      sprintf "%.10f"
      where %.10 represents 10 digits of precision. Thanks to Luke Hall for this one :)
    • Feb 08 2010 | 10:17 pm
      Thanks for the reply! But I'm sorry, that's not the problem. In the inspector I can only define how many decimals are shown. But from the 8th decimal on, I cannot adjust the number accurately.
      The main problem is, however, Max doesn't seem to take in account more than 7 significant digits in it's calculations.
      Try connecting a flonum object through a + 100. object to another flonum object. Set Number of Decimal Places to 12 in both, then adjust the number in the first. The second flonum box will only follow the first flonum box accurately to four decimals (7 significant digits). At least this is what happens on both of my computers...
      Is this just a built in limitation that I can't do anything about?
    • Feb 08 2010 | 10:27 pm
      I think what you're referring to is the limitations of single-precision 32-bit floating point numbers. It is a "limitation" of Max and MSP, but 32-bit float is pretty standard, and I would think it is unlikely to cause you any problems. Here is a better explanation than I could give of what I'm talking about:
    • Feb 08 2010 | 10:36 pm
      Thanks Tim! That explains it. I knew the basic floating point was limited, but didn't realise it was that limited... Actually, it did already cause me some problems, but I then decided to rework my plans and redesign my patch to turn the problems into advantages instead. :)
    • Feb 09 2010 | 8:55 am
      and if this is of any help, try the hr objects by Mr Clayton (on cycling's share page) as they provide 60-ish bit precision.
    • Aug 30 2012 | 9:18 pm
      thanks Tim, your link helped me to understand some strangeness i recently came across with floats in max. O.