sqrt gives me wrong numbers

Feb 14, 2013 at 7:44am

sqrt gives me wrong numbers

Recently I found out that SQRT gives me wrong numbers from the 8th digit onward in decimal numbers.
For example SQRT 2. gives me 1.414213538170. The correct one is, of course, 1.41421356237.
Does anyone know why?

Feb 14, 2013 at 8:30am


of course, 1.41421356237 is also not the correct value, since sqrt(2) is an irrational number, and as such, you can’t represent it as a finite decimal number. In a digital environment, you have to round it at some point to some value. 1.41421356237 is a good approximation in the decimal system.

Floating-point numbers in Max are 4-byte ‘single-precision floating point’ numbers, using the binary system. 1.414213538170 is the decimal representation of the binary number that is the best approximation of sqrt(2) using the number of binary digits allowed by the single-precision representation.

Hope this helps,

Feb 14, 2013 at 10:02am

Thanks, Ádám, for your quick reply!
No, I did not write correctly. I meant “the correct one goes like 1.4142135623730…..”

I just read the WIKI page, what “sigle precision floating point” and “single-precision representation” mean. Not so easy to understand, but I think I got it. It helps me later so that I should no more stupidly change the number of decimal places in the inspector window of FLONUM ( more than 7?). Thanks!

Feb 14, 2013 at 11:05am

@Shinji Kanki : I can not find the wiki pages that you refered to ; can you post the link ?

This Python page ( http://docs.python.org/3.3/tutorial/floatingpoint.html ) is easy to understand.

Feb 14, 2013 at 11:17am
Feb 14, 2013 at 11:38am

And thanks, Nicolas. I just read the Python page. I understood perfectly.

Feb 14, 2013 at 11:51am


Thanks ; ah wikipedia ! I thought you was talking about the cycling74 wiki ;-)


You must be logged in to reply to this topic.