sqrt gives me wrong numbers

Feb 14, 2013 at 7:44am

sqrt gives me wrong numbers

Recently I found out that SQRT gives me wrong numbers from the 8th digit onward in decimal numbers.
For example SQRT 2. gives me 1.414213538170. The correct one is, of course, 1.41421356237.
Does anyone know why?

#66506
Feb 14, 2013 at 8:30am

Hi,

of course, 1.41421356237 is also not the correct value, since sqrt(2) is an irrational number, and as such, you can’t represent it as a finite decimal number. In a digital environment, you have to round it at some point to some value. 1.41421356237 is a good approximation in the decimal system.

Floating-point numbers in Max are 4-byte ‘single-precision floating point’ numbers, using the binary system. 1.414213538170 is the decimal representation of the binary number that is the best approximation of sqrt(2) using the number of binary digits allowed by the single-precision representation.

Hope this helps,
Ádám

#239365
Feb 14, 2013 at 10:02am

No, I did not write correctly. I meant “the correct one goes like 1.4142135623730…..”

I just read the WIKI page, what “sigle precision floating point” and “single-precision representation” mean. Not so easy to understand, but I think I got it. It helps me later so that I should no more stupidly change the number of decimal places in the inspector window of FLONUM ( more than 7?). Thanks!

#239366
Feb 14, 2013 at 11:05am

@Shinji Kanki : I can not find the wiki pages that you refered to ; can you post the link ?

This Python page ( http://docs.python.org/3.3/tutorial/floatingpoint.html ) is easy to understand.

#239367
Feb 14, 2013 at 11:17am
#239368
Feb 14, 2013 at 11:38am

And thanks, Nicolas. I just read the Python page. I understood perfectly.

#239369
Feb 14, 2013 at 11:51am

Hi,

Thanks ; ah wikipedia ! I thought you was talking about the cycling74 wiki ;-)

#239370

You must be logged in to reply to this topic.