Forums > MaxMSP

### sqrt gives me wrong numbers

Feb 14 2013 | 7:44 am

Recently I found out that SQRT gives me wrong numbers from the 8th digit onward in decimal numbers.
For example SQRT 2. gives me 1.414213538170. The correct one is, of course, 1.41421356237.
Does anyone know why?

Feb 14 2013 | 8:30 am

Hi,

of course, 1.41421356237 is also not the correct value, since sqrt(2) is an irrational number, and as such, you can’t represent it as a finite decimal number. In a digital environment, you have to round it at some point to some value. 1.41421356237 is a good approximation in the decimal system.

Floating-point numbers in Max are 4-byte ‘single-precision floating point’ numbers, using the binary system. 1.414213538170 is the decimal representation of the binary number that is the best approximation of sqrt(2) using the number of binary digits allowed by the single-precision representation.

Hope this helps,
Ádám

Feb 14 2013 | 10:02 am

No, I did not write correctly. I meant "the correct one goes like 1.4142135623730….."

I just read the WIKI page, what "sigle precision floating point" and "single-precision representation" mean. Not so easy to understand, but I think I got it. It helps me later so that I should no more stupidly change the number of decimal places in the inspector window of FLONUM ( more than 7?). Thanks!

Feb 14 2013 | 11:38 am

And thanks, Nicolas. I just read the Python page. I understood perfectly.

Viewing 5 posts - 1 through 5 (of 5 total)

Forums > MaxMSP