sqrt gives me wrong numbers
Feb 14, 2013 at 7:44am
sqrt gives me wrong numbersRecently I found out that SQRT gives me wrong numbers from the 8th digit onward in decimal numbers. 

Feb 14, 2013 at 8:30am
Hi, of course, 1.41421356237 is also not the correct value, since sqrt(2) is an irrational number, and as such, you can’t represent it as a finite decimal number. In a digital environment, you have to round it at some point to some value. 1.41421356237 is a good approximation in the decimal system. Floatingpoint numbers in Max are 4byte ‘singleprecision floating point’ numbers, using the binary system. 1.414213538170 is the decimal representation of the binary number that is the best approximation of sqrt(2) using the number of binary digits allowed by the singleprecision representation. Hope this helps, 

Feb 14, 2013 at 10:02am
Thanks, Ádám, for your quick reply! I just read the WIKI page, what “sigle precision floating point” and “singleprecision representation” mean. Not so easy to understand, but I think I got it. It helps me later so that I should no more stupidly change the number of decimal places in the inspector window of FLONUM ( more than 7?). Thanks! 

Feb 14, 2013 at 11:05am
@Shinji Kanki : I can not find the wiki pages that you refered to ; can you post the link ? This Python page ( http://docs.python.org/3.3/tutorial/floatingpoint.html ) is easy to understand. 

Feb 14, 2013 at 11:17am
This is the page. http://en.wikipedia.org/wiki/Singleprecision_floatingpoint_format 

Feb 14, 2013 at 11:38am
And thanks, Nicolas. I just read the Python page. I understood perfectly. 

Feb 14, 2013 at 11:51am
Hi, Thanks ; ah wikipedia ! I thought you was talking about the cycling74 wiki ;) 
You must be logged in to reply to this topic.