Very weird issue - is this just happening on my system?
Hi all, something weird is happening with this very, very simple patch:
scrolling up through values outputs a float with two decimal places - as you would expect, until I get to 1603 at which point the output of [/100.] starts looking pretty odd - according to this patch 1603 / 100. = 16.030001 !! If I continue to scroll up through higher values, more and more weird values get spat out by [/100.].
Can anyone else confirm this issue or is it just my system? I'm running Max 6.1.1 on OS X 10.8.3
floating point numbers magic.
Furthermore, [round] can't rectify the problem. This seems very weird!
@Andrzej Kopeć
Thanks for the reply Andrzej, that is an interesting link. If I want to convert a number like 1603 to 16.03, how should I do it then? I'd always presumed that dividing it by 100. was the way forward, with that not working and [round] not working either I'm not sure how to proceed.
This issue has been discussed many times on the forums. The point is that this number is not computable with finite precision floating point numbers - you are seeing the closest binary representation of this number.
If you want to limit your flonum's display to two decimal places you can do so by setting the "Number of Decimal Places" attribute in the flonum inspector. But the value that is being passed will still be 16.030001.
@Jesse
Sorry for any duplication, a quick check of the forum brought up nothing (should have looked harder!) so I thought this might be an issue with having updated to 6.1.1. Thanks for taking the time to explain! :)
32-bit floating point system cannot exactly represent it, so there must be some tradeoffs. The issue is rather about displaying this number (visual interpretation). If you multiply it by 100, you will get the expected result. "Floats" are not decimal fractions.