Forums > MaxMSP

Very weird issue – is this just happening on my system?


Barry Farrimond
Apr 07 2013 | 6:49 pm

Hi all, something weird is happening with this very, very simple patch:

-- Pasted Max  Patch, click to expand. --  





scrolling up through values outputs a float with two decimal places – as you would expect, until I get to 1603 at which point the output of [/100.] starts looking pretty odd – according to this patch 1603 / 100. = 16.030001 !! If I continue to scroll up through higher values, more and more weird values get spat out by [/100.].


Can anyone else confirm this issue or is it just my system? I’m running Max 6.1.1 on OS X 10.8.3

ak
Apr 07 2013 | 6:56 pm

floating point numbers magic.

http://en.wikipedia.org/wiki/Floating_point#Accuracy_problems


Barry Farrimond
Apr 07 2013 | 6:57 pm

Furthermore, [round] can’t rectify the problem. This seems very weird!



-- Pasted Max  Patch, click to expand. --

Barry Farrimond
Apr 07 2013 | 7:20 pm

@Andrzej Kopeć
Thanks for the reply Andrzej, that is an interesting link. If I want to convert a number like 1603 to 16.03, how should I do it then? I’d always presumed that dividing it by 100. was the way forward, with that not working and [round] not working either I’m not sure how to proceed.


Jesse
Apr 07 2013 | 10:44 pm

This issue has been discussed many times on the forums. The point is that this number is not computable with finite precision floating point numbers – you are seeing the closest binary representation of this number.

If you want to limit your flonum’s display to two decimal places you can do so by setting the "Number of Decimal Places" attribute in the flonum inspector. But the value that is being passed will still be 16.030001.


Barry Farrimond
Apr 07 2013 | 10:48 pm

@Jesse
Sorry for any duplication, a quick check of the forum brought up nothing (should have looked harder!) so I thought this might be an issue with having updated to 6.1.1. Thanks for taking the time to explain! :)


ak
Apr 07 2013 | 11:10 pm

32-bit floating point system cannot exactly represent it, so there must be some tradeoffs. The issue is rather about displaying this number (visual interpretation). If you multiply it by 100, you will get the expected result. "Floats" are not decimal fractions.

Viewing 7 posts - 1 through 7 (of 7 total)
Forums > MaxMSP