pseudostereo

If I try to enter the number 29.97 as an item in umenu or tab, it immediately changes to 29.969999 - I guess due to floating point rounding errors. (But 29.97 does display correctly in comment, message and textbutton objects)

Is there any workaround for this? I'm trying to make a umenu of typical frame rates and 29.97 is a standard rate for NTSC video -- it's exasperating that I can't display it.

I tried turning off Float Display Correction in Max preferences, but it had no effect. 

floating-point-bug-in-umenu-tab

sometimes i also feel that the float display correction flag sometimes

check out if it is really set correctly, just as the font size it can be applied

"to all objects" differently from to "the whole patch"

sometimes i also feel that the float display correction flag sometimes
stop working.

check out if it is really set correctly, just as the font size it can be applied
"to all objects" differently from to "the whole patch"

Chris & Raja, thanks for the quick (and precisely synchronized) responses.

Roman, how can I determine whether the float correction flag is set correctly? I set it in Max Preferences, quit and restarted - is there a per-patch setting as well?

Anyway, here's my workaround - the goal is to read the frame rate off of incoming videos & select the correct umenu item automatically - so the frame rate always comes in as a number & has to be converted to a symbol to select the umenu item - but if it's not the right number it doesn't create the right symbol. Workaround is to use a split object to check the number before it gets converted to a symbol, then select just that item with an integer.

The odd thing is that example #1 shouldn't work but does, and example #2 should work but doesn't - mysterious.

29.97 needs to be a symbol, because this value doesn't have a valid 32 bits representation.

i dont know, i just see that the correction setting seems out of order

from time to time, then i restart max and all is ok.

i dont know, i just see that the correction setting seems out of order
from time to time, then i restart max and all is ok.


I'm just getting started with Max, but I know a fair amount about floating-point arithmetic.

When you say that "29.97 doesn't have a valid 32-bit representation," that claim is literally true. However, there is no technical reason that a floating-point system cannot produce the string "29.97" as its favored representation for the representable number that is closest to 29.97 in value. I am guessing it would require some internal changes to Max that might not be feasible, but it's not impossible.

The most tempting strategy for converting between floating point and string is to produce, for conversion in each direction, an appropriate rounding of the closest value to the infinite-precision conversion. This strategy typically leads to converted values such as 29.969999.

However, there is another strategy, which some programming languages actually mandate, that works much better in practice:

1) When you convert a string to floating point, you conceptually form the infinite-precision binary representation of the string, and then round it to the precision of your floating-point system.

2) When you convert a floating-point number to a string, the result should be the shortest string that when converted back to floating point yields exactly the same result as the number you are converting.

A programming language that uses these conversion rules has two very pleasant properties:

1) A string converted to floating point and back to a string will *never* yield a result that is longer than the original string, though it might yield a shorter result (i.e. 29.970000000001 might yield 29.97, but the reverse will never happen).

2) Writing a floating-point number as a string and then reading it again will *always* yield exactly the same number.

http://kurtstephens.com/files/p372-steele.pdf

When you say that "29.97 doesn't have a valid 32-bit representation," that claim is literally true.  However, there is no technical reason that a floating-point system cannot produce the string "29.97" as its favored representation for the representable number that is closest to 29.97 in value.  I am guessing it would require some internal changes to Max that might not be feasible, but it's not impossible.

The most tempting strategy for converting between floating point and string is to produce, for conversion in each direction, an appropriate rounding of the closest value to the infinite-precision conversion.  This strategy typically leads to converted values such as 29.969999.

1) A string converted to floating point and back to a string will *never* yield a result that is longer than the original string, though it might  yield a shorter result (i.e. 29.970000000001 might yield 29.97, but the reverse will never happen).

More information can be found here:  http://kurtstephens.com/files/p372-steele.pdf


floating point bug in umenu, tab