rounding fun times

example?

Just try and type a decimal longer than 6 places in a messagebox, see what happens

and then try to do [* 1000.] - you should get your decimals right back ;) (at least up to your 32-bits, iirc.)

I could certainly be wrong, but it sounds as if what you're seeing has to do with floating point representations and the way Max works. If that's possibly the case, then I would recommend to you Peter Elsea's explanation (from his tutorial "Max and Numbers"), which remains a touchstone of clarity on this point. And I quote:

"The float data type seems like a familiar sort of number-most of us have been writing decimals since grade school. However, there are some features of the way computers deal with floating point numbers that you should be aware of. Even if a number is shown on the screen as a decimal number, the computer is still working with bits, and has a limited number of them at its disposal. In the case of Max, that's 32 bits. These are used to represent the infinite range of decimal numbers by a scheme called floating point notation. You may be familiar with a version of this known as scientific notation- to represent 4823, you write 4.823 X 10^4.

To encode floating point numbers, Max follows a convention known as IEEE single precision floating point format. This uses 32 bits as follows:

1 bit to indicate sign (0 for positive)
8 bits for the exponent
23 bits for the significand, which has the form b(sub m).bbbbbbbbbbbbbbbbbbbbbbb where each b is either 1 or 0.

The actual value of the number is ± significand x 2^(exponent)

There is further massaging to save bits or processing time where possible. For instance, the exponent can be positive or negative, but instead of having a sign bit in the exponent, 127 is subtracted from the exponent when the number is converted. With 8 bits, the exponent may range from –127 to 128. In addition b(sub m) is unnecessary since it's always the same. ( 1 for most numbers, 0 if the exponent is –127.)

The result of all this is that floats can sometimes surprise you. For instance, you cannot represent all possible numbers with this scheme. In fact there are fewer floating point numbers than there are ints. The difference is that while ints are simply limited in magnitude, floats are spotty, jumping from value to value. With large numbers, the jumps can be pretty big, say from 134217712 to 134217720.

With small numbers some odd things happen too. You can see this by stretching a float box, typing in 0, and scrubbing the value up. You'll see the numbers change by steps of 0.01 up to 1.14, but the next value is 1.149999. That's because you can't actually represent 0.01000000000000000 as a binary number times some binary power of two. The value is really something like 0.0099999999999, which will round up to 0.01, but if you keep adding it in, eventually the rounding error catches up with you and 1.149999 pops up."

Is it possible that what you're seeing is related to this?

Havent gone through Gregory's entire explanation, but if What Wetterberg says is true that's actually pretty terrible. It means that if I look at a number I have no way whatsoever of knowing what its actual precision is. Since much of what I do depends on accurate division this is... a problem.

It's Peter Elsea's explanation, not mine - *he* deserves the credit for this lucid explanation. Unless I'm misunderstanding you, you appear to be complaining about the fact that 32-bit floating point mathematical representations are not "precise." They never have been on ANY machine or piece of software that works in 32-bits - I'm surprised you've never run into this before. It's one of the reasons that 64-bit numerical calculations are more accurate and desireable in scientific calculations, etc. It's also what denormalized numbers are all about, and the reason for a hundred billion Max list posts about stuff that "never reaches N" when working with signal values.

It's everybody's problem.

http://www.cprogramming.com/tutorial/floating_point/understanding_floating_point_representation.html

Yeah I was about to say, isn't this just a problem with computers in general.

So one can either say "Well that's unacceptable!", and go on to create their own computer system without these limitations, or one can say "Isn't it amazing that we can even do this much with computers?", and proceed to find solutions that work around the limitations inherent to all computer systems.

My guess is one would find the second option a tad more manageable.

Um, no. I know all about floats and macheps and all that fun stuff(not the same but related, I know). My problem is that, if what wettenberg says is true, I cannot look at a message (and god knows what else) to see the precision of the float in question. I could care less about max rounding stuff to the nearest whatever, as long as it is consistent, transparent, and well-documented.

Yes, but, you see, that is in official docs, which is exactly my point. Whereas I had to make a whiny forum post to learn about this situation in max. Hay poquita problema aqui.