When I store a float using pattr, it is written to the XML file with an accuracy of only 6 decimals. Also, numbers like 3.70021 are stored as 3.7.

As I'm working with chaotic functions, a difference of 0.0000001 leads to radically different results. I would need to store the exact state of my float variables, down to the last bit. Is there any way to do this?

When I store a float using pattr, it is written to the XML file with an accuracy of only 6 decimals. Also, numbers like 3.70021 are stored as 3.7. 

As I'm working with chaotic functions, a difference of 0.0000001 leads to radically different results. I would need to store the exact state of my float variables, down to the last bit. Is there any way to do this?


storing-floats-accurately-in-pattrstorage

I have found one way to work around this, that is to manually edit the XML file and enter the missing decimals there. Though this does slow down the programming quite a lot and I would appreciate any advice on how to save them directly from Max.

I believe you're dealing with issues associated with the fact that Max works with 32-bit floating point, so it's unlikely you're going to get more resolution for floating-point values than you're seeing.

Peter Elsea's explanation (from his tutorial "Max and Numbers") remains a touchstone of clarity on this point. And I quote:

"The float data type seems like a familiar sort of number-most of us have been writing decimals since grade school. However, there are some features of the way computers deal with floating point numbers that you should be aware of. Even if a number is shown on the screen as a decimal number, the computer is still working with bits, and has a limited number of them at its disposal. In the case of Max, that's 32 bits. These are used to represent the infinite range of decimal numbers by a scheme called floating point notation. You may be familiar with a version of this known as scientific notation- to represent 4823, you write 4.823 X 10^4.

To encode floating point numbers, Max follows a convention known as IEEE single precision floating point format. This uses 32 bits as follows:

23 bits for the significand, which has the form b(sub m).bbbbbbbbbbbbbbbbbbbbbbb where each b is either 1 or 0.

The actual value of the number is ± significand x 2^(exponent)

There is further massaging to save bits or processing time where possible. For instance, the exponent can be positive or negative, but instead of having a sign bit in the exponent, 127 is subtracted from the exponent when the number is converted. With 8 bits, the exponent may range from –127 to 128. In addition b(sub m) is unnecessary since it's always the same. ( 1 for most numbers, 0 if the exponent is –127.)

The result of all this is that floats can sometimes surprise you. For instance, you cannot represent all possible numbers with this scheme. In fact there are fewer floating point numbers than there are ints. The difference is that while ints are simply limited in magnitude, floats are spotty, jumping from value to value. With large numbers, the jumps can be pretty big, say from 134217712 to 134217720.

With small numbers some odd things happen too. You can see this by stretching a float box, typing in 0, and scrubbing the value up. You'll see the numbers change by steps of 0.01 up to 1.14, but the next value is 1.149999. That's because you can't actually represent 0.01000000000000000 as a binary number times some binary power of two. The value is really something like 0.0099999999999, which will round up to 0.01, but if you keep adding it in, eventually the rounding error catches up with you and 1.149999 pops up."

I suppose it's possible that there's a bug with the way that pattr stores 32-bit floats, but if so I'm not seeing it on a cursory look (Macbook Pro 10.5.8, current version of Max). In fact, the enclosed patch suggests to me that you might be able to type in whatever value you wish, but the result is still truncated to 32-bit on recall. Here's the patch:

And here's the .json file with the preset information in it (this is raw text):

        "slots" :         {

            "1" :             {

                "id" : 1,

                "data" :                 {

                    "nmbr" : [ 1.618034 ]

            "2" :             {

                "id" : 2,

                    "nmbr" : [ 1.61803398 ]

On my system, presets 1 and 2 are recalled to the same floating-point value of 1.618034 (the nearest 32-bit value), as I'd expect.

I believe you're dealing with issues associated with the fact that Max works with 32-bit floating point, so it's unlikely you're going to get more resolution for floating-point values than you're seeing. 

"The float data type seems like a familiar sort of number-most of us have been writing decimals since grade school. However, there are some features of the way computers deal with floating point numbers that you should be aware of. Even if a number is shown on the screen as a decimal number, the computer is still working with bits, and has a limited number of them at its disposal. In the case of Max, that's 32 bits. These are used to represent the infinite range of decimal numbers by a scheme called floating point notation. You may be familiar with a version of this known as scientific notation- to represent 4823, you write 4.823 X 10^4. 

To encode floating point numbers, Max follows a convention known as IEEE single precision floating point format. This uses 32 bits as follows: 

1 bit to indicate sign (0 for positive)
8 bits for the exponent
23 bits for the significand, which has the form  b(sub m).bbbbbbbbbbbbbbbbbbbbbbb where each b is either 1 or 0. 

The actual value of the number is ± significand x 2^(exponent) 

There is further massaging to save bits or processing time where possible. For instance, the exponent can be positive or negative, but instead of having a sign bit in the exponent, 127 is subtracted from the exponent when the number is converted. With 8 bits, the exponent may range from –127 to 128. In addition b(sub m) is unnecessary since it's always the same. ( 1 for most numbers, 0 if the exponent is –127.) 

The result of all this is that floats can sometimes surprise you. For instance, you cannot represent all possible numbers with this scheme. In fact there are fewer floating point numbers than there are ints. The difference is that while ints are simply limited in magnitude, floats are spotty, jumping from value to value. With large numbers, the jumps can be pretty big, say from 134217712 to 134217720. 

{
	"pattrstorage" : 	{
		"name" : "ffooz",
		"slots" : 		{
			"1" : 			{
				"id" : 1,
				"data" : 				{
					"nmbr" : [ 1.618034 ]
				}

			}
,
			"2" : 			{
				"id" : 2,
				"data" : 				{
					"nmbr" : [ 1.61803398 ]
				}

On my system, presets 1 and 2 are recalled to the same floating-point value of 1.618034 (the nearest 32-bit value), as I'd expect.


Storage in JSON files seems OK here. I wonder if you're putting numbers into [flonum] objects whose precision is lower than you want? AFAIK, the width of a [flonum] determines its precision.

Hi Gregory, thanks for the reply! In an earlier query of mine, the problem was the resolution of the 32-bit floating point, but this is different.

Pattrstorage uses a maximum of 6 decimal places, but a 32-bit floating point variable can accurately represent up to 7. Keep in mind that a floating point value has 7 significant digits regardless of the decimal point. Try to redo your example above with a 0 before the decimal point, then you won't get the nearest 32-bit value, but a trunctated value. Even 0.00001234567 is an accurate 32-bit float value, this is stored as 0.000012

I should also point out that all numbers that I wish to store are results from calculations within max, therefore they are all in the form of 32-bit floating point numbers. I do not try to store numbers that I have input manually into the patch. From what I've seen, the flonum box is not even capable of displaying any value that isn't a valid 32-bit float value.

Hi Gregory, thanks for the reply! In an earlier query of mine, the problem was the resolution of the 32-bit floating point, but this is different. 

I should also point out that all numbers that I wish to store are results from calculations within max, therefore they are all in the form of 32-bit floating point numbers. I do not try to store numbers that I have input manually into the patch. From what I've seen, the flonum box is not even capable of displaying any value that isn't a valid 32-bit float value.


not sure if this is relevant but you may also want to check the max prefs. the float display correction option may be causing you to see the rounded float as opposed to the un-rounded float.

i got caught with coll and floats, bcos of this... not sure how pattr deals with this.

I can't comment for definite on whether or not this is a bug but a preliminary look suggests that it is, in that floats are stored in the xml/json file as decimal representations of no more than 6 decimal places. I would think that storing floats in decimal format would always have the possibility of creating problems (and be unwise)....

"Even 0.00001234567 is an accurate 32-bit float value" - I don't think it is - That number can't be exactly represented as a floating point as far as I can tell because it can't be expressed exactly in terms of powers of two. This website is useful for looking at this stuff:

http://babbage.cs.qc.cuny.edu/IEEE-754/Decimal.html

Note that the number you suggest continues beyond at 32 float (by checking the representation as a double).

Basically it's best to leave floating point stuff as floating point to maintain maximum accuracy.

A possible workaround is to convert floats to 32 bit ints (using the same bit pattern) before saving them to pattr, and convert back when you load them. This would definitely work, as what is needed here is to store the bi pattern (or binary representation of the float), rather than an approximate decimal representation - but I'm not sure of the easiest way to achieve this using max objects (without a dedicated external). Possibly something in java could be put together quite easily, or it may be that some objects exist to do this already fairly easily.

"Even 0.00001234567 is an accurate 32-bit float value"  - I don't think it is - That number can't be exactly represented as a floating point as far as I can tell because it can't be expressed exactly in terms of powers of two. This website is useful for looking at this stuff:

Basically it's best to leave floating point stuff as floating point to maintain maximum accuracy. 

Thanks Alex! I shall look up the possibility to convert them to 32 bit ints.

Yes, I'm sorry I was a bit unclear. 0.00001234567 is indeed not an accurate float number. What I meant to say was that it is a different float number from 0.00001234568. Both of which are treated as the same numbers by pattrstorage. Or actually, even pattrstorage treats them as different 32-bit numbers, until it converts them to text form for the XML-file. I can reload my accurate states from pattrstorage until I close the patch and it writes the XML.

For accuracy, it would be enough for me with 10-12 decimals, then the nearest float value should always be the correct value. I was hoping there was some settings to allow this (or a way to store floats in binary form), but apparently there isn't... Got to find a workaround then.

Another thought - much easier to achieve and I imagine it should work. Multiply by a largish power of two (try 65536 or more) to shift the float exponent larger (the significand and sign will remain the same), then divide by the same number when you read it back out.....

I think that'll gain you accuracy without the need for anything complex...

Thanks again Alex! This seems to work! You saved the day! :)

I had already tried multiplying and dividing by 10000, but that didn't work. I didn't realize it had to be a power of 2.

If you use a power of two it ensures that you keep the results accurate (unless the numbers involved are VERY big or VERY small) because of the way floats work - otherwise you may suffer some potential inaccuracy of calculation that makes the whole thing pointless. The silly thing here is that larger floats are fine (with more significant digits) -the limitation by decimal places is not a particularly suitable approach to a *floating* point number. Limiting the number of significant bits would be a lot more sensible.

will convert the float to a symbol with 10 decimal places of "precision". It will store correctly as a symbol in [pattr], when you get it back out you need [fromsymbol] and be sure the [float] it goes into has enough decimal places chosen in the Inspector. However, you can't interpolate symbols in [pattrstorage], but maybe you don't need to do that.

The multiply/divide technique above sounds like it works fine too, this is just another thing I picked up (thanks Luke H.) when I was trying to store high-precision floats into a [text] file and the truncating was driving me nuts!

I did try the sprintf earlier, but I couldn't even figure out what object to use to store it. What object holds a symbol and appears in my storagewindow when I edit scripting name? I could only make objects with numeral values appear there...

I don't need interpolation between the values.

I did try the sprintf earlier, but I couldn't even figure out what object to use to store it. What object holds a symbol and appears in my storagewindow when I edit scripting name? I could only make objects with numeral values appear there... 

I don't need interpolation between the values.


Whew, I thought I had a quick answer, but there are only more questions... maybe someone can take a look at this and see where it breaks down? It *almost* seems to do what we're looking for, with [umenu] storing symbols...

if things seem to "break", re-create the [pattrstorage] and/or the [pattr] and try again...keep checking the client/storagewindows as you change the data, very strange...

Storing floats accurately in pattrstorage