timbyates

I'm working on set of boxes for controlling floating-point number entry into Max. I've come across what seems to be a rounding error in floating-point number boxes, and I was wondering if this was a mistake on my part or whether it's a bug.

I've attached the files below. To make the patch work you type an integer into the left hand side of the number and scroll through the decimal portion using the + and - signs.

If the integer value is '0' it works fine, but as the value increases an error is introduced into the calculation when the decimal portion is changed. The javascript is outputing its values to the MAX window for comparison and the numbers seem to be fine, so the error seems to be happening in Max.

The error gets larger the higher a value you use, so it doesn't happen at 1 - 10, starts at around 20 and by the time you get to 100,000 is around 0.002.

I was wondering if anyone else has come across this and if there's a fix.

I've attached the patch along with the relevant javascript files.

floating-point-rounding-error

Hmm ..... I don't think the attachment worked so I've pasted the files below. The names of the three javascript files are at the top of the code for each:

//Increments or decrements a number box by one

//Rolls over between 9 and 0 and outputs a bang

//if so outputs int otherwise outputs zero

obj = this.patcher.getnamed("tbx_Number");

//Joins elements from number boxers to make float

//Increments or decrements a number box by one
//Rolls over between 9 and 0 and outputs a bang

function add()
{
if (x == 9)    {
    x = 0;
    outlet(1, "bang");
    }
else    {
    x = x+1;
    }
outlet(0, x);
}

function sub()
{
if (x == 0)    {
    x = 9;
    outlet(2, "bang");
    }
else    {
    x = x-1;
    }
outlet(0, x);
}

function zero()
{
x = 0;
outlet (0, x);
}

//tests input to see if it is an int
//if so outputs int otherwise outputs zero

function text(t)
{
    obj = this.patcher.getnamed("tbx_Number");
    t = parseInt(t, 10);

    if (isNaN(t))    {
        t=0;
    }

//buildnum.js
//Joins elements from number boxers to make float

function wholenum(a)
{
whole = parseInt(a, 10);
result();
}

function tennum(b)
{
ten = b;
result();
}

numout = whole + (ten/10)
outlet (0, numout);
post(numout);
post();
}


I can't see any rounding error other than the basic inaccuracy of the standard 32-bit floating point number. For more info see here: 

The javascript output in the max window doesn't show this error for small values as it is rounded to two decimal digits. If you set the integer part to 1,000,000, you'll notice that the javascript output features the same rounding error. If you set the integer part to 10,000,000, you won't get any decimal part at all in this patch (at least I don't).

I can't see any rounding error other than the basic inaccuracy of the standard 32-bit floating point number. For more info see here: http://www.lahey.com/float.htm

The javascript output in the max window doesn't show this error for small values as it is rounded to two decimal digits. If you set the integer part to 1,000,000, you'll notice that the javascript output features the same rounding error. If you set the integer part to 10,000,000, you won't get any decimal part at all in this patch (at least I don't).


Floating-point rounding error?