Cycling 74 » Topic: Compensating for loss of float accuracy in JavaScript? http://cycling74.com/forums/topic/compensating-for-loss-of-float-accuracy-in-javascript/feed Wed, 19 Jun 2013 04:40:01 +0000 http://bbpress.org/?v=2.2.4 http://cycling74.com/forums/topic/compensating-for-loss-of-float-accuracy-in-javascript/#post-32694 <![CDATA[Compensating for loss of float accuracy in JavaScript?]]> http://cycling74.com/forums/topic/compensating-for-loss-of-float-accuracy-in-javascript/#post-32694 Sat, 30 Jun 2007 17:45:49 +0000 nick rothwell / cassiel Here’s a little problemette I’m having: when I pass a float like
“4.53″ into JavaScript from the Max world, it gets turned into 4.53
+noise digits. Internally as a constant in JavaScript, it’s fine.
(Example enclosed.)

I assume the noise digits are due to rounding (the number 4.5 is
fine, presumably because it has a precise IEEE representation). I
don’t know why it works internally in JS: perhaps that’s doing some
kind of symbolic interpretation.

So, my question is: how do I truncate or round the accuracy of
rendering floats into strings in JS? I couldn’t find anything in the
documentation regarding floating point formatting a la sprintf().

– N.

— Patcher file:

max v2;
#N vpatcher 10 59 272 236;
#P window setfont Monaco 9.;
#P window linecount 1;
#P newex 60 83 88 262153 js me-test.js;
#P message 60 60 58 262153 foo 4.53;
#P connect 0 0 1 0;
#P pop;

— JS file (me-test.js):

function foo() {
post(“arg gives: ” + arguments[0] + “n”);
post(“literal gives: ” + 4.53 + “n”);
}

Nick Rothwell / Cassiel.com Limited
http://www.cassiel.com
http://www.myspace.com/cassieldotcom
http://www.loadbang.net

]]> http://cycling74.com/forums/topic/compensating-for-loss-of-float-accuracy-in-javascript/#post-108029 <![CDATA[Re: Compensating for loss of float accuracy in JavaScript?]]> http://cycling74.com/forums/topic/compensating-for-loss-of-float-accuracy-in-javascript/#post-108029 Sat, 30 Jun 2007 17:55:07 +0000 Emmanuel Jourdan On 30 juin 07, at 19:45, Nick Rothwell wrote:

> So, my question is: how do I truncate or round the accuracy of
> rendering floats into strings in JS? I couldn’t find anything in
> the documentation regarding floating point formatting a la sprintf().

Somthing like that should work (no tested though):

function bang()
{
var toto = 0.576528356;
post(toto.toFixed(3));
}

ej

]]> http://cycling74.com/forums/topic/compensating-for-loss-of-float-accuracy-in-javascript/#post-108030 <![CDATA[Re: Compensating for loss of float accuracy in JavaScript?]]> http://cycling74.com/forums/topic/compensating-for-loss-of-float-accuracy-in-javascript/#post-108030 Sat, 30 Jun 2007 18:33:30 +0000 Pierre Alexandre Tremblay

]]> http://cycling74.com/forums/topic/compensating-for-loss-of-float-accuracy-in-javascript/#post-108031 <![CDATA[Re: Compensating for loss of float accuracy in JavaScript?]]> http://cycling74.com/forums/topic/compensating-for-loss-of-float-accuracy-in-javascript/#post-108031 Sat, 30 Jun 2007 18:58:58 +0000 Emmanuel Jourdan On 30 juin 07, at 20:33, Pierre Alexandre Tremblay wrote:

> I always wonder: is it really a lost of resolution, or is it the
> opposite: we just see a 32 bit float being properly represented in
> 64-bit double float?

JS works with 64 bit double float. but for representing the number as
a string you usually need to display it with less precision.

ej

]]> http://cycling74.com/forums/topic/compensating-for-loss-of-float-accuracy-in-javascript/#post-108032 <![CDATA[Re: Compensating for loss of float accuracy in JavaScript?]]> http://cycling74.com/forums/topic/compensating-for-loss-of-float-accuracy-in-javascript/#post-108032 Sat, 30 Jun 2007 19:18:23 +0000 nick rothwell / cassiel On 30 Jun 2007, at 18:55, Emmanuel Jourdan wrote:

> var toto = 0.576528356;
> post(toto.toFixed(3));

Cool – thanks. I’m sure I didn’t come across this when I dug through
the JS 1.5 docs I have here…

– N.

Nick Rothwell / Cassiel.com Limited
http://www.cassiel.com
http://www.myspace.com/cassieldotcom
http://www.loadbang.net

]]> http://cycling74.com/forums/topic/compensating-for-loss-of-float-accuracy-in-javascript/#post-108033 <![CDATA[Re: Compensating for loss of float accuracy in JavaScript?]]> http://cycling74.com/forums/topic/compensating-for-loss-of-float-accuracy-in-javascript/#post-108033 Sat, 30 Jun 2007 23:08:57 +0000 Mattijs Moving this thread to the javascript forum, please direct your replies to the javascript list.

]]>