Johan van Mol .org
HomeHome
ArticlesArticles
BlogBlog
Advanced SearchAdvanced Search
Home arrow Articles arrow Flash arrow parseFloat("512.56")*100 doesn't equal 51256
parseFloat("512.56")*100 doesn't equal 51256 Print E-mail
Wednesday, 22 March 2006

Math.floor( parseFloat("512.56") * 100 ) results in 51255, while I (and my client) was expecting 51256.
This is very annoying when your coding an online tax calculation program in Flash. So I tried to pin down the problem.

What happens in the program. The front-end is written in Flash, the back-end is a webservice written in C++. To avoid roundoff errors, the webservice works with integers only. All amounts are multiplied by 100, to be able to work with cents and still use integers. This is common practice in financial software.   Since € 110.125 is an invalid amount, (1 cent is the lowest valid amount), we decided to chop off the digits entered by the user after the 2nd digit after the decimal point.
So the user enters an amount in a TextField. This is a string. I convert it to a Float with the parseFloat() function. Then I multiply the Float by 100 and round off to lowest Integer. "512.5677" would result in 51256, thus ignoring digits behind the second digit after the decimal point. "512.56" Should result in 51256 too, but it doens't.

Try this in Flash:
trace(Math.floor(100*parseFloat("512.56")))
-> 51255

Let's break this down:
trace (parseFloat("512.56"));
-> 512.56

trace (parseFloat("512.56") * 100);
-> 51256
trace (Math.floor(parseFloat("512.56") * 100));
-> 51255

Keep in mind here that we're not rounding off 512.56, but 51256. So I would guess that the number 51256 generated by the function (parseFloat("512.56") * 100) is internally represented as 51255.99999999 or so, which is displayed as 51256.

Let's try another one:
trace (Number("280.03"));
> 280.03
trace (Number("280.03") * 100);
> 28003
trace (Math.floor(Number("280.03") * 100));
> 28002

Now you would expect that in Math.round(28003) is the same as Math.floor(28003), but it's not:
trace (Math.round(Number("280.03") * 100));
> 28003

It took some experimenting before I found out that it actually is a roundoff error. At first I thought that the parseFloat() function was the problem. But without the parseFloat() function we get the same results:
trace (Math.floor(512.56 * 100));
> 51255
trace (Math.floor(51256.0));
> 51256
trace (Math.floor(51256));
> 51256

So (512.56 * 100) doesn't equal 51256.

Macromedia technote 13989 will tell you that roundoff errors are a fact of life. I can live with that, but I wasn't expecting them to show up in the 2nd digit after the decimal point already.

Lessons learned: never mix currencies and floating point numbers.
Use a function like this to parse user input:

function parseInput(inputText:String):Number {
    var a:Array = inputText.split('.');
    var fraction;
    var decimal = a[0];
   
    if (a[1] == undefined || a[1].length == 0) {
        fraction = '00';
    } else if (a[1].length == 1) {
        fraction = a[1]+'0';
    } else {
        fraction = a[1].substr(0, 2);
    }
    return parseInt(decimal+fraction);
}


Examples:
trace (parseInput("512.56"));
-> 51256
trace (parseInput("512.568"));
-> 51256
trace (parseInput("512.4"));
-> 51240
trace (parseInput("512."));
-> 51200
trace (parseInput("512"));
-> 51200

 

Comments
First of all: why would you trust Flash to give the 'correct' result for the arithmetic in the first place? I don't know Flash, but chances are that the arithmetic is done using floating-point binary numbers with 53 bits of precision. If you don't trust C++ to handle the issue correctly, why would you trust Flash? It's still the same binary floating-point format, isn't it?

Second: see this page:
http://www.adobe.com/support/flash/action_scripts/actionscript_dictionary/actionscript_dictionary621.html
Especially the part about octal.
parseInt('075') would give 61, not 75, wouldn't it?

Oh, my goodness... I keep seeing the same stupid errors over and over again!

  Posted by Robert Lozyniak, on Friday, 10 August 2007 at 12:31


 1 
Page 1 of 1 ( 1 comments )
©2005 MosCom

You are not authorized to leave comments - please login.