|
Math.floor( parseFloat("512.56") * 100 ) results in 51255, while I (and my client) was expecting 51256. This is very annoying when your coding an online tax calculation program in Flash. So I tried to pin down the problem. What happens in the program. The front-end is written in Flash, the back-end is a webservice written in C++. To avoid roundoff errors, the webservice works with integers only. All amounts are multiplied by 100, to be able to work with cents and still use integers. This is common practice in financial software. Since € 110.125 is an invalid amount, (1 cent is the lowest valid amount), we decided to chop off the digits entered by the user after the 2nd digit after the decimal point. So the user enters an amount in a TextField. This is a string. I convert it to a Float with the parseFloat() function. Then I multiply the Float by 100 and round off to lowest Integer. "512.5677" would result in 51256, thus ignoring digits behind the second digit after the decimal point. "512.56" Should result in 51256 too, but it doens't.
Try this in Flash: trace(Math.floor(100*parseFloat("512.56"))) -> 51255 Let's break this down: trace (parseFloat("512.56")); -> 512.56 trace (parseFloat("512.56") * 100); -> 51256 trace (Math.floor(parseFloat("512.56") * 100)); -> 51255 Keep in mind here that we're not rounding off 512.56, but 51256. So I would guess that the number 51256 generated by the function (parseFloat("512.56") * 100) is internally represented as 51255.99999999 or so, which is displayed as 51256. Let's try another one: trace (Number("280.03")); > 280.03 trace (Number("280.03") * 100); > 28003 trace (Math.floor(Number("280.03") * 100)); > 28002 Now you would expect that in Math.round(28003) is the same as Math.floor(28003), but it's not: trace (Math.round(Number("280.03") * 100)); > 28003 It took some experimenting before I found out that it actually is a roundoff error. At first I thought that the parseFloat() function was the problem. But without the parseFloat() function we get the same results: trace (Math.floor(512.56 * 100)); > 51255 trace (Math.floor(51256.0)); > 51256 trace (Math.floor(51256)); > 51256 So (512.56 * 100) doesn't equal 51256. Macromedia technote 13989 will tell you that roundoff errors are a fact of life. I can live with that, but I wasn't expecting them to show up in the 2nd digit after the decimal point already. Lessons learned: never mix currencies and floating point numbers. Use a function like this to parse user input: function parseInput(inputText:String):Number { var a:Array = inputText.split('.'); var fraction; var decimal = a[0]; if (a[1] == undefined || a[1].length == 0) { fraction = '00'; } else if (a[1].length == 1) { fraction = a[1]+'0'; } else { fraction = a[1].substr(0, 2); } return parseInt(decimal+fraction); } Examples: trace (parseInput("512.56")); -> 51256 trace (parseInput("512.568")); -> 51256 trace (parseInput("512.4")); -> 51240 trace (parseInput("512.")); -> 51200 trace (parseInput("512")); -> 51200 |