My brain hurts--It just doesn't add up

2007 May 31
tags: Code · ColdFusion
by Nick

I just spent a good part of the last 2 hours trying to debug a calculation problem on a fairly simple report. Basically, it lists "open-bills" from an e-commerce application. The logic sorts through records to find any orders where the Total minus the Payment plus any Credits does not equal zero and displays.

Well... the complaint was that orders were appearing on the open-bills report which didn't belong. I quickly verified this, however, something didn't quite make sense.

Although this isn't the actual code, the following calculation is consistently returning "3.5527136788E-15" instead of zero on ColdFusion version 7.02:

<cfscript>

Balance = 14; //account balance

Payment = 35.94; // payment amount

Credit = 21.94; // credit


// calculate result
Result = ((Balance - Payment) + Credit);


// Result value is "3.5527136788E-15"

writeoutput(Result);

</cfscript>

Changing the values slightly will produce the expected result... however, I noticed changing the Payment and Credit values to 32.95 and 21.95 will produce a negative "3.5527136788E-15".

I've been able to replicate this on 2 different installations of 7.02 so far... It would be great to see if anyone else had any ideas or input about this.

UPDATE

I'm not sure if I've identified a ColdFusion bug or not... however, I found that applying some kind of format (numberFormat() or dollarFormat()) to the Result will display the correct value.

3 Responses leave one →
  1. Nate
    Nate PERMALINK
    Jun 13, 2007 at 9:19 PM

    The trick is, you need to hack into the mainframe and override the fire walls. Didn't you watch Johnny Mnemonic?

  1. Jeff Douglas
    Jeff Douglas PERMALINK
    Apr 4, 2009 at 8:23 AM

    Nick, I had the same problem last week when writing a purchasing requisition application in Java. Wonder if it's related to the JVM that CF uses? I had to implement my own rounding method to ensure the values were correct:

    private double roundDouble(double d) {
    java.math.BigDecimal roundValue = new
    java.math.BigDecimal(d).setScale(2,java.math.BigDecimal.ROUND_HALF_UP);
       return new Double(roundValue.doubleValue());
    }

  1. Nick
    Nick PERMALINK *
    Apr 5, 2009 at 9:20 PM

    I just tested this same code on Railo 3.0.4 and it returns 0, yet ColdFusion 8.01 is still "3.5527136788E-015"

Leave a Reply