Weird substraction result

[DavidM29]

Hello,

I'm asking my software to make a substraction and print the result in a Text and I found a situation where the result was weird. I thought it was because I had something wrong in my program so I tried the same calculation on a simple program and I get the same result.

Here is the problem :
I'm trying to make this calculation : 25-0.9-0.9 which should result as 23.2
But what I'm having is : 23.200000000000003

Here is the code I run :

main.qml

import QtQuick 2.9
import QtQuick.Window 2.2

Window {
    visible: true
    width: 640
    height: 480
    title: qsTr("Hello World")


    Text{
        id: value
        text : qsTr((25-0.9-0.9).toString())
        anchors.centerIn: parent
    }
}

Does anybody has any idea why it behave that way ?
I already found a way to correct this by using the toFixed(1) method but I wonder why I do have this behaviour. I've never seen a computer being wrong on a calculation so I believe I have done something wrong.

[VRonin]

Welcome to the magic world Floating-point arithmetic.

I've never seen a computer being wrong on a calculation

You are in for a rough ride!

[J.Hilk]

@VRonin best reply ever 😂👌

@DavidM29 you should read this wikipedia article, probably the most (in)famous case of a computer being wrong in calculations
https://en.m.wikipedia.org/wiki/Pentium_FDIV_bug

But for the behaviour you ecounterd, read up on floating point numbers and precision
https://en.m.wikipedia.org/wiki/Floating-point_arithmetic

[Cyrille de Brebisson]

Hello,

I will try to make it short and easy to understand...

Let us assume that you are doing math by had, with 3 digits of precision and execute the following operation:
1 - (1/3 * 3) = 1-(0.3333*3) = 1-0.999 = 0.001
You know that mathematically you should get 0, however, when YOU do the calculations, you get 0.001!

Well, computers have similar issues, however, since they use base 2 instead of base 10, the issues are to be found in different operations than the ones that you are used to. Hence the "stupefaction".

In your case, some of your numbers (0.9) do not have an exact representation in base 2, so you get rounding (down) on these numbers and end up with the result that you see.

Now, you need to be VERY carefull, because, in addition to this, the computer will often "hide" this to you by "rounding" the number (display 23.2 while in fact it has 23.20000...002 in memory). Then you THINK that you have the correct number, but if you perform another operation (like do an == test with 23.2), you do not get what you think!

Best advise, never use floats unless you HAVE to.
Never do == or != operations on floats (do abs(value1-value2)<small number)

Cyrille

[DavidM29]

Thank you all your answers are useful. I understand better now. Did not think about this.

[kkoehne]

For those interested in more details, I can recommend reading "What Every Computer Scientist Should Know About Floating-Point Arithmetic", see e.g. https://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html . It's a classic.