Important: Please read the Qt Code of Conduct - https://forum.qt.io/topic/113070/qt-code-of-conduct

Calculations with real numbers - wrong results



  • Hi,

    I just ran into another issue I am not able to fix. In my app I do some simple calculation with real numbers and sometime the results are incorrect. I cannot trigger it, it happens randomly like:
    adding 0.2 to 20.775 = 20.974999999999998

    The 0.2 is a number I extract from a json import.
    The 20.775 is a property from a QML component
    The 20.974999999999998 is the same poperty after I added 0.2

    The calculation is done

    warehouse.total += jsonData.cost
    

    In most cases this is not an issue but sometimes I see this odd results.



  • @D_Tec
    This is not a QML issue, it is an issue with the representation of floating point numbers. Use qFuzzyCompare to compare floating point numbers for equality.


  • Moderators

    It is impossible to store 0.2 exactly as a floating point value.



  • Thanks for the hints. I thought it was a rounding issue but I was not aware of the situation. I will try to round prior displaying any float.


  • Qt Champions 2018

    Rounding before won't help you, use toFixed instead



  • @D_Tec said in Calculations with real numbers - wrong results:

    warehouse.total += jsonData.cost

    Based on your variable names it looks like you want to do computations with a monetary unit. It is advisable that you do not use floating point for this. If you add small amounts to your warehouse.total you will eventually have true rounding errors. Instead, you should do all computations with integers. If you decide to have at most 3 digits after the point, you would represent your numbers like this: 200 (=0.2) and 20775 (=20.775). Now, you don't have any loss during computations. When you want to show the number you just need to add the point in the right place (making sure you have enough leading zeros).


Log in to reply