Calculations with real numbers - wrong results

D_Tec

Hi,

I just ran into another issue I am not able to fix. In my app I do some simple calculation with real numbers and sometime the results are incorrect. I cannot trigger it, it happens randomly like:
adding 0.2 to 20.775 = 20.974999999999998

The 0.2 is a number I extract from a json import.
The 20.775 is a property from a QML component
The 20.974999999999998 is the same poperty after I added 0.2

The calculation is done

warehouse.total += jsonData.cost

In most cases this is not an issue but sometimes I see this odd results.

JonB

@D_Tec
This is not a QML issue, it is an issue with the representation of floating point numbers. Use qFuzzyCompare to compare floating point numbers for equality.

JKSH

It is impossible to store 0.2 exactly as a floating point value.

This website has a good explanation: https://floating-point-gui.de/
You can experiment with this: https://www.h-schmidt.net/FloatConverter/IEEE754.html (it only processes 32-bit numbers, but the same concepts apply to 64-bit numbers)

D_Tec

Thanks for the hints. I thought it was a rounding issue but I was not aware of the situation. I will try to round prior displaying any float.

GrecKo

Rounding before won't help you, use toFixed instead

SimonSchroeder

@D_Tec said in Calculations with real numbers - wrong results:

warehouse.total += jsonData.cost

Based on your variable names it looks like you want to do computations with a monetary unit. It is advisable that you do not use floating point for this. If you add small amounts to your warehouse.total you will eventually have true rounding errors. Instead, you should do all computations with integers. If you decide to have at most 3 digits after the point, you would represent your numbers like this: 200 (=0.2) and 20775 (=20.775). Now, you don't have any loss during computations. When you want to show the number you just need to add the point in the right place (making sure you have enough leading zeros).