How to avoid floating point problems?

mhn2

Hi!

In my application, I get a floating point number from the user, store it in the memory, and then later I will save it as a number in a json file.

The problem is somewhere in this process the floating part gets changed. (For example when the input is 13.23 the number written in the json file is 13.229999542236328)

I understand where is the problem coming from (computers saving numbers in base 2 but the input being in base 10 and etc.)

Luckily when I read the json file back in my own application, the problem cancels itself and it will show 13.23 to the user. But this json file is meant to be used in other applications as well and I can't be sure that it won't also be a problem there.

Is there a way, for me to store the number in the memory and to the json file, and be sure that the number written in the json file is the exact same number in base 10 as the number that was the input?

(I am using a QLineEdit to get the number, but I can't save the number as a string in the final json)

JonB

@mhn2
I think not, because presumably 13.229999542236328 is as close as it gets to storing 13.23 as a float. [I am assuming your code does not do any unnecessary conversions.] You only have two choices:

Store as a float (JSON calls it and treats it as "a number"), and accept whatever representation in the JSON.
Store it as a string, where you can control the number of digits.

P.S.
You can read about this by searching for something like floating point numbers in JSON files. You are interested in how they get serialized, e.g. Why would you use a string in JSON to represent a decimal number.

Axel Spoerl

@mhn2
Hi!
Please share a simple reproducing code snippet, to show how the user input ends up in the JSON file. Without that, it's hard to guess where the problem is.
Cheers
Axel

mhn2

@JonB
I am fine with storing the number as a string. But how can I tell QJsonDocument to use my string as a number, without converting it to a double first? Should I use a custom json parser instead?

@Axel-Spoerl
For sake of avoiding confusion I'm only including the related parts of the code and renaming variable names to their type.

// Some where in the dialogs constructor
ui->lineEdit->setValidator(new QDoubleValidator());

// Some where in a slot
float value = m_ui->lineEdit->text().toFloat();

// Some where in the saving json process:
QJsonObject json;
json["r"] = value;
// ...
QJsonDocument document(json);
QByteArray data = document.toJson();

These are currently the only places the value gets read and used.

When the value of the lineEdit is 13.23, the json in the byte array says:

{
  "r": 13.229999542236328,
  // ...
}

JonB

@mhn2 said in How to avoid floating point problems?:

But how can I tell QJsonDocument to use my string as a number

You cannot! Either you store a number, in which case JSON formats the output, or you store it as a string, in which case you do, but then the reading program must be prepared to accept a string and convert to number.

Your code looks as good as it gets. You have the two choices I listed in my first reply.

J.Hilk

@mhn2 try

json["r"] = QString::number(value, 'f', 2);

JonB

@J-Hilk QString::number() returns a string, so it will be stored as a string, which the OP says he does not want....

J.Hilk

@mhn2 said in How to avoid floating point problems?:

@JonB
I am fine with storing the number as a string

I read this as storing it in the json file as string

JonB

@J-Hilk
The whole point of this conversation is that the OP knows he can store "13.23" so it outputs like that to the file. Which will be read as a string. Which may not be right for the receiving program. He wants it to output 13.23 to the file. Which as he has discovered is not going to happen....

mhn2

Well it seems like I either have to accept the floating point problem or use strings to store the number in json.

I will discuss this with my other team members and settle on one of these.

Thanks to everyone (especially JonB) for helping!

JonB

@mhn2
My inclination is to store it as a number, and accept the precision. It should not be your job to convert to strings when it's a number. I think you can assume that if (Qt) JSON outputs 13.23 this way then any other JSON reader will be convert the 13.2299... back in just the same way. I imagine this is anticipated/standard in JSON data numeric transfer, else everyone else would come up against this issue.

J.Hilk

@JonB I seriously fail to conclude that from what was written so far.

Anyway there's still the option of post processing.

Save everything as a String, Mark/Store the stringyfied numbers in a list. parse the previously written json file into memory, remove "from marked string/numbers, save the file back.

mhn2

@JonB
Yes it is very likely that the application that will read the json will have the same precision problems, however it might be better to not risk it. Whether or not I should take this risk is something I'm going to have to discuss.

mhn2

@J-Hilk
This process can be complicated (For example if two different parts of the json are using the same property name, or if custom user data is getting stored as string)

I think it would be easier to use a custom parser instead than to do this.

JonB

@mhn2
Certainly you should discuss. But I believe you are laboring under a misunderstanding/assumption here. You are thinking if you can get the number 13.23 or the string "13.23" into the file that will make it so when the reader fetches it it will convert those digits which make up a string (even if it's stored as a 13.23 number the JSON will still have to convert string to number when it reads it) to exactly the binary, floating point representation of 13.23. But it won't, because (I believe) you are showing that the IEEE binary representation of 13.23 simply is 13.229999542236328, and that's as close as you going to get whatever you do.