Float and QString: Strange problem

fcarney

I can reproduce:

    double_string(30.9999999999999999,60.9999999999999999);
    double_string(30.99999999999999,60.99999999999999);
    double_string(30.999999999999,60.999999999999);

"long calculated: 31.0000000 grade_long: 31.0000000 min_long: 0.0000000"
"lat calculated: 61.0000000 grade_lat: 61.0000000 min_lat: 0.0000000"
"long calculated: 31.0000000 grade_long: 30.0000000 min_long: 60.0000000"
"lat calculated: 61.0000000 grade_lat: 60.0000000 min_lat: 60.0000000"
"long calculated: 31.0000000 grade_long: 30.0000000 min_long: 60.0000000"
"lat calculated: 61.0000000 grade_lat: 60.0000000 min_lat: 60.0000000"

You will need to round the number to a lesser precision than the displayed number of 7.

    Long = double(int(Long*1000000)/1000000.0);
    Lat = double(int(Lat*1000000)/1000000.0);

Result:

"lat calculated: 61.0000000 grade_lat: 61.0000000 min_lat: 0.0000000"
"long calculated: 30.9999990 grade_long: 30.0000000 min_long: 59.9999400"
"lat calculated: 60.9999990 grade_lat: 60.0000000 min_lat: 59.9999400"
"long calculated: 30.9999990 grade_long: 30.0000000 min_long: 59.9999400"
"lat calculated: 60.9999990 grade_lat: 60.0000000 min_lat: 59.9999400"

kshegunov

@Marce said in Float and QString: Strange problem:

I need to convert it to the format gg mm.mmmmmmm

Which I gave you the solution for. Truncate once to get the degrees and then subtract the whole part, multiply by 60 to get the minutes and that's it.

If I initialize the variable with 61.0 ° the result is correct so I assume that the variable is stored in 60.9999999999999999999999999

You should realize first that flotaing point numbers are approximations. No real number can be represented exactly, and to add insult to injury not every rational number can be represented exactly. Take the simplest - 1/3, how do you suppose is this represented. Firstly, even in decimal it's 0.33(3), secondly the floating point numbers in computers are powers of 2. There are no common divisors of 2 and 3, so you can get an approximation of 0.33 only, much less something that's infinitely long sequence of 3s. Exactly representable numbers are 1/2 (0.5), 1/4 (0.25), 1/8 (0.125), so any number that can be stored is a sum of such things, nothing more, nothing less.

@fcarney said in Float and QString: Strange problem:

You will need to round the number to a lesser precision than the displayed number of 7.

Long = double(int(Long*1000000)/1000000.0);
Lat = double(int(Lat*1000000)/1000000.0);

Before doing a multiplication, truncation and division, which is terribly inefficient, have you considered making a simple addition?

Long += 0.0000005;

Display up to 6 digits and you get your rounding correctly.

J.Hilk

Theres also qRound to add my 2 cents :)

fcarney

@kshegunov said in Float and QString: Strange problem:

Long += 0.0000005;

Doesn't that just move the problem though? If it was close to the rounding, adding to the value could put it right at the rounding location. Somehow the value needs to be rounded/truncated to less precision than the display precision or the problem will persist.

JonB

@fcarney

If it was close to the rounding, adding to the value could put it right at the rounding location

I don't get what you mean by that. So far as I am aware, like @kshegunov I believe that will always mean that truncating the result after the addition at 6 decimal places will give the correctly-rounded result.

fcarney

@JonB said in Float and QString: Strange problem:

I don't get what you mean by that.

Here is a value in which the value output from the function will produce grade 60 min 60:

60.99999999999999

Yes, adding to that value will produce 61 instead. However if I have this value:

60.99999949999999

adding 0.0000005 will result in 60.99999999999999

The problem is just moved to a different location.

JonB

@fcarney
Somewhere we're talking at cross-purposes.

60.99999949999999 truncate at 6 dps => 60.999999 => correct
60.99999959999999 truncate at 6 dps => 60.999999 => wrong (assuming it's supposed to round to nearest 6 dps)

OTOH :
60.99999949999999 + 0.0000005 = 60.99999999999999 truncate at 6 dps => 60.999999 => correct
60.99999959999999 + 0.0000005 = 61.00000099999999 truncate at 6 dps => 61.000000 => correct too (assuming it's supposed to round to nearest 6 dps)

fcarney

@JonB said in Float and QString: Strange problem:

truncate at 6 dps

That is the issue right there:

    double testvalue = 0.99999999999999;
    qInfo() << QString::number(testvalue,'f',14);
    qInfo() << QString::number(testvalue,'f',12);
    qInfo() << QString::number(testvalue,'f',7);
    qInfo() << QString::number(testvalue,'f',6);

Result:

"0.99999999999999"
"1.000000000000"
"1.0000000"
"1.000000"

The QString::number is not just truncating, but rounding as well.

kshegunov

@fcarney said in Float and QString: Strange problem:

The QString::number is not just truncating, but rounding as well.

Right, meaning you don't need to do anything when using QString::number, right? :)

Here is a value in which the value output from the function will produce grade 60 min 60:

The value is exactly correct. What you mean, I presume, is that when you display it with QString it gets rounded. You could tryo to change the rounding mode and use asprintf instead. Something like this:

const int rmode = fegetround( );
fesetround(FE_TOWARDZERO);
QString result = QString::asprintf("%.6f", &minutes);
fesetround(rmode);

fcarney

@kshegunov said in Float and QString: Strange problem:

Right, meaning you don't need to do anything when using QString::number, right? :)

No, because QString::number is producing:

"long calculated: -61.0000000 grade_long: 60.0000000 min_long: 60.0000000"
instead of:
"long calculated: -61.0000000 grade_long: 60.0000000 min_long: 59.9999999"

kshegunov

@fcarney said in Float and QString: Strange problem:

No, because QString::number is producing

Edited the post.

fcarney

@kshegunov said in Float and QString: Strange problem:

const int rmode = fegetround( );
fesetround(FE_TOWARDZERO);
QString result = QString::asprintf("%.6f", &minutes);
fesetround(rmode);

I was wondering if there was a way to do that. I noticed sprintf did the same thing as number did. So all I could think of was the math route or doing post string output:

    QString tstr = QString::number(testvalue,'f',16);
    QStringList tstrs = tstr.split('.');
    tstrs[1].resize(7);
    QString tstr2 = tstrs[0] + '.' + tstrs[1];
    qInfo() << tstr;
    qInfo() << tstr2;

I think your way of changing the rounding behavior is better though.

kshegunov

Yeah, but there are two problems with it: It can be finicky and secondly you can't load it directly into QString (turns out). So you need to do something like this:

#include <fenv.h>
#include <cmath>

#include <QString>

int main(int, char **)
{

    double coordinate = 60.99999999999999;

    int degree = coordinate;
    coordinate = (coordinate - degree) * 60;

    char minuteString[100] = {0};

    const int rmode = fegetround();
    fesetround(FE_TOWARDZERO);
    sprintf(minuteString, "%.6f", coordinate);
    fesetround(rmode);

    QString minutes = QString::fromLatin1(minuteString);

    return 0;
}

Another option would be to do something like this:

int degree = coordinate;
coordinate = (coordinate - degree) * 60;

if (int(coordinate) >= 60)
    coordinate -= 0.0000005; // Round down

QString minute = QString::number(coordinate, 'f', 6);

Or better yet:

coordinate += 0.0000005 / 60; // Round the minutes before doing whatever it is.

int degree = coordinate;
coordinate = (coordinate - degree) * 60;

QString minute = QString::number(coordinate, 'f', 6);

fcarney

@kshegunov said in Float and QString: Strange problem:

coordinate += 0.0000005 / 60;

Isn't this just adding an offset though? Moving the problem somewhere else? If its not then please explain how it works.

Edit:
This solution makes a lot of sense though:

if (int(coordinate) >= 60)
    coordinate -= 0.0000005; // Round down

kshegunov

@fcarney said in Float and QString: Strange problem:

Isn't this just adding an offset though? Moving the problem somewhere else? If its not then please explain how it works.

No, it's not moving it somewhere else, but my example is incomplete. Sorry about that.

coordinate += 0.0000005 / 60; // Round the minutes before doing whatever it is.

int degree = coordinate;
coordinate = (coordinate - degree) * 60 - 0.0000005; //< This last part was missed, sorry about that.

QString minute = QString::number(coordinate, 'f', 6);