Float and QString: Strange problem

J.Hilk

Theres also qRound to add my 2 cents :)

fcarney

@kshegunov said in Float and QString: Strange problem:

Long += 0.0000005;

Doesn't that just move the problem though? If it was close to the rounding, adding to the value could put it right at the rounding location. Somehow the value needs to be rounded/truncated to less precision than the display precision or the problem will persist.

JonB

@fcarney

If it was close to the rounding, adding to the value could put it right at the rounding location

I don't get what you mean by that. So far as I am aware, like @kshegunov I believe that will always mean that truncating the result after the addition at 6 decimal places will give the correctly-rounded result.

fcarney

@JonB said in Float and QString: Strange problem:

I don't get what you mean by that.

Here is a value in which the value output from the function will produce grade 60 min 60:

60.99999999999999

Yes, adding to that value will produce 61 instead. However if I have this value:

60.99999949999999

adding 0.0000005 will result in 60.99999999999999

The problem is just moved to a different location.

JonB

@fcarney
Somewhere we're talking at cross-purposes.

60.99999949999999 truncate at 6 dps => 60.999999 => correct
60.99999959999999 truncate at 6 dps => 60.999999 => wrong (assuming it's supposed to round to nearest 6 dps)

OTOH :
60.99999949999999 + 0.0000005 = 60.99999999999999 truncate at 6 dps => 60.999999 => correct
60.99999959999999 + 0.0000005 = 61.00000099999999 truncate at 6 dps => 61.000000 => correct too (assuming it's supposed to round to nearest 6 dps)

fcarney

@JonB said in Float and QString: Strange problem:

truncate at 6 dps

That is the issue right there:

    double testvalue = 0.99999999999999;
    qInfo() << QString::number(testvalue,'f',14);
    qInfo() << QString::number(testvalue,'f',12);
    qInfo() << QString::number(testvalue,'f',7);
    qInfo() << QString::number(testvalue,'f',6);

Result:

"0.99999999999999"
"1.000000000000"
"1.0000000"
"1.000000"

The QString::number is not just truncating, but rounding as well.

kshegunov

@fcarney said in Float and QString: Strange problem:

The QString::number is not just truncating, but rounding as well.

Right, meaning you don't need to do anything when using QString::number, right? :)

Here is a value in which the value output from the function will produce grade 60 min 60:

The value is exactly correct. What you mean, I presume, is that when you display it with QString it gets rounded. You could tryo to change the rounding mode and use asprintf instead. Something like this:

const int rmode = fegetround( );
fesetround(FE_TOWARDZERO);
QString result = QString::asprintf("%.6f", &minutes);
fesetround(rmode);

fcarney

@kshegunov said in Float and QString: Strange problem:

Right, meaning you don't need to do anything when using QString::number, right? :)

No, because QString::number is producing:

"long calculated: -61.0000000 grade_long: 60.0000000 min_long: 60.0000000"
instead of:
"long calculated: -61.0000000 grade_long: 60.0000000 min_long: 59.9999999"

kshegunov

@fcarney said in Float and QString: Strange problem:

No, because QString::number is producing

Edited the post.

fcarney

@kshegunov said in Float and QString: Strange problem:

const int rmode = fegetround( );
fesetround(FE_TOWARDZERO);
QString result = QString::asprintf("%.6f", &minutes);
fesetround(rmode);

I was wondering if there was a way to do that. I noticed sprintf did the same thing as number did. So all I could think of was the math route or doing post string output:

    QString tstr = QString::number(testvalue,'f',16);
    QStringList tstrs = tstr.split('.');
    tstrs[1].resize(7);
    QString tstr2 = tstrs[0] + '.' + tstrs[1];
    qInfo() << tstr;
    qInfo() << tstr2;

I think your way of changing the rounding behavior is better though.

kshegunov

Yeah, but there are two problems with it: It can be finicky and secondly you can't load it directly into QString (turns out). So you need to do something like this:

#include <fenv.h>
#include <cmath>

#include <QString>

int main(int, char **)
{

    double coordinate = 60.99999999999999;

    int degree = coordinate;
    coordinate = (coordinate - degree) * 60;

    char minuteString[100] = {0};

    const int rmode = fegetround();
    fesetround(FE_TOWARDZERO);
    sprintf(minuteString, "%.6f", coordinate);
    fesetround(rmode);

    QString minutes = QString::fromLatin1(minuteString);

    return 0;
}

Another option would be to do something like this:

int degree = coordinate;
coordinate = (coordinate - degree) * 60;

if (int(coordinate) >= 60)
    coordinate -= 0.0000005; // Round down

QString minute = QString::number(coordinate, 'f', 6);

Or better yet:

coordinate += 0.0000005 / 60; // Round the minutes before doing whatever it is.

int degree = coordinate;
coordinate = (coordinate - degree) * 60;

QString minute = QString::number(coordinate, 'f', 6);

fcarney

@kshegunov said in Float and QString: Strange problem:

coordinate += 0.0000005 / 60;

Isn't this just adding an offset though? Moving the problem somewhere else? If its not then please explain how it works.

Edit:
This solution makes a lot of sense though:

if (int(coordinate) >= 60)
    coordinate -= 0.0000005; // Round down

kshegunov

@fcarney said in Float and QString: Strange problem:

Isn't this just adding an offset though? Moving the problem somewhere else? If its not then please explain how it works.

No, it's not moving it somewhere else, but my example is incomplete. Sorry about that.

coordinate += 0.0000005 / 60; // Round the minutes before doing whatever it is.

int degree = coordinate;
coordinate = (coordinate - degree) * 60 - 0.0000005; //< This last part was missed, sorry about that.

QString minute = QString::number(coordinate, 'f', 6);