Unsolved Float and QString: Strange problem
-
@JonB said in Float and QString: Strange problem:
I don't get what you mean by that.
Here is a value in which the value output from the function will produce grade 60 min 60:
60.99999999999999
Yes, adding to that value will produce 61 instead. However if I have this value:
60.99999949999999 adding 0.0000005 will result in 60.99999999999999
The problem is just moved to a different location.
-
@fcarney
Somewhere we're talking at cross-purposes.60.99999949999999
truncate at 6 dps =>60.999999
=> correct
60.99999959999999
truncate at 6 dps =>60.999999
=> wrong (assuming it's supposed to round to nearest 6 dps)OTOH :
60.99999949999999 + 0.0000005 = 60.99999999999999
truncate at 6 dps =>60.999999
=> correct
60.99999959999999 + 0.0000005 = 61.00000099999999
truncate at 6 dps =>61.000000
=> correct too (assuming it's supposed to round to nearest 6 dps) -
@JonB said in Float and QString: Strange problem:
truncate at 6 dps
That is the issue right there:
double testvalue = 0.99999999999999; qInfo() << QString::number(testvalue,'f',14); qInfo() << QString::number(testvalue,'f',12); qInfo() << QString::number(testvalue,'f',7); qInfo() << QString::number(testvalue,'f',6);
Result:
"0.99999999999999" "1.000000000000" "1.0000000" "1.000000"
The QString::number is not just truncating, but rounding as well.
-
@fcarney said in Float and QString: Strange problem:
The QString::number is not just truncating, but rounding as well.
Right, meaning you don't need to do anything when using
QString::number
, right? :)Here is a value in which the value output from the function will produce grade 60 min 60:
The value is exactly correct. What you mean, I presume, is that when you display it with
QString
it gets rounded. You could tryo to change the rounding mode and useasprintf
instead. Something like this:const int rmode = fegetround( ); fesetround(FE_TOWARDZERO); QString result = QString::asprintf("%.6f", &minutes); fesetround(rmode);
-
@kshegunov said in Float and QString: Strange problem:
Right, meaning you don't need to do anything when using QString::number, right? :)
No, because QString::number is producing:
"long calculated: -61.0000000 grade_long: 60.0000000 min_long: 60.0000000" instead of: "long calculated: -61.0000000 grade_long: 60.0000000 min_long: 59.9999999"
-
@fcarney said in Float and QString: Strange problem:
No, because QString::number is producing
Edited the post.
-
@kshegunov said in Float and QString: Strange problem:
const int rmode = fegetround( );
fesetround(FE_TOWARDZERO);
QString result = QString::asprintf("%.6f", &minutes);
fesetround(rmode);I was wondering if there was a way to do that. I noticed sprintf did the same thing as number did. So all I could think of was the math route or doing post string output:
QString tstr = QString::number(testvalue,'f',16); QStringList tstrs = tstr.split('.'); tstrs[1].resize(7); QString tstr2 = tstrs[0] + '.' + tstrs[1]; qInfo() << tstr; qInfo() << tstr2;
I think your way of changing the rounding behavior is better though.
-
Yeah, but there are two problems with it: It can be finicky and secondly you can't load it directly into
QString
(turns out). So you need to do something like this:#include <fenv.h> #include <cmath> #include <QString> int main(int, char **) { double coordinate = 60.99999999999999; int degree = coordinate; coordinate = (coordinate - degree) * 60; char minuteString[100] = {0}; const int rmode = fegetround(); fesetround(FE_TOWARDZERO); sprintf(minuteString, "%.6f", coordinate); fesetround(rmode); QString minutes = QString::fromLatin1(minuteString); return 0; }
Another option would be to do something like this:
int degree = coordinate; coordinate = (coordinate - degree) * 60; if (int(coordinate) >= 60) coordinate -= 0.0000005; // Round down QString minute = QString::number(coordinate, 'f', 6);
Or better yet:
coordinate += 0.0000005 / 60; // Round the minutes before doing whatever it is. int degree = coordinate; coordinate = (coordinate - degree) * 60; QString minute = QString::number(coordinate, 'f', 6);
-
@kshegunov said in Float and QString: Strange problem:
coordinate += 0.0000005 / 60;
Isn't this just adding an offset though? Moving the problem somewhere else? If its not then please explain how it works.
Edit:
This solution makes a lot of sense though:if (int(coordinate) >= 60) coordinate -= 0.0000005; // Round down
-
@fcarney said in Float and QString: Strange problem:
Isn't this just adding an offset though? Moving the problem somewhere else? If its not then please explain how it works.
No, it's not moving it somewhere else, but my example is incomplete. Sorry about that.
coordinate += 0.0000005 / 60; // Round the minutes before doing whatever it is. int degree = coordinate; coordinate = (coordinate - degree) * 60 - 0.0000005; //< This last part was missed, sorry about that. QString minute = QString::number(coordinate, 'f', 6);