C++ routine - Rounding error and unexpected result.

[MortyMars]

Hi all,

I'm creating a little routine to convert decimal degrees into DMS degrees and I'm having a problem with the rounding during the calculation, which is causing a result that is very far from reality.
My reasoning, which is probably not very elegant, is nevertheless correct because when I apply it in a calculation in Excel, I get the expected result.
In my transposition to Qt C++, I get the right results for degrees and minutes but not for seconds and hundredths.

Here's the result in console :

And here is the code extract :

bool ok;
double dblLatDec = strLatDec.mid(car1,strLatDec.length()-car1).toFloat(&ok);

// Traitement des degrés
int D = floorl(dblLatDec);
if (D < 10) strLatDMS.append("0"+ std::to_string(D));
else strLatDMS.append(std::to_string(D));

// Traitement des minutes
int M = floorl((dblLatDec - D) * 60);
if (M < 10) strLatDMS.append("0"+ std::to_string(M));
else strLatDMS.append(std::to_string(M));

// Traitement des secondes et centièmes
int S = floorl((dblLatDec - (D + M/60)) * 3600 *100);
if (S < 1000) strLatDMS.append("0"+ std::to_string(S));
else strLatDMS.append(std::to_string(S));

txtMatrix.at(7).at(i)->setText(strLatDMS);

qDebug() << "strLatDec   = " << strLatDec;
qDebug() << "dblLatDec   = " << dblLatDec;
qDebug() << "Degrés      = " << D;
qDebug() << "Minutes     = " << M;
// qDebug() << "Recalcul D + M/60    = " << D + ((M/(60*10000)))*10000;
// qDebug() << "Recalcul secondes    = " << (dblLatDec - (D + (M/(60*100))*100)) * 3600;
qDebug() << "Sec et /100 = " << S;
qDebug() << "strLatDMS   = " << strLatDMS;

Where is my error?
Thanks for your help

[MortyMars]

Thank you @JonB ,

Your reminder to be a bit more consistent in terms of variable type led me to rework my code in this direction...
And when I did I realised that the problem was actually due to the use, as the routine progressed, of variables that had been wrongly transformed into integers, and that's what was distorting the results.
The new, fully functional code is as follows:

bool ok;
            float fltLatDec = strLatDec.mid(car1,strLatDec.length()-car1).toFloat(&ok);

            // Traitement des degrés
            int D = floor(fltLatDec);
            if (D < 10) strLatDMS.append("0"+ std::to_string(D));
            else strLatDMS.append(std::to_string(D));

            // Traitement des minutes
            int M = floor((fltLatDec - D) * 60);
            if (M < 10) strLatDMS.append("0"+ std::to_string(M));
            else strLatDMS.append(std::to_string(M));

            // Traitement des secondes et centièmes
            int S = floor((fltLatDec - (floor(fltLatDec) + floor((fltLatDec - D) * 60)/60)) * 3600 * 100);
            if (S < 1000) strLatDMS.append("0"+ std::to_string(S));
            else strLatDMS.append(std::to_string(S));

            txtMatrix.at(7).at(i)->setText(strLatDMS);

            qDebug() << "strLatDec   = " << strLatDec;
            qDebug() << "dblLatDec   = " << fltLatDec;
            qDebug() << "Degrés    D = " << D;
            qDebug() << "Minutes   M = " << M;
            qDebug() << "Recalcul D + M/60    = " << floor(fltLatDec) + floor((fltLatDec - D) * 60)/60;
            qDebug() << "Recalcul secondes    = " << (fltLatDec - (floor(fltLatDec) + floor((fltLatDec - D) * 60)/60)) * 3600 * 100;
            qDebug() << "Sec et /100 = " << S;
            qDebug() << "strLatDMS   = " << strLatDMS;

[JonB]

@MortyMars
I have not analyzed your code, but:

• floorl() takes a long double and returns a long double. Since you only pass it a double (from toFloat(), which only returns a float, let alone a double as per toDouble()) I don't see the point.
• I believe qDebug() on a floating point only produces 6 significant digits in the resulting string. Try something like QString QString::arg(double a, int fieldWidth = 0, char format = 'g', int precision = -1, QChar fillChar = u' ') const with a higher precision to get a better view of the number?

[MortyMars]

Thank you @JonB ,

Your reminder to be a bit more consistent in terms of variable type led me to rework my code in this direction...
And when I did I realised that the problem was actually due to the use, as the routine progressed, of variables that had been wrongly transformed into integers, and that's what was distorting the results.
The new, fully functional code is as follows:

bool ok;
            float fltLatDec = strLatDec.mid(car1,strLatDec.length()-car1).toFloat(&ok);

            // Traitement des degrés
            int D = floor(fltLatDec);
            if (D < 10) strLatDMS.append("0"+ std::to_string(D));
            else strLatDMS.append(std::to_string(D));

            // Traitement des minutes
            int M = floor((fltLatDec - D) * 60);
            if (M < 10) strLatDMS.append("0"+ std::to_string(M));
            else strLatDMS.append(std::to_string(M));

            // Traitement des secondes et centièmes
            int S = floor((fltLatDec - (floor(fltLatDec) + floor((fltLatDec - D) * 60)/60)) * 3600 * 100);
            if (S < 1000) strLatDMS.append("0"+ std::to_string(S));
            else strLatDMS.append(std::to_string(S));

            txtMatrix.at(7).at(i)->setText(strLatDMS);

            qDebug() << "strLatDec   = " << strLatDec;
            qDebug() << "dblLatDec   = " << fltLatDec;
            qDebug() << "Degrés    D = " << D;
            qDebug() << "Minutes   M = " << M;
            qDebug() << "Recalcul D + M/60    = " << floor(fltLatDec) + floor((fltLatDec - D) * 60)/60;
            qDebug() << "Recalcul secondes    = " << (fltLatDec - (floor(fltLatDec) + floor((fltLatDec - D) * 60)/60)) * 3600 * 100;
            qDebug() << "Sec et /100 = " << S;
            qDebug() << "strLatDMS   = " << strLatDMS;

[ChrisW67]

@MortyMars I know I am late to the party but this is the approach I take to minimie any floating point quirks (i.e. switch to integer-only operations as early as possible):

double dblLatDec = 49.123456789;

char hemisphere = dblLatDec >= 0.0 ? 'N' : 'S';
int work = std::fabs(std::round(dblLatDec * 360000)); // original in hundredths of arcseconds
int hundredths = work % 100;
work /= 100;
int seconds = work % 60;
work /= 60;
int minutes = work % 60;
int degrees = work / 60;

std::ostringstream s;
s.fill('0');
s << hemisphere;
s.width(2);        s << degrees;
s.width(2);        s << minutes;
s.width(2);        s << seconds ;
s << '.';
s.width(2);        s << hundredths;
std::string output = s.str();

// std::format() in C++20 is neater
std::string output2 = std::format("{}{:0>2d}{:0>2d}{:0>2d}.{:0>2d}\n", 
        hemisphere, degrees, minutes, seconds, hundredths);

// or Qt
QString result =  QString("%1%2%3%4.%5")
                        .arg(hemisphere)
                        .arg(degrees,    2, 10, QChar('0'))
                        .arg(minutes,    2, 10, QChar('0'))
                        .arg(seconds,    2, 10, QChar('0'))
                        .arg(hundredths, 2, 10, QChar('0'));

[MortyMars]

Hi @ChrisW67

I'm sorry I haven't replied to your message (was on holiday with the family...).

Thank you for your suggestion.
I'm going to keep the associated code in my tips ;-)