[Solved] Precision and converting float variables into strings

  • @
    float start = 92.195781;
    QString show = QString::number(start, 'g', 5);
    The above code example has a precision of 5 and outputs 92.196 (3 decimal places).
    Let's say start changes to 101.392093 - the output now becomes 101.39 (2 decimal places).
    And then start changes to 6.928321 - the output is 6.9283 (4 decimal places).
    I want the output to show 2 decimal places regardless of how big the number is. Is there an easy way to do this?

  • Probaly you have problem with the "g" parameter.
    Because: "For the 'g' and 'G' formats, the precision represents the maximum number of significant digits (trailing zeroes are omitted).". I think, you have to use the "e" or "E" or "f" paramater, because: "For the 'e', 'E', and 'f' formats, the precision represents the number of digits after the decimal point.".

  • Calculate the magitude of your number, and adjust the value of precision accordingly.

    The maginitude here is a basically the value of n in 10^n, like you use for scientific notation. In your examples, that would be 1, 2 and 0 repectively: the largest value of n where the result of 10^n is smaller than the number you working with. In your case, it seems you want to precision to be n+2.

    There are several ways to calculate the magnitude. Mathematically, it is simply 10Log(x), but depending on the range you are interested in and the performance you need, a simple loop dividing by 10 or even a big if/else if construct or something like that may be enough for your needs.

Log in to reply

Looks like your connection to Qt Forum was lost, please wait while we try to reconnect.