Collectives™ on Stack Overflow

Find centralized, trusted content and collaborate around the technologies you use most.

Teams

Q&A for work

Connect and share knowledge within a single location that is structured and easy to search.

In my project I read from a json file with QJsonDocument::fromJson() . This works great, however when I try to write the QJsonDocument back to file with toJson() some of the doubles have messed up precision.

For example, calling toJson() on a document with a QJsonValue with a double value of 0.15 will save to file as 0.14999999999999999 . I do not want this.

This is because the Qt source file qjsonwriter.cpp at line 126 (Qt 5.6.2) reads:

json += QByteArray::number(d, 'g', std::numeric_limits<double>::digits10 + 2); // ::digits10 is 15
That +2 at the end there is messing me up. If this same call to QByteArray::number() instead has a precision of 15 (instead of 17), the result is exactly as I need... 0.15.
I understand how the format of floating point precision causes the double to be limited in what it can represent. But if I limit the precision to 15 instead of 17, this has the effect of matching the input double precision, which I want.
How can I get around this?
Obviously... I could write my own Json parser, but that's last resort. And obviously I could edit the Qt source code, however my software is already deployed with the Qt5Core.dll included in everyone's install directory, and my updater is not designed to update any dll's. So I cannot edit the Qt source code.
Fingers crossed someone has a magic fix for this :)
                Is it possible to save numbers as hex float? (%a, see the table here for more explanation)
– Henri Menke
                Jul 10, 2018 at 5:22
                I do not have control over what QJsonDocument does after I call toJson(). If I did, I could surely save it in whatever format I want. At least, thats what im trying to confirm here.
– mrg95
                Jul 10, 2018 at 5:24
                Correct, but as I stated, I do not have control over the Qt source. That line of code is not mine. I showed it because it is the crux of the issue. Precision of 17 is hard coded.
– mrg95
                Jul 10, 2018 at 5:28
                My question was never how to write the double, it was how to get QJson to do what I wanted.
– mrg95
                Jul 11, 2018 at 21:33
This request doesn't make much sense. A double doesn't carry any information about its precision - it only carries a value. 0.15, 0.1500 and 0.14999999999999999 are the exact same double value, and the JSON writer has no way to know how it was read from the file in first place (if it was read from a file at all). 
In general you cannot ask for maximum 15 digits of precision as you propose, as, depending from the particular value, up to 17 are required for a precise double->text->double roundtrip, so you would write incorrectly rounded values. What some JSON writers do however is to write numbers with the minimum number of decimals required to read the same double back. This is far from trivial to do numerically correctly unless you do - as many do - a loop from 15 to 17, write the number with such precision, parse it back and see if it comes back as the exact same double value. While this generates "nicer" (and smaller) output, it's more work and slows down the JSON write, so that's why probably Qt doesn't do this.
Still, you can write your own JSON write code and have this feature, for a simple recursive implementation I expect ~15 lines of code. 
That being said, again, if you want to precisely match your input this won't save you - as it's simply impossible.
                To rephrase, the json file I'm loading has been generated from other software that writes doubles with a lower precision than 17. I need control over how QJson write doubles so it matches what the output would be from the other software.  My solution was to write my own parser.
– mrg95
                Jul 11, 2018 at 2:13
                If you write all the values with precision 15 you are going to lose precision, as it's not enough for many values. I'm almost sure that, if that JSON comes from any of the "mainstream" JSON writers, it writes down numbers with the minimum precision to represent the value exactly, not fixed 15. Also, as I said above, from a machine standpoint it makes no difference if you write down the number with more precision than needed - it's effectively the same number. Finally, a parser is some code that converts textual representation to machine representation, while what you wrote is the opposite.
– Matteo Italia
                Jul 11, 2018 at 5:46
                How does one determine what the "exact" value is then? Clearly, in my situation, the number I was trying to write was "exactly" 0.14999999999999999. The value changes depending on the precision I give it. So how does one determine what the "true" value is supposed to be in order to compare it?
– mrg95
                Jul 11, 2018 at 5:53
                @mrg95: the "true" value is what the 8 bytes of the double contain; OTOH, there are many decimal representations that will yield exactly the same double value when parsed. 0.14999999999999999, 0.15, 0.150, 0.14999999999999999445 all parse back to the same value. What you want to output is a matter of preference. Do you want to optimize speed? Just write with 17 decimals, it'll always be right. For human readability? Use the loop I said above - try %0.15g, if it parses back as the same double you are ok; otherwise try %0.16g, otherwise %0.17g.
– Matteo Italia
                Jul 11, 2018 at 7:01
                Ah, I understand what you're saying now. Sorry I didn't catch that earlier. I'll implement this into my pars.... um.... writer ;)
– mrg95
                Jul 11, 2018 at 7:34
I just encountered this as well.  Rather than replace an entire Qt JSON implementation with a third party library (or roll my own!), however, I kludged a solution...
My full code base related to this is too extensive and elaborate to post and explain here.  But the gist of the solution to this point is simple enough.
First, I use a QVariantMap (or QVariantHash) to collect my data, and then convert that to json via the built-in QJsonObject::fromVariantMap or QJsonDocument::fromVariant functions.  To control the serialization, I define a class called DataFormatOptions which has a decimalPrecision member (and sets up easy expansion to other such formatting options..) and then I call a function called toMagicVar to create "magic variants" for my data structure to be converted to json bytes.  To control for the number format / precision toMagicVar converts doubles and floats to strings that are in the desired format, and surrounds the string value with some "magic bytes".  The way my actual code is written, one can easily do this on any "level" of the map/hash I'm building / formatting via recursive processing, but I've omitted those details...
const QString NO_QUOTE( "__NO_QUOT__" );
QVariant toMagicVar( const QVariant &var, const DataFormatOptions &opt )
    const QVariant::Type type( var.type() );
    const QMetaType::Type metaType( (QMetaType::Type)type );
    if( opt.decimalPrecision != DataFormatOptions::DEFAULT_PRECISION
        && (type == QVariant::Type::Double || metaType == QMetaType::Float) )
            static const char FORMAT( 'f' );
            static const QRegExp trailingPointAndZeros( "\\.?0+$" );
            QString formatted( QString::number(
                var.toDouble(), FORMAT, opt.decimalPrecision ) );
            formatted.remove( trailingPointAndZeros );
            return QVariant( QString( NO_QUOTE + formatted + NO_QUOTE ) );
Note that I trim off any extraneous digits via formatted.remove.  If you want the data to always include exactly X digits after the decimal point, you may opt to skip that step.  (Or you might want to control that via DataFormatOptions?)
Once I have the json bytes I'm going to send across the network as a QByteArray, I remove the magic bytes so my numbers represented as quoted strings become numbers again in the json.
// This is where any "magic residue" is removed, or otherwise manipulated,
// to produce the desired final json bytes...
void scrubMagicBytes( QByteArray &bytes )
    static const QByteArray EMPTY, QUOTE( "\"" ),
        NO_QUOTE_PREFIX( QUOTE + NO_QUOTE.toLocal8Bit() ),
        NO_QUOTE_SUFFIX( NO_QUOTE.toLocal8Bit() + QUOTE );
    bytes.replace( NO_QUOTE_PREFIX, EMPTY );
    bytes.replace( NO_QUOTE_SUFFIX, EMPTY );
        
Thanks for contributing an answer to Stack Overflow!
Please be sure to answer the question. Provide details and share your research!
But avoid …
Asking for help, clarification, or responding to other answers.
Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.