How to parse nested array in a Json file
-
@mrjj
Yes I'm sure I didn't made any modification, I have the same set of test data in my database from the beginning (and to be sure I've checked directly in the DB). -
Hmm ok
so it seems to be a differnce betweenQJsonDocument::fromJson
and
QJsonDocument allPagesJsonDoc(allPagesArray);Not a good explanation for that, im afraid.
-
Can you please provide a minimal example for the behavior? I doubt there is a problem in the Qt api but more in how you convert from/to QString/QByteArray.
-
@mrjj, @Christian-Ehrlicher
It might come from the fact that when the file is retrieve is it firstly put inside a QByteArray.I've tried to qDebug the QByteArray copy and paste the content inside a text editor and then find ² symbol. It seems that the sympbol is converted as xB2.
@Christian-Ehrlicher:
I'm not sure to fully understand your request but I will try to give you the best overview:- First I send a set of test data to my API with Postman here is an example of the request body:
[ { "name" : "Alpha Blob", "sign" : "A", "count" : 5, "rank" : 1, "type" : "TYPE_A", "tags": [ { "id": 1 } ] }, etc... ]
- Then I use my Qt App to get all the datas with the following functions:
This function is called when I click on a button
void Dialog::getAllBlobs() { QJsonDocument allPagesJsonDoc; QString url = "blobj/all?pageNumber="; if(blobStore.getIsUpToDate()) { allPagesJsonDoc = blobStore.getBlobs(); } else { allPagesJsonDoc = httpService.getAll(url); QString storeStatus = blobStore.storeBlobs(allPagesJsonDoc); if(storeStatus != "OK") { QMessageBox::critical(this, "Error", storeStatus); } } displayResponse(allPagesJsonDoc); }
On the first call (after the application opened) datas are always fetched through the API.
Here is how I request the API:QJsonDocument httpService::getAll(QString &url) { qDebug() << "Get all blobs from API"; int pageNumber = 0; bool isLast = false; QJsonArray allPagesArray; QJsonObject infos; do { QString requestUrl = url.append(QString::number(pageNumber)); QJsonDocument objectList = get(requestUrl); if(objectList.object()["content"].isNull()) { break; } allPagesArray.append(objectList.object()["content"]); isLast = objectList.object()["last"].toBool(); pageNumber++; if(isLast == true) { infos.insert("totalPages", objectList.object().value("totalPages")); infos.insert("totalElements", objectList.object().value("totalElements")); infos.insert("size", objectList.object().value("size")); allPagesArray.append(infos); } } while(!isLast); QJsonDocument allPagesJsonDoc(allPagesArray); return allPagesJsonDoc; }
Here is how the GET http request is made:
QJsonDocument httpService::get(QString &url) { QString requestUrl = m_apiUrl + url; QNetworkRequest request(requestUrl); request.setHeader(QNetworkRequest::ContentTypeHeader, "application/json"); QNetworkReply *reply = nam.get(request); while (!reply->isFinished()) { qApp->processEvents(); } QByteArray response_data = reply->readAll(); QJsonDocument json = handleHTTPErrors(response_data, reply); reply->deleteLater(); return json; }
Then I store the data in a file:
QString blobStore::storeBlobs(QJsonDocument &blobsToSave) { qDebug() << "Storing blobs..."; QString returnMsg; QString blobStore = QDir::currentPath() + QDir::separator() + "blobStore.json"; QFile file(blobStore); if(!file.open(QIODevice::WriteOnly)) { returnMsg = file.errorString(); return returnMsg; } QTextStream stream(&file); stream << QString(blobsToSave.toJson()); file.close(); m_isUpToDate = true; returnMsg = "OK"; return returnMsg; }
- So now my store is considered as up to date. So if I want to retrieve all datas once again, it going to be retrieve from the file.
Here is how:
QJsonDocument blobStore::getBlobs() { qDebug() << "Get all blobs from STORE"; QJsonParseError jsonError; QString blobStore = QDir::currentPath() + QDir::separator() + "blobStore.json"; QFile file(blobStore); if(!file.exists()) { qWarning() << "Blob store file not found"; } if(!file.open(QIODevice::ReadOnly)) { qWarning() << "Cannot open Blob store file: " << file.errorString(); } QByteArray storeContentArray = file.readAll(); file.close(); QJsonDocument storeContentJson = QJsonDocument::fromJson(storeContentArray, &jsonError); if(jsonError.error != QJsonParseError::NoError) { qDebug() << QString("JsonError: %1").arg(jsonError.errorString()); qDebug() << jsonError.offset; } else if(storeContentJson.isEmpty()) { qDebug() << "storeContentJsonArray is Empty"; } return storeContentJson; }
- Finally datas are dipslaued in a simple Plain Text Editor.
Hope it is what you wanted.
If you want to see all of the code:
https://github.com/AyNeumann/DraftQtProject -
Please don't convert the QByteArray to a QString just to pass it to a QTextStream/QFile. Simply use QFile::write(QByteArray()).
And when you read from the file you get a parser error? Can you provide a minimal example (e.g. writing out a simple json array with "²" and read it in again so we can test it? What OS do you use? -
@Christian-Ehrlicher
If I have the following Json:
{
"sign": "²"
}and try to read it, I have the following error:
"JsonError: invalid UTF8 string"
The QJsonParseError::offset return 13 which is the " character juste before the ² symbol.
I'm on Windows 10.
@Christian-Ehrlicher said in How to parse nested array in a Json file:
Please don't convert the QByteArray to a QString just to pass it to a QTextStream/QFile. Simply use QFile::write(QByteArray()).
I put the response of the get request inside a QJsonDocument because it is for me easier to manipulate to create a consitent object.
I will try to use the raw QByteArray to write into the file but it will take some time as it implies a lot of refactoring. -
@Aymeric_Qt said in How to parse nested array in a Json file:
I will try to use the raw QByteArray to write into the file but it will take some time as it implies a lot of refactoring.
Why? It's just a simple change in storeBlobs() which removes two lines and adds one new
-
And this is a simple reproducer for me:
int main(int argc, char *argv[]) { QApplication app(argc, argv); QJsonArray arr; arr += QJsonValue(QString(QChar(0xb2))); QFile f(QLatin1String("tmp.txt")); f.open(QIODevice::WriteOnly); f.write(QJsonDocument(arr).toJson()); f.close(); f.open(QIODevice::ReadOnly); QJsonParseError err; QJsonDocument doc(QJsonDocument::fromJson(f.readAll(), &err)); if (err.error != QJsonParseError::NoError) { fprintf(stderr, "Parse error at: %d", err.offset); return 1; } arr = doc.array(); for (const auto &ele : arr) QMessageBox::information(0, QString(), ele.toString()); return 0; }
-
@Aymeric_Qt said in How to parse nested array in a Json file:
{ "sign": "²" }
That's a "funny" character, isn't the error message
"JsonError: invalid UTF8 string"
telling you it's not a UTF-8 character? Seems pretty likely to me.
-
Sorry I did not see your message early enough.
For info I'm a beginner in Qt and in C++ in general so yes, even something which is going to be nothing for you could be really complicated to me.
As you said:
@Christian-Ehrlicher said in How to parse nested array in a Json file:Please don't convert the QByteArray to a QString just to pass it to a QTextStream/QFile. Simply use QFile::write(QByteArray()).
So I've 'refactor' storeBlob so it can take QByteArray as an argument and save it into a file using:
file.write(blobsToSave);
And in this case it seems to work correctly.
But to be sure I will also try your reproducer.
@JonB :
Yes that what I've thought but it in this case why can it be received from the API, read, displayed and save but not read from a file? -
@Aymeric_Qt
Because characters have different representations when saved into file. -
@Christian Ehrlicher,
Ok so I've tested with your reproducer (copy and paste in the main function) and nothing blows up.
So I've tried with the 'storeBlobs' I had and just change the:QTextStream stream(&file); stream << QString(blobsToSave.toJson());
into:
file.write(blobsToSave.toJson());
as you did in the code sample you gave me and now it's working I can read data from the file without any problem.
So the problem might came from the fact that I was converting the QJsonDocument into a QTextStream (maybe because I did not did it the right way).
Thank you all for your replies!
-
@Aymeric_Qt said in How to parse nested array in a Json file:
So the problem might came from the fact that I was converting the QJsonDocument into a QTextStream
It is (and therefore my question about the used OS) - see https://doc.qt.io/qt-5/qtextstream.html#details
"By default, QTextCodec::codecForLocale() is used for reading and writing,"