Skip to content
  • Categories
  • Recent
  • Tags
  • Popular
  • Users
  • Groups
  • Search
  • Get Qt Extensions
  • Unsolved
Collapse
Brand Logo
  1. Home
  2. Qt Development
  3. General and Desktop
  4. How to parse nested array in a Json file
Forum Updated to NodeBB v4.3 + New Features

How to parse nested array in a Json file

Scheduled Pinned Locked Moved Solved General and Desktop
21 Posts 4 Posters 4.7k Views 1 Watching
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • Christian EhrlicherC Online
    Christian EhrlicherC Online
    Christian Ehrlicher
    Lifetime Qt Champion
    wrote on last edited by
    #11

    Can you please provide a minimal example for the behavior? I doubt there is a problem in the Qt api but more in how you convert from/to QString/QByteArray.

    Qt Online Installer direct download: https://download.qt.io/official_releases/online_installers/
    Visit the Qt Academy at https://academy.qt.io/catalog

    1 Reply Last reply
    1
    • A Offline
      A Offline
      Aymeric_Qt
      wrote on last edited by Aymeric_Qt
      #12

      @mrjj, @Christian-Ehrlicher
      It might come from the fact that when the file is retrieve is it firstly put inside a QByteArray.

      I've tried to qDebug the QByteArray copy and paste the content inside a text editor and then find ² symbol. It seems that the sympbol is converted as xB2.

      @Christian-Ehrlicher:
      I'm not sure to fully understand your request but I will try to give you the best overview:

      • First I send a set of test data to my API with Postman here is an example of the request body:
      [
      	{
      		"name" : "Alpha Blob",
      		"sign" : "A",
      		"count" : 5,
      		"rank" : 1,
      		"type" : "TYPE_A",
      		"tags": [
              	{
                  	"id": 1
              	}
          	]
      	},
              etc...
      ]
      
      • Then I use my Qt App to get all the datas with the following functions:

      This function is called when I click on a button

      void Dialog::getAllBlobs()
      {
          QJsonDocument allPagesJsonDoc;
      
          QString url = "blobj/all?pageNumber=";
          if(blobStore.getIsUpToDate())
          {
              allPagesJsonDoc = blobStore.getBlobs();
          }
          else
          {
              allPagesJsonDoc = httpService.getAll(url);
              QString storeStatus = blobStore.storeBlobs(allPagesJsonDoc);
              if(storeStatus != "OK")
              {
                  QMessageBox::critical(this, "Error", storeStatus);
              }
          }
      
          displayResponse(allPagesJsonDoc);
      }
      

      On the first call (after the application opened) datas are always fetched through the API.
      Here is how I request the API:

      QJsonDocument httpService::getAll(QString &url)
      {
          qDebug() << "Get all blobs from API";
      
          int pageNumber = 0;
          bool isLast = false;
          QJsonArray allPagesArray;
          QJsonObject infos;
      
          do {
              QString requestUrl = url.append(QString::number(pageNumber));
              QJsonDocument objectList = get(requestUrl);
      
              if(objectList.object()["content"].isNull())
              {
                  break;
              }
      
              allPagesArray.append(objectList.object()["content"]);
      
              isLast = objectList.object()["last"].toBool();
              pageNumber++;
      
              if(isLast == true)
              {
                  infos.insert("totalPages", objectList.object().value("totalPages"));
                  infos.insert("totalElements", objectList.object().value("totalElements"));
                  infos.insert("size", objectList.object().value("size"));
                  allPagesArray.append(infos);
              }
      
          } while(!isLast);
      
          QJsonDocument allPagesJsonDoc(allPagesArray);
      
          return allPagesJsonDoc;
      }
      

      Here is how the GET http request is made:

      QJsonDocument httpService::get(QString &url)
      {
          QString requestUrl = m_apiUrl +  url;
          QNetworkRequest request(requestUrl);
          request.setHeader(QNetworkRequest::ContentTypeHeader, "application/json");
      
          QNetworkReply *reply = nam.get(request);
      
          while (!reply->isFinished())
          {
              qApp->processEvents();
          }
      
          QByteArray response_data = reply->readAll();
      
          QJsonDocument json = handleHTTPErrors(response_data, reply);
      
          reply->deleteLater();
      
          return json;
      }
      

      Then I store the data in a file:

      QString blobStore::storeBlobs(QJsonDocument &blobsToSave)
      {
          qDebug() << "Storing blobs...";
          QString returnMsg;
      
          QString blobStore = QDir::currentPath() + QDir::separator() + "blobStore.json";
          QFile file(blobStore);
      
          if(!file.open(QIODevice::WriteOnly))
          {
              returnMsg = file.errorString();
              return returnMsg;
          }
      
          QTextStream stream(&file);
          stream << QString(blobsToSave.toJson());
          file.close();
      
          m_isUpToDate = true;
      
          returnMsg = "OK";
      
          return returnMsg;
      }
      
      • So now my store is considered as up to date. So if I want to retrieve all datas once again, it going to be retrieve from the file.
        Here is how:
      QJsonDocument blobStore::getBlobs()
      {
      
          qDebug() << "Get all blobs from STORE";
          QJsonParseError jsonError;
      
          QString blobStore = QDir::currentPath() + QDir::separator() + "blobStore.json";
          QFile file(blobStore);
      
          if(!file.exists())
          {
              qWarning() << "Blob store file not found";
          }
      
          if(!file.open(QIODevice::ReadOnly))
          {
              qWarning() << "Cannot open Blob store file: " << file.errorString();
          }
      
          QByteArray storeContentArray = file.readAll();
      
          file.close();
      
          QJsonDocument storeContentJson = QJsonDocument::fromJson(storeContentArray, &jsonError);
      
          if(jsonError.error != QJsonParseError::NoError)
          {
              qDebug() << QString("JsonError: %1").arg(jsonError.errorString());
              qDebug() << jsonError.offset;
          }
          else if(storeContentJson.isEmpty())
          {
              qDebug() << "storeContentJsonArray is Empty";
          }
      
          return storeContentJson;
      }
      
      • Finally datas are dipslaued in a simple Plain Text Editor.

      Hope it is what you wanted.

      If you want to see all of the code:
      https://github.com/AyNeumann/DraftQtProject

      1 Reply Last reply
      0
      • Christian EhrlicherC Online
        Christian EhrlicherC Online
        Christian Ehrlicher
        Lifetime Qt Champion
        wrote on last edited by
        #13

        Please don't convert the QByteArray to a QString just to pass it to a QTextStream/QFile. Simply use QFile::write(QByteArray()).
        And when you read from the file you get a parser error? Can you provide a minimal example (e.g. writing out a simple json array with "²" and read it in again so we can test it? What OS do you use?

        Qt Online Installer direct download: https://download.qt.io/official_releases/online_installers/
        Visit the Qt Academy at https://academy.qt.io/catalog

        1 Reply Last reply
        2
        • A Offline
          A Offline
          Aymeric_Qt
          wrote on last edited by
          #14

          @Christian-Ehrlicher
          If I have the following Json:
          {
          "sign": "²"
          }

          and try to read it, I have the following error:

          "JsonError: invalid UTF8 string"
          

          The QJsonParseError::offset return 13 which is the " character juste before the ² symbol.

          I'm on Windows 10.

          @Christian-Ehrlicher said in How to parse nested array in a Json file:

          Please don't convert the QByteArray to a QString just to pass it to a QTextStream/QFile. Simply use QFile::write(QByteArray()).

          I put the response of the get request inside a QJsonDocument because it is for me easier to manipulate to create a consitent object.
          I will try to use the raw QByteArray to write into the file but it will take some time as it implies a lot of refactoring.

          JonBJ 1 Reply Last reply
          0
          • Christian EhrlicherC Online
            Christian EhrlicherC Online
            Christian Ehrlicher
            Lifetime Qt Champion
            wrote on last edited by
            #15

            @Aymeric_Qt said in How to parse nested array in a Json file:

            I will try to use the raw QByteArray to write into the file but it will take some time as it implies a lot of refactoring.

            Why? It's just a simple change in storeBlobs() which removes two lines and adds one new

            Qt Online Installer direct download: https://download.qt.io/official_releases/online_installers/
            Visit the Qt Academy at https://academy.qt.io/catalog

            1 Reply Last reply
            2
            • Christian EhrlicherC Online
              Christian EhrlicherC Online
              Christian Ehrlicher
              Lifetime Qt Champion
              wrote on last edited by
              #16

              And this is a simple reproducer for me:

              int main(int argc, char *argv[])
              {
                  QApplication app(argc, argv);
                  
                  QJsonArray arr;
                  arr += QJsonValue(QString(QChar(0xb2)));
                  QFile f(QLatin1String("tmp.txt"));
                  f.open(QIODevice::WriteOnly);
                  f.write(QJsonDocument(arr).toJson());
                  f.close();
                  
                  f.open(QIODevice::ReadOnly);
                  QJsonParseError err;
                  QJsonDocument doc(QJsonDocument::fromJson(f.readAll(), &err));
                  if (err.error != QJsonParseError::NoError) {
                      fprintf(stderr, "Parse error at: %d", err.offset);
                      return 1;
                  }
                  arr = doc.array();
                  for (const auto &ele : arr)
                      QMessageBox::information(0, QString(), ele.toString());
                  return 0;
              }
              

              Qt Online Installer direct download: https://download.qt.io/official_releases/online_installers/
              Visit the Qt Academy at https://academy.qt.io/catalog

              1 Reply Last reply
              2
              • A Aymeric_Qt

                @Christian-Ehrlicher
                If I have the following Json:
                {
                "sign": "²"
                }

                and try to read it, I have the following error:

                "JsonError: invalid UTF8 string"
                

                The QJsonParseError::offset return 13 which is the " character juste before the ² symbol.

                I'm on Windows 10.

                @Christian-Ehrlicher said in How to parse nested array in a Json file:

                Please don't convert the QByteArray to a QString just to pass it to a QTextStream/QFile. Simply use QFile::write(QByteArray()).

                I put the response of the get request inside a QJsonDocument because it is for me easier to manipulate to create a consitent object.
                I will try to use the raw QByteArray to write into the file but it will take some time as it implies a lot of refactoring.

                JonBJ Online
                JonBJ Online
                JonB
                wrote on last edited by JonB
                #17

                @Aymeric_Qt said in How to parse nested array in a Json file:

                {
                 "sign": "²"
                }
                

                That's a "funny" character, isn't the error message

                "JsonError: invalid UTF8 string"

                telling you it's not a UTF-8 character? Seems pretty likely to me.

                1 Reply Last reply
                1
                • A Offline
                  A Offline
                  Aymeric_Qt
                  wrote on last edited by Aymeric_Qt
                  #18

                  @Christian-Ehrlicher

                  Sorry I did not see your message early enough.

                  For info I'm a beginner in Qt and in C++ in general so yes, even something which is going to be nothing for you could be really complicated to me.

                  As you said:
                  @Christian-Ehrlicher said in How to parse nested array in a Json file:

                  Please don't convert the QByteArray to a QString just to pass it to a QTextStream/QFile. Simply use QFile::write(QByteArray()).

                  So I've 'refactor' storeBlob so it can take QByteArray as an argument and save it into a file using:

                  file.write(blobsToSave);
                  

                  And in this case it seems to work correctly.

                  But to be sure I will also try your reproducer.

                  @JonB :
                  Yes that what I've thought but it in this case why can it be received from the API, read, displayed and save but not read from a file?

                  JonBJ 1 Reply Last reply
                  0
                  • A Aymeric_Qt

                    @Christian-Ehrlicher

                    Sorry I did not see your message early enough.

                    For info I'm a beginner in Qt and in C++ in general so yes, even something which is going to be nothing for you could be really complicated to me.

                    As you said:
                    @Christian-Ehrlicher said in How to parse nested array in a Json file:

                    Please don't convert the QByteArray to a QString just to pass it to a QTextStream/QFile. Simply use QFile::write(QByteArray()).

                    So I've 'refactor' storeBlob so it can take QByteArray as an argument and save it into a file using:

                    file.write(blobsToSave);
                    

                    And in this case it seems to work correctly.

                    But to be sure I will also try your reproducer.

                    @JonB :
                    Yes that what I've thought but it in this case why can it be received from the API, read, displayed and save but not read from a file?

                    JonBJ Online
                    JonBJ Online
                    JonB
                    wrote on last edited by
                    #19

                    @Aymeric_Qt
                    Because characters have different representations when saved into file.

                    1 Reply Last reply
                    3
                    • A Offline
                      A Offline
                      Aymeric_Qt
                      wrote on last edited by
                      #20

                      @Christian Ehrlicher,

                      Ok so I've tested with your reproducer (copy and paste in the main function) and nothing blows up.
                      So I've tried with the 'storeBlobs' I had and just change the:

                      QTextStream stream(&file);
                      stream << QString(blobsToSave.toJson());
                      

                      into:

                      file.write(blobsToSave.toJson());
                      

                      as you did in the code sample you gave me and now it's working I can read data from the file without any problem.

                      So the problem might came from the fact that I was converting the QJsonDocument into a QTextStream (maybe because I did not did it the right way).

                      Thank you all for your replies!

                      1 Reply Last reply
                      0
                      • Christian EhrlicherC Online
                        Christian EhrlicherC Online
                        Christian Ehrlicher
                        Lifetime Qt Champion
                        wrote on last edited by
                        #21

                        @Aymeric_Qt said in How to parse nested array in a Json file:

                        So the problem might came from the fact that I was converting the QJsonDocument into a QTextStream

                        It is (and therefore my question about the used OS) - see https://doc.qt.io/qt-5/qtextstream.html#details

                        "By default, QTextCodec::codecForLocale() is used for reading and writing,"

                        Qt Online Installer direct download: https://download.qt.io/official_releases/online_installers/
                        Visit the Qt Academy at https://academy.qt.io/catalog

                        1 Reply Last reply
                        2

                        • Login

                        • Login or register to search.
                        • First post
                          Last post
                        0
                        • Categories
                        • Recent
                        • Tags
                        • Popular
                        • Users
                        • Groups
                        • Search
                        • Get Qt Extensions
                        • Unsolved