How to insert entry to json file

[JonB]

@lukutis222 said in How to insert entry to json file:

Is that correct?

Yes.

[J.Hilk]

@lukutis222 keep in mind, that json does not specify an order of attributes/entries. Simply scanning and counting lines to modify your json document is bound to break very soon

[lukutis222]

@J-Hilk
Yeah I figured. Now I am working directly with json instead of counting lines.. Should be more reliable this way

[lukutis222]

@JonB

I can insert a new entry to my Array, however it is not fully clear to me how can I overwrite the whole file with the inserted entry.

        QFile file("./testplans/filename.json");
        file.open(QIODevice::ReadWrite | QIODevice::Text);
        QByteArray commands = file.readAll();

        QJsonDocument jsonresponse = QJsonDocument::fromJson(commands);
        QJsonObject jsonObject = jsonresponse.object();
        QJsonArray commandsArray = jsonObject.value("commands").toArray();

        // insert new entry
        QJsonObject data_to_insert = QJsonObject(
        {
        qMakePair(QString("Response type"),QJsonValue("test")),
        qMakePair(QString("Command name"), QJsonValue("test")),
        qMakePair(QString("Command"), QJsonValue("test")),
        qMakePair(QString("Expected response"), QJsonValue("test")),
        qMakePair(QString("Response retry count"), QJsonValue("test")),
        qMakePair(QString("Command retry count"), QJsonValue("test"))
        });
        commandsArray.insert(commandsArray.count(),data_to_insert); // insert new entry at the end

      // now my commandsArray consist a new entry

[JonB]

@lukutis222
QByteArray QJsonDocument::toJson(QJsonDocument::JsonFormat format = Indented) const. Save that to file. Do not open it QIODevice::ReadWrite!

[lukutis222]

@JonB

QFile file("./testplans/filename.json");
        file.open(QIODevice::ReadWrite | QIODevice::Text);
        QByteArray commands = file.readAll();

        QJsonDocument jsonresponse = QJsonDocument::fromJson(commands);
        QJsonObject jsonObject = jsonresponse.object();
        QJsonArray commandsArray = jsonObject.value("commands").toArray();

        // insert new entry
        QJsonObject data_to_insert = QJsonObject(
        {
        qMakePair(QString("Response type"),QJsonValue("test")),
        qMakePair(QString("Command name"), QJsonValue("test")),
        qMakePair(QString("Command"), QJsonValue("test")),
        qMakePair(QString("Expected response"), QJsonValue("test")),
        qMakePair(QString("Response retry count"), QJsonValue("test")),
        qMakePair(QString("Command retry count"), QJsonValue("test"))
        });
        commandsArray.insert(commandsArray.count(),data_to_insert); // insert new entry at the end

         // now my commandsArray consist a new entry

         // create new QJsonDocument with updated commandsArray
        QJsonDocument json_doc(commandsArray);
        QByteArray bytearray = json_doc.toJson();
        qDebug() << "json response is " << bytearray;

Is there any particular reason why I cannot see the whole json response when I try to print it to the console using

qDebug() << "json response is " << bytearray;

I see:

json response is  "[\n    {\n        \"Command\": \"ping\",\n        \"Command name\": \"Test UART0 ping\",\n        \"Command retry count\": 1,\n        \"Expected response\": \"pong\",\n        \"Response retry count\": 5,\n        \"Response type\": \"STRING\"\n    },\n    {\n        \"Command\": \"ping\",\n        \"Command name\": \"Test UART0 ping2\",\n        \"Command retry count\": 1,\n        \"Expected response\": \"pong\",\n        \"Response retry count\": 5,\n        \"Response type\": \"STRING\"\n    },\n    {\n        \"Command\": \"ping\",\n        \"Command name\": \"Test UART0 ping3\",\n        \"Command retry count\": 1,\n        \"Expected response\": \"pong\",\n        \"Response retry count\": 5,\n        \"Response type\": \"STRING\"\n    },\n    {\n        \"Command\": \"ping\",\n        \"Command name\": \"Test UART0 ping4\",\n        \"Command retry count\": 1,\n        \"Expected response\": \"pong\",\n        \"Response retry count\": 5,\n        \"Response type\": \"STRING\"\n    "��

The last command printed out is "Test UART0 ping4" which is definately not the last one.

Also, what is the problem of opening in ReadWrite, since I am reading the file and also will be overwriting it.

[JonB]

@lukutis222
Maybe qDebug() << QByteArray only prints a certain maximum of characters, I don't know. Maybe qDebug() << QString(bytearray) would work better, I don't know. In any case relying on the output of qDebug() is not a good test.

Also, what is the problem of opening in ReadWrite, since I am reading the file and also will be overwriting it.

You are not reading and writing/overwriting the file at the same time. You are/should be:

• Open file for read.
• Read whole file.
• Close file.
• Change in-memory stuff.
• Open file for write/overwrite (i.e. with truncate).
• Write whole file.
• Close file.

Do not do seeks and writes on a read-write file. You will get away with it if the file happens to be longer than before, but you will be left with incorrect extra stuff if it happens to be shorter.

[mpergand]

@lukutis222 said in How to insert entry to json file:

qDebug() << "json response is " << bytearray;

For easier reading use:
qDebug().noquote()<<bytearray;

[lukutis222]

I am almost there! (I have fixed the issue with qDebug not printing out the whole text and it turned out to be a syntax error inside json).

My current code

        QFile file("./testplans/filename.json");
        file.open(QIODevice::ReadOnly | QIODevice::Text);
        QByteArray commands = file.readAll();

        QJsonDocument jsonresponse = QJsonDocument::fromJson(commands);
        QJsonObject jsonObject = jsonresponse.object();
        QJsonArray commandsArray = jsonObject.value("commands").toArray();

        // insert new entry
        QJsonObject data_to_insert = QJsonObject(
        {
        qMakePair(QString("Response type"),QJsonValue("test")),
        qMakePair(QString("Command name"), QJsonValue("test")),
        qMakePair(QString("Command"), QJsonValue("test")),
        qMakePair(QString("Expected response"), QJsonValue("test")),
        qMakePair(QString("Response retry count"), QJsonValue("test")),
        qMakePair(QString("Command retry count"), QJsonValue("test"))
        });
        commandsArray.insert(commandsArray.count(),data_to_insert); // insert new entry at the end

         // now my commandsArray consist a new entry

         // create new QJsonDocument with updated commandsArray
        QJsonDocument json_doc(commandsArray);
        QByteArray bytearray = json_doc.toJson();
        qDebug() << "json response is " << bytearray;


        file.open(QFile::WriteOnly|QFile::Truncate);
        file.write(bytearray);
        file.close();

Everything works as expected, just one issue:

Initially, my json looks like (I have shorted it up just to make easier to work with):

{
	"commands":
	[
	{
		"Response type": "STRING",
		"Command name":"Test UART0 ping",
		"Command": "ping",
		"Expected response": "pong",
		"Response retry count":5,
		"Command retry count":1
	},
	
		{
		"Response type": "STRING",
		"Command name":"Test UART0 ping2",
		"Command": "ping",
		"Expected response": "pong",
		"Response retry count":5,
		"Command retry count":1
	},
	
		{
		"Response type": "STRING",
		"Command name":"Test UART0 ping3",
		"Command": "ping",
		"Expected response": "pong",
		"Response retry count":5,
		"Command retry count":1
	}
	]
}

After overwriting:

[
    {
        "Command": "ping",
        "Command name": "Test UART0 ping",
        "Command retry count": 1,
        "Expected response": "pong",
        "Response retry count": 5,
        "Response type": "STRING"
    },
    {
        "Command": "ping",
        "Command name": "Test UART0 ping2",
        "Command retry count": 1,
        "Expected response": "pong",
        "Response retry count": 5,
        "Response type": "STRING"
    },
    {
        "Command": "ping",
        "Command name": "Test UART0 ping3",
        "Command retry count": 1,
        "Expected response": "pong",
        "Response retry count": 5,
        "Response type": "STRING"
    },
    {
        "Command": "test",
        "Command name": "test",
        "Command retry count": 2,
        "Expected response": "test",
        "Response retry count": 1,
        "Response type": "test"
    }
]

As you can see from above, there is an issue with:

        QJsonDocument json_doc(commandsArray);
        QByteArray bytearray = json_doc.toJson();

I need to make sure that my commandsArray is encapsulated inside an array named "commands". I am now looking into how I can achieve this.

Also, the order is all messed up but I have read that everyone have this issue.

[JonB]

@lukutis222
You cannot alter the order/formatting of the toJson() output, JSON data is unordered, what it produces is legal JSON, may not be in same order/format as the original file.

You created a complete QJsonDocument via QJsonDocument json_doc(commandsArray). That is not what it originally was, you have lost what came before it/where the array was in the document.

I can't remember the details of how QJsonDocument works. Hopefully the changes you made went into the QJsonDocument jsonresponse directly. Did you try jsonresponse.toJson() to verify that (after your updates), no creation of a new QJsonDocument json_doc?

[lukutis222]

@JonB

Thanks for clarifying regarding the order.

Yes I now realize that I created QJsonDocument using commandsArray that I modified and it does not contain the main JsonObject "commands" . I have try to check if the change I made to commandsArray affect jsonresponse but unfortunately it does not.

I looked up how to add jsonarray inside another json object here (for java language):
https://stackoverflow.com/questions/12142238/add-jsonarray-to-jsonobject

and they use put() method, so something like:

QJsonObject jsonobject;
jsonobject.put("commands", commandsArray);

Unfortunately, QT does not have this method and I was not able to find a relevant method for adding a QJsonArray inside another QJsonObject.

[jsulm]

@lukutis222 said in How to insert entry to json file:

QT does not have this method and I was not able to find a relevant method for adding a QJsonArray inside another QJsonObject.

https://doc.qt.io/qt-5/qjsonobject.html#insert

[lukutis222]

@jsulm
Yes you are right.

        QJsonObject new_object;
        new_object.insert("commands",commandsArray);
        QJsonDocument json_doc(new_object);
        QByteArray bytearray = json_doc.toJson();
        file.open(QFile::WriteOnly|QFile::Truncate);
        file.write(bytearray);
        file.close();

Works without any issues. The file now contains everything:

{
    "commands": [
        {
            "Command": "ping",
            "Command name": "Test UART0 ping",
            "Command retry count": 1,
            "Expected response": "pong",
            "Response retry count": 5,
            "Response type": "STRING"
        },
        {
            "Command": "ping",
            "Command name": "Test UART0 ping2",
            "Command retry count": 1,
            "Expected response": "pong",
            "Response retry count": 5,
            "Response type": "STRING"
        },
        {
            "Command": "ping",
            "Command name": "Test UART0 ping3",
            "Command retry count": 1,
            "Expected response": "pong",
            "Response retry count": 5,
            "Response type": "STRING"
        },
        {
            "Command": "test",
            "Command name": "test",
            "Command retry count": 2,
            "Expected response": "test",
            "Response retry count": 1,
            "Response type": "test"
        }
    ]
}

[JonB]

@lukutis222

I looked up how to add jsonarray inside another json object here (for java language):

I am lost. You are not trying to add a new array inside anything, you are trying to add a new element to an existing array. As for looking at Java, that won't get you anywhere :)

I have try to check if the change I made to commandsArray affect jsonresponse but unfortunately it does not.

I don't think this is correct, but I do not have time right now to check. I suggest you try a tiny example to verify this.

Works without any issues. The file now contains everything:

OK, so you are good now anyway!

[ChrisW67]

@lukutis222 said in How to insert entry to json file:

Also, the file size (in lines) can be much bigger than 100. This is just an example

That may be so, but your code created an array of 100 QStrings to load the lines into:

 int line_count=0;
QString line[100];
...
while( !in.atEnd())
    {
        line[line_count]=in.readLine();
        line_count++;
    }

It does not seem to me that this will handle a file with 101 lines.