How to use QTcpSocket in static library to interact with non-Qt server

[FredT]

I am trying to implement a library to connect to an external resource and interact with it via an XML-based API. However I don't know how to get all the data before processing it. I am using QTcpSocket on Windows.

The way I have to interact with the resource is as follow:

1. Connect to the server
2. Send an XML message with credentials
3. Get the response from the server
4. Parse the XML response to extract the session ID
5. Send other requests in XML with the session ID in it and parse the XML responses.
6. Use the result in my application

Of course, depending on the request the message will not have the same structure so I need to parse the XML in the same function, right? And if so, how do I know that reading is done so I can start processing the XML data in this case?

Thanks,
Fred

[SGaist]

Hi,

Sounds like you should rather use QNetworkAccessManager.

[FredT]

Hi SGaist,
I took a look at QNetworkAccessManager, but I can't figure how to send an XML message to a server. How can I do that?

This is my current class using QTcpSocket for now:

Client.h

#ifndef Client_H
#define Client_H

#include <QObject>
#include <QTcpSocket>
#include <QByteArray>

#include <map>
#include <set>

#include "pugixml.hpp"

class Client : public QObject
{
    Q_OBJECT
public:
    Client(QObject* parent = 0) : QObject(parent) {}
    Client(const QString& server, const QString& port, QObject* parent = 0);

    // Connection
    void connectToServer(const QString& server, const QString& port);
    void login(const QString& user, const QString& password, const QString& project);
    void logout();

    QString const getSessionId() { return sessionId_; }

    void throwOnError(const pugi::xml_document& doc);

    QString sessionId() const { return sessionId_; }

    QString outMessage;     // For testing
    QString resultMessage;  // For testing

signals:
    void ready();
    void startReading();
    void finishedReading();
    void taskCompleted();

private slots:
    void getResult();

private:
    void sendMessage(const QString& message);    

    QTcpSocket socket_;

    QString sessionId_;
    QString projectName_;

    QByteArray result_;


};

#endif // Client_H

Client.cpp

#include "client.h"

#include <pugixml.hpp>

#include <map>
#include <sstream>
#include <string>

#include "task.h" // Generate the API message, not relevant


Client::Client(const QString& server, const QString& port, QObject* parent):
    QObject(parent)
{
    connectToServer(server, port);
    connect(&socket_, &QTcpSocket::readyRead, this, &Client::getResult);
}

void Client::connectToServer(const QString& server, const QString& port)
{
    bool ok;
    int portNumber = port.toInt(&ok);

    if (ok) {
        if (socket_.state() == QTcpSocket::SocketState::ConnectedState)
            socket_.disconnectFromHost();
        socket_.connectToHost(server, portNumber);
    } else {
        throw tr("Cannot connect to server %1 on port %2. Make sure the provided information are correct.")
            .arg(server)
            .arg(port);
    }
}

void Client::throwOnError(const pugi::xml_document& doc)
{
    pugi::xpath_node_set errors = doc.select_nodes("/EXECUTION/TASK/RESULTSET/RESULT/MESSAGE");

    std::string error_message = "";
    for (pugi::xpath_node_set::const_iterator it = errors.begin(); it != errors.end(); ++it)
    {
        pugi::xml_node node = it->node();
        if (std::string(node.attribute("type").value()) == "Error" ||
            std::string(node.attribute("type").value()) == "Warning")
            error_message += node.child_value();
    }

    if (!error_message.empty())
        throw std::exception(error_message.c_str());
}

void Client::sendMessage(const QString &message)
{
    outMessage = message;
    result_.clear();
    socket_.write(message.toUtf8());
}

void Client::getResult()
{
    emit startReading();
    while (socket_.bytesAvailable() > 0) {
        result_.append(socket_.readAll());
        socket_.flush();
    }

    resultMessage = QString(result_);
    emit finishedReading();
}

void Client::login(const QString& user, const QString& password, const QString& project)
{
    std::map<QString, QString> whereFields {{"userName", user}, {"password", password}};
    QString request = prepareMessage("Login", "Security", std::map<QString, QString>(), whereFields); // Generates the XML message for the API
    sendMessage(request);

    // Wait for data to arrive - How ?

    std::stringstream xmlResult = getXmlData(result_); // Remove the header from the API response and convert the QByteArray to a std::stringstream

    pugi::xml_document doc;
    pugi::xml_parse_result result = doc.load(xmlResult);

    throwOnError(doc);

    pugi::xpath_node session = doc.select_node("/EXECUTION/TASK/RESULTSET/DATASETS/DATASET/secId");
    sessionId_ = QString::fromStdString(session.node().first_child().value());
    projectName_ = project;

    emit taskCompleted();
}

[VRonin]

@FredT said in How to use QTcpSocket in static library to interact with non-Qt server:

but I can't figure how to send an XML message to a server.

http://doc.qt.io/qt-5/qnetworkaccessmanager.html#post-1

QByteArray dataBuffer;
QXmlStreamWriter xmlWrite(&dataBuffer);
// use xmlWrite to write stuff in xml
qnam->post(QNetworkRequest("http://mysite.com"),dataBuffer);

[FredT]

Hi VRonin,

Thanks, I'll llook into this.

One thing I fail to mention is that it is not a Web server. It is a server running on the internal network and to communicate with it I need to send messages with a custom header.

From the look of it, QNetworkAccessManager seems to be oriented towards Web programming. Is there any way to use it with a custom protocol?

[VRonin]

@FredT said in How to use QTcpSocket in static library to interact with non-Qt server:

Web programming

no, you can use it in local addresses but it handles known protocols.

For custom protocols QTcpSocket is indeed the right choice, you can pass the socket directly to QXmlStreamWriter constructor and use that wo send the data

[FredT]

Thanks for the information.

Currently I am using a QString to send the XML message because I also need to send a header as well for the protocol as well. Also, worth noting, not all answers from the server are in XML - depending on the request they can also be sent in a custom format.

So I guess I am back to my initial problem: how can I wait for all the data to be there before processing it? This is actually the only point I am stuck at.

[jsulm]

@FredT said in How to use QTcpSocket in static library to interact with non-Qt server:

So I guess I am back to my initial problem: how can I wait for all the data to be there before processing it?

Doesn't this depend on your protocol? I mean: there must be some notion of "END OF DATA" in what the server is sending to you (all what you send to server). Just read everything into a buffer until you get this "END OF DATA" and then process it. Use http://doc.qt.io/qt-5/qiodevice.html#readyRead signal to get the notification that there is new data and read it using http://doc.qt.io/qt-5/qiodevice.html#readAll

[VRonin]

@FredT said in How to use QTcpSocket in static library to interact with non-Qt server:

Currently I am using a QString to send the XML message because I also need to send a header as well for the protocol as well.

Not necessary, you can use a QTextStream and a QXmlStreamWriter on the same socket

how can I wait for all the data to be there before processing it?

QDataStream socketStream(socket);
socketStream.startTransaction();
// read data using socketStream
if(socketStream.commitTransaction()){
// all data was received and it was ok
}
else{
// not enough data or something went wrong
}

[FredT]

@jsulm
The problem with the end of data is that it is relative to the message sent. Furthermore, there might be extra characters at the end of the message (usually spaces) and other times the data will be incomplete for random reasons (for instance a tag that does not exist in the system will not be skipped or no error message will be sent but a partial data will be sent)

@VRonin
Here is my first try with the QDataStream as you suggested:

void Client::login(const QString& user, const QString& password, const QString& project)
{
    std::map<QString, QString> whereFields {{"userName", user}, {"password", password}};
    QString request = prepareMessage("Login", "Security", std::map<QString, QString>(), whereFields); // Generates the XML message for the API
    sendMessage(request);

   QDataStream socketStream(&socket_);

    socketStream.startTransaction();
    while (!socketStream.commitTransaction()) {
        continue;
    }

    QString socketResult;
    socketStream >> socketResult;
    qDebug() << socketResult;

    std::stringstream xmlResult = getXmlData(result_); // Remove the header from the API response and convert the QByteArray to a std::stringstream

    pugi::xml_document doc;
    pugi::xml_parse_result result = doc.load(xmlResult);

    throwOnError(doc);

    pugi::xpath_node session = doc.select_node("/EXECUTION/TASK/RESULTSET/DATASETS/DATASET/secId");
    sessionId_ = QString::fromStdString(session.node().first_child().value());
    projectName_ = project;

    emit taskCompleted();
}

In this case the socketResult is empty.

My second try:

void Client::getResult()
{
    socketStream_ >> result_;

    if (!socketStream_.commitTransaction())
        return;

    resultMessage = QString(result_);
    emit finishedReading();
}

void Client::login(const QString& user, const QString& password, const QString& project)
{
    std::map<QString, QString> whereFields {{"userName", user}, {"password", password}};
    QString request = prepareMessage("Login", "Security", std::map<QString, QString>(), whereFields); // Generates the XML message for the API
    sendMessage(request);

   QEventLoop loop;
    connect(this, &MapleClient::finishedReading, &loop, &QEventLoop::quit);
    loop.exec();

    std::stringstream xmlResult = getXmlData(result_); // Remove the header from the API response and convert the QByteArray to a std::stringstream

    pugi::xml_document doc;
    pugi::xml_parse_result result = doc.load(xmlResult);

    throwOnError(doc);

    pugi::xpath_node session = doc.select_node("/EXECUTION/TASK/RESULTSET/DATASETS/DATASET/secId");
    sessionId_ = QString::fromStdString(session.node().first_child().value());
    projectName_ = project;

    emit taskCompleted();
}

However this time it gets stuck at loop.exec(); and the getResult slot is never triggered.

Can you explain me what I am doing wrong, please?

[VRonin]

while (!socketStream.commitTransaction()) { continue; }

That's not how transaction works

[FredT]

Well, I tried both cases and I did not see any difference. My initial try was

void Client::login(const QString& user, const QString& password, const QString& project)
{
    std::map<QString, QString> whereFields {{"userName", user}, {"password", password}};
    QString request = prepareMessage("Login", "Security", std::map<QString, QString>(), whereFields); // Generates the XML message for the API
    sendMessage(request);
    socketStream.startTransaction();
    if (socketStream.commitTransaction()) {
        QString socketResult;
        socketStream >> socketResult;
        qDebug() << socketResult;
    } else {
        throw "Error!"
    }
}

But regardless of the way I did it the socketResult was empty.