QByteArray::data() returns small buffer

[_ove_]

Does anyone know why QByteArray::data() return a buffer with size 100 bytes only?
I have read 'QByteArray array' from QNetworkAccessManager, and
char *str = array.data() result in a 100 bytes buffer.

Qt Creator 4.11.1
Based on Qt 5.14.1 (MSVC 2017, 32 bit)

Thank you in advance
ove

[J.Hilk]

hi @_ove_ welcome

that is easy to answer, because you only stored 100 bytes of data in it.

[Christian Ehrlicher]

@_ove_ said in QByteArray::data() returns small buffer:

result in a 100 bytes buffer.

In which buffer? How to you know this?

[_ove_]

I downloaded a web page > 9K into a QByteArray. Then i tried to access the QByteArray buffer via the QByteArray::data() methode.
<snip>
char *buffer = byteArray.data();
<snip>
This left me with a 1oo byte long string (aka. char *).
The QByteArray reports the correct size, and contains all the 9K data.

[SGaist]

Hi and welcome to devnet,

How are you measuring these 100 bytes ?

[jsulm]

@_ove_ said in QByteArray::data() returns small buffer:

char *buffer = byteArray.data();

I guess the buffer contains 0 bytes which means that when interpreted as string (char*) it will cut at the first 0 byte.
What encoding is used? UTF-8, something else? Why do you interpret it as char*? Convert it to QString instead.

[_ove_]

@_ove_ said in QByteArray::data() returns small buffer:

Qt Creator 4.11.1

I'm using Qt Creator 4.11.1, the debugger shows the content of the char *, 100 bytes from the start of the QByteArray. The debugger also shows the content of the QByteArray to be > 9K
In order to parse the content I want to use a char *pointer that can be increased in steps of 1, in order to point to the consecutive characters in the 'char *buffer' string.

[J.Hilk]

@_ove_ please show an exact copy of your code.

[_ove_]

I have made a short poc, but how do I upload the files? Tried to 'upload image', but forum says I don't have enough privileges.

I can try past them in her or ?

ove

[Christian Ehrlicher]

We won't images but text.
Apart from this - please reread what @jsulm wrote about \0 terminated char* arrays!

[_ove_]

Ok, as an old 'c' programmer I know a few things about \0 terminated strings,
I will try to paste the files here:
main.cpp:
#include "mainwindow.h"
#include <QApplication>
int main(int argc, char *argv[])
{
QApplication a(argc, argv);
MainWindow w;
w.show();
w.GetPage("https://www.google.com"); // or whatever
return a.exec();
}

mainwindow.h:
#ifndef MAINWINDOW_H
#define MAINWINDOW_H
#include "webreader.h"
#include "webreaderthread.h"
#include <QObject>
#include <QMutex>
#include <QUrl>
#include <QMainWindow>
QT_BEGIN_NAMESPACE
namespace Ui { class MainWindow; }
QT_END_NAMESPACE
class MainWindow : public QMainWindow
{
Q_OBJECT
public:
MainWindow(QWidget *parent = nullptr);
~MainWindow();
void GetPage(QString page);
signals:
void signal_GetPage(QUrl);
void signal_Close();
private:
WebReader m_Reader;
WebReaderThread *m_pWebThread;
QByteArray m_ReadBuffer;
QMutex m_Mutex;
void ParseDocument(char *source);
private slots:
void OnSignalReply(QByteArray &reply);
void OnThreadFinished() { m_Mutex.unlock(); }
void OnSignalStarted() { m_Mutex.unlock(); }
};
#endif // MAINWINDOW_H

mainwindow.cpp:
#include "mainwindow.h"
#include <QCoreApplication>
MainWindow::MainWindow(QWidget *parent)
: QMainWindow(parent)
, m_Reader()
, m_pWebThread(new WebReaderThread())
{
m_Mutex.lock();
connect(this, SIGNAL(signal_GetPage(QUrl)), &m_Reader, SLOT(OnSignalGetPage(QUrl)));
connect(this, SIGNAL(signal_Close()), &m_Reader, SLOT(OnSignalClose()));
connect(m_pWebThread, &WebReaderThread::finished, this, &MainWindow::OnThreadFinished);
connect(&m_Reader, SIGNAL(signal_Reply(QByteArray &)), this, SLOT(OnSignalReply(QByteArray &)), Qt::DirectConnection);
connect(&m_Reader, SIGNAL(signal_Started()), this, SLOT(OnSignalStarted()));
m_Reader.moveToThread(m_pWebThread);
m_pWebThread->start();
}
MainWindow::~MainWindow()
{
if ( nullptr != m_pWebThread ) {
emit signal_Close();
m_Mutex.lock();
delete m_pWebThread;
}
else
m_Mutex.tryLock();
m_Mutex.unlock();
}
void MainWindow::GetPage(QString page)
{
emit signal_GetPage(QUrl(page));
m_Mutex.lock();
char characters = m_ReadBuffer.data();
// To illustrate the use of the char string
ParseDocument(characters);
exit(0);
}
void MainWindow::OnSignalReply(QByteArray &reply)
{
m_ReadBuffer = reply;
m_Mutex.unlock();
}
void MainWindow::ParseDocument(char *source)
{
char *ptr = source;
for ( ; *ptr != '\0'; ptr++ ) {
if ( '\n' == *ptr ) {
// Do something
}
}
}

webreaderthread.h:
#ifndef WEBREADERTHREAD_H
#define WEBREADERTHREAD_H
#include <QObject>
#include <QThread>
class WebReaderThread : public QThread
{
Q_OBJECT
public:
WebReaderThread(QObject *parent = nullptr);
private:
void Run();
};
#endif // WEBREADERTHREAD_H

webreaderthread.cpp:
#ifndef WEBREADERTHREAD_H
#define WEBREADERTHREAD_H
#include <QObject>
#include <QThread>
class WebReaderThread : public QThread
{
Q_OBJECT
public:
WebReaderThread(QObject *parent = nullptr);
private:
void Run();
};
#endif // WEBREADERTHREAD_H

webreader.h:
#ifndef WEBREADER_H
#define WEBREADER_H
#include <QNetworkAccessManager>
#include <QNetworkRequest>
#include <QNetworkReply>
#include <QObject>
#include <QUrl>
#include <QMutex>
class WebReader : public QObject
{
Q_OBJECT
public:
WebReader(QObject *pParent = nullptr);
~WebReader();
signals:
void signal_Reply(QByteArray &);
private:
QNetworkAccessManager *m_pAccessMgr;
QNetworkReply *m_pReply;
QByteArray m_szPage;
bool m_bError;
bool m_bSslError;
bool CreateAccessManager();
private slots:
void OnSignalClose();
void OnSignalGetPage(QUrl url);
// On signals from QNetworkAccessManager
void OnFinished(QNetworkReply *);
// On signals from QNetworkReply
void OnReadyRead();
void OnError(QNetworkReply::NetworkError code);
void OnReplySslError(const QList<QSslError> &errors);
};
#endif // WEBREADER_H

webreader.cpp;
#include "webreader.h"
WebReader::WebReader(QObject *pParent)
: QObject(pParent)
, m_pAccessMgr(nullptr)
, m_pReply(nullptr)
, m_bError(false)
, m_bSslError(false)
{
}

WebReader::~WebReader()
{
if ( NULL != m_pAccessMgr ) {
delete m_pAccessMgr;
}
}
void WebReader::OnSignalGetPage(QUrl url)
{
if ( nullptr == m_pAccessMgr ) {
if ( !CreateAccessManager() )
return;
}
QNetworkRequest request(url);
m_pReply = m_pAccessMgr->get(request);
if ( m_bSslError )
return;
connect(m_pReply, &QNetworkReply::readyRead, this, &WebReader::OnReadyRead, Qt::DirectConnection);
connect(m_pReply, SIGNAL(error(QNetworkReply::NetworkError)), this, SLOT(OnError(QNetworkReply::NetworkError)), Qt::DirectConnection);
connect(m_pReply, &QNetworkReply::sslErrors, this, &WebReader::OnReplySslError, Qt::DirectConnection);
m_szPage.clear();
m_pReply->readAll();
}
void WebReader::OnError(QNetworkReply::NetworkError code)
{
m_bError = true;
}

void WebReader::OnReplySslError(const QList<QSslError> &errors)
{
m_bSslError = true;
}
void WebReader::OnSignalClose()
{
if ( NULL != m_pReply ) {
delete m_pReply;
m_pReply = nullptr;
}
if ( NULL != m_pAccessMgr ) {
delete m_pAccessMgr;
m_pAccessMgr = nullptr;
}
exit(0);
}
void WebReader::OnFinished(QNetworkReply *)
{
emit signal_Reply(m_szPage);
}
void WebReader::OnReadyRead()
{
m_szPage += m_pReply->readAll();
}
bool WebReader::CreateAccessManager()
{
m_pAccessMgr = new QNetworkAccessManager();
if ( nullptr == m_pAccessMgr )
return false;
m_pAccessMgr->setAutoDeleteReplies(true);
connect(m_pAccessMgr, &QNetworkAccessManager::finished, this, &WebReader::OnFinished, Qt::DirectConnection);
return true;
}

Project file QByteArrayTest.pro:
QT += core gui
QT += network
greaterThan(QT_MAJOR_VERSION, 4): QT += widgets
CONFIG += c++11
DEFINES += QT_DEPRECATED_WARNINGS
SOURCES +=
main.cpp
mainwindow.cpp
webreader.cpp
webreaderthread.cpp
HEADERS +=
mainwindow.h
webreader.h
webreaderthread.h
qnx: target.path = /tmp/$${TARGET}/bin
else: unix:!android: target.path = /opt/$${TARGET}/bin
!isEmpty(target.path): INSTALLS += target

Hope this will work
ove

[Christian Ehrlicher]

First this is not really minimal and has a lot of unneeded stuff which makes it hard to find out what should go wrong, second where does the error occur now in your code?

[_ove_]

Sorry, messed up one small file:
webreaderthread.cpp:
#include "webreaderthread.h"
#include <QObject>
WebReaderThread::WebReaderThread(QObject *parent)
: QThread(parent)
{

}
void WebReaderThread::Run()
{
exec();
}

ove

[Christian Ehrlicher]

Why do you need this class at all to reproduce the issue? It does nothing more than a plain QThread does. Where exactly is now the issue?

[_ove_]

Here is the short one;
#include <QCoreApplication>
#include <QString>
int main(int argc, char *argv[])
{
QString s(2000, QChar('c'));
// s now contains 2000 occurences of 'c'
char *str = s.toUtf8().data();
// str now contains 100 occurences of, what?
// Why doesnt the debugger show the content of str?
// But there still in only 100
exit(0);
}

[Christian Ehrlicher]

@_ove_ said in QByteArray::data() returns small buffer:

char *str = s.toUtf8().data();

C++ basics - you're working on a dangling pointer

// str now contains 100 occurences of, what?

str does not contain anything. It points to a (now deleted due to your other error) memory location. How you interpret this is up to you. Since it's a char* pointer, the debugger interprets it as characters and prints them out. Since a pointer does not contain any length information, functions like e.g. strlen, strcpy need a stop marker which is \0. So when your memory contains a \0, the output stops.
And also the debugger only prints a certain amount of memory - so if there is no \0 character anywhere in your memory, it will not print all but stop somewhere. Maybe even at 100 bytes, don't know and is not important. Important is on how you interpret this pointer and what you're doing with them. Your code does nothing with them so there is no error which can be fixed (apart from the dangling pointer).

[_ove_]

Ok, I thought that str should point to the internal data structure of the QByteArray, containing the data.
Anyway, I have figured out the original problem:
when i retrieve the char* pointer from a QByteArray by means of the data() method, the debugger only displays the first100 bytes of the content of the pointer, although the rest of the data is there also. On the other hand, the debugger displays the entire content for the QByteArray variable, not only the first 100 bytes.

Thanks for your effort
ove