Overriding QIODevice::readLineData() not working as expected

kodiak32

If need to read in text files line by line with different line ending termination. Since Qt only handles LF, CR+LF but I also need CR I followed the documentation to override QIODevice::readLineData().

class ReplayFile : public QFile {
...
qint64 readLineData(char* data, qint64 maxlen) {
    char character;
    bool success = false;
    qint64 bytesRead;

    for(bytesRead = 0; bytesRead <= maxlen;) {
        success = getChar(&character);

        // on error quit immediatly, but only if no data was read
        if(!success && (bytesRead == 0)) {
            qDebug() << "error";
            return -1;
        }

        if (character == 0) {
            qDebug() << "character 0";
            return bytesRead;
        }
        data[bytesRead++] = character;

        if (character == '\r') {
            getChar(&character);
            if (character == '\n') {
                data[bytesRead++] = character;
                qDebug() << "CR and LF: " << QString(data);
            }
            else {
                ungetChar(character);
                qDebug() << "CR only: " << QString(data);
            }
            return bytesRead;
        }
        else if(character == '\n') {
            qDebug() << "LF only: " << QString(data);
            return bytesRead;
        }
    }
    qDebug() << "should not happen";
    return bytesRead;
}

// do the actual call here
while(true) {
        file.readLine();
}

First time call to ReplayFile::readLine() outputs correctly the first line from a CR+LF terminated file:
"CR and LF: "$C221.8P1.0R1.4T27.9Mx77.09My-64.65Mz290.41Ax0.020Ay0.027Az1.0793B*"
, but second invocation does not invoke my override.

I tracked down the difference from the first to the second call in qiodevice.cpp, Qt4.8 in line 1088 if (!d->buffer.isEmpty()) {

I have no idea why or why not d->buffer should be empty. It seems that Qt buffers the whole file, and my override is invoked again only after EOF is reached. Hopefully somebody can tell me what I am doing wrong to accomplish my task.

Update: opening the stream with QIODevice::Unbuffered seems to make that example working. So is it a bug then?

VRonin

Looking at the source readLine does the work in itself on buffered data in the snipped below so you'd need to reimplement that too.

if (readSoFar) {
#if defined QIODEVICE_DEBUG
        printf("%p \tread from buffer: %lld bytes, last character read: %hhx\n", this,
               readSoFar, data[readSoFar - 1]);
        debugBinaryString(data, int(readSoFar));
#endif
        if (data[readSoFar - 1] == '\n') {
            if (d->openMode & Text) {
                // QRingBuffer::readLine() isn't Text aware.
                if (readSoFar > 1 && data[readSoFar - 2] == '\r') {
                    --readSoFar;
                    data[readSoFar - 1] = '\n';
                }
            }
            data[readSoFar] = '\0';
            return readSoFar;
        }
    }

On top of that, since you are reading text you should use something that is codec aware so QTextStream the worst part is that this class handles the line end in the private API so there is no way to overload it other then recompiling Qt.

Probably the fast solution is QTextStream::read(1) and scan the read character