Reading file with QTextStream looping over file



  • Situation:
    In my program, I want to read a file. So I put my file in a QTextStream.
    @if(file.open(QIODevice::ReadOnly | QIODevice::Text))
    {
    QTextStream in(&file);
    in.setAutoDetectUnicode(true);

        while(!in.atEnd())
        {
            progressDialog.setValue(in.pos());
            QString line = QString();
            line = QString(in.readLine());
            if (!line.isEmpty())
            {
                 ...
            }
        }
    

    }@

    Problem:
    The file loops over the first 154 lines (of 235). And it doesn't stop.
    So it reads 65% of my file, loops over that 65% and just skips the other 35%.

    Anyone got an idea about this problem?



  • Is there anything special at line 154? Does it happen with any file?



  • I tried it with 2 files, one where he got to 87% and one to 65%

    There is nothing special on that line, I even tried to delete line 153, 154 and 155, but nothing else happened.

    I can't post the file, cause it contains confedential information. A screenshot with dummytext is possible:
    !http://wouterverbruggen.be/dummy.jpg(Dummy.jpg)!

    Even if I copy those 3 line to the first line, it does the same.



  • Try to remove QIODevice::Text option.



  • Just tried it and isn't really working.

    If I don't put it in a QTextStream it works, but I need special characters to be read. So I need
    @in.setAutoDetectUnicode(true);@



  • If you expect to have unicode characters why don't you set codec to one of the utf's? Try "setting codec":http://qt-project.org/doc/qt-4.8/qtextstream.html#setCodec for your QTextStream object to UTF-8. Unless you set one globally before...



  • Can you send me your file 'as is'.



  • That doesn't help with the fact I don't have to use a QTextStream. I think (read: I'm quiet sure) that the problem has to do with the QTextStream. But where that is, I really haven't got a clue.

    Edit: I can't really send you the file, sorry. But there isn't really something wrong with the line, nor the file itself.



  • Ok, maybe, you may post which special characters appear in the file?



  • In this file the only special characters are ë (on Daniël, Joëlle, ...), é (on Conté, Romanée, ...), ö (on Buddenböhmer), ü (on Büller)

    But none of these characters are on line 153, 154, 155, 156



  • Does this:
    @
    while(true)
    qDebug() << in.readLine();
    @

    read past that magic line 154?
    Have you validated the strings that readLine returns? Maybe 154 is actually the last line in the file because some apparent multiple lines in the file are read joined together (for whatever crazy unicode reason).

    bq. I tried it with 2 files, one where he got to 87% and one to 65%

    I suppose the other file was also such a name list with that (maybe broken) unicode encoding. Do the following:
    Open a text editor and type by hand some lines that reproduce the visual appearance (i.e. which characters come when) in one of the unicode files. This is to make sure there aren't any strange unicode characters inside. Then either compare them in a hex editor or just try to read that hand-crafted file (maybe multiply the lines by copy-pasting to 200 or so).
    Now that should absolutely work, because your code is very simple. And if it does, you know that the original files contain crazy unicode control characters that need to be rectified first. It may be an effect due to the automatic unicode detection. If it's not detected correctly, who knows what happens. Why don't you set the codec yourself as proposed already?



  • Do you sure the file is really unicode? It seems you have ASCII file that detected by QTextCodec as UTF-8.



  • I don't think my decoding is the problem here.
    If I don't try to decode it, it does exactly the same as with the autoDetect and the setCodec("utf-8").

    I'll try to write some of it again, but that's not a solution for 30k lines-file :)



  • Going to update Qt, I know have 4.7.4, my colleague has 4.8.2.
    And on his PC it works fine.



  • It seems you may have been affected by bug "QTBUG-9814":https://bugreports.qt-project.org/browse/QTBUG-9814.



  • Mmh that indeed could be possible. Thnx for the tip!



  • It is 'solved'. I installed Qt 4.8.2 and it does it's job just fine. Takes the whole file and treats every line properly.

    I don't think I need to put [SOLVED] in the title cause the problem in 4.7.4 isn't solved.


Log in to reply
 

Looks like your connection to Qt Forum was lost, please wait while we try to reconnect.