Qt Forum

    • Login
    • Search
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Search
    • Unsolved

    Update: Forum Guidelines & Code of Conduct

    Solved reading file line by line crashes

    General and Desktop
    qt 4.8 qtextstream readline
    5
    17
    2021
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • M
      mourad_bilog last edited by

      Hello everyone,

      I've a program that open a file (380 Mo) into a QTextStream and read data line by line with the readLine function.
      Recenly a have encoutred a crash problem when reading data and I've tried some manipulation by the crash persists.
      Note that every line contains some big data and lines ends with \n\r.

      Below my code :

      QFile srfile(mScriptFileName);  
      
      	if (!srfile.open(QIODevice::ReadOnly | QIODevice::Text))
      	{
      		mExistMajData = false;	
      		mIsError = true;
      		return ;
      	}
      
      	QTextStream in(&srfile);
      
      	while (!in.atEnd() && !mIsError) 
      	{
      		qApp->processEvents(QEventLoop::WaitForMoreEvents);
      
      		//lecture des données lignes par ligne		
      		try 
      		{ 
      			line_data = in.readLine(); // it crashes here (eg. PJ)
      			record_data = line_data.split("<|>");
      			qDebug() << "Fin lecture d'une ligne n°" << QString::number(++nb_cmd) << record_data[3] << QTime::currentTime().toString("hh:mm") << endl ;
      		} 
      		catch ( const std::bad_alloc & ) 
      		{ 
      			 std::cerr << "Erreur : mémoire insuffisante.\n"; 
      		} 
      		catch ( const std::out_of_range & ) 
      		{ 
      			 std::cerr << "Erreur : débordement de mémoire.\n"; 
      		} 
      		catch ( const std::exception & e ) 
      		{ 
      			std::cerr << e.what(); 
      		}
      

      The call stack indicated the following :

       	msvcr80d.dll!memcpy(unsigned char * dst=0x28446f80, unsigned char * src=0x01b26ffa, unsigned long count=32768)  Ligne 188	Asm
       	QtCored4.dll!QString::append(const QString & str={...})  Ligne 1244 + 0x2d octets	C++
       	QtCored4.dll!QString::operator+=(const QString & s={...})  Ligne 225 + 0x13 octets	C++
       	QtCored4.dll!QTextStreamPrivate::fillReadBuffer(__int64 maxBytes=-1)  Ligne 568 + 0x5b octets	C++
       	QtCored4.dll!QTextStreamPrivate::scan(const QChar * * ptr=0x0014a42c, int * length=0x0014a434, int maxlen=0, QTextStreamPrivate::TokenDelimiter delimiter=EndOfLine)  Ligne 730 + 0x2a octets	C++
       	QtCored4.dll!QTextStream::readLine(__int64 maxlen=0)  Ligne 1588 + 0x16 octets	C++
      >	TheriaqueUpdater.exe!TheriaUpdater::NewUpdateBase()  Ligne 3377 + 0x14 octets	C++
       	TheriaqueUpdater.exe!TheriaUpdater::start()  Ligne 1479	C++
       	TheriaqueUpdater.exe!TheriaUpdater::Update()  Ligne 1199	C++
       	TheriaqueUpdater.exe!TheriaUpdater::qt_metacall(QMetaObject::Call _c=InvokeMetaMethod, int _id=8, void * * _a=0x0014b154)  Ligne 96 + 0x8 octets	C++
       	QtCored4.dll!QMetaObject::activate(QObject * sender=0x01aa4888, int from_signal_index=29, int to_signal_index=30, void * * argv=0x0014b154)  Ligne 3063 + 0x46 octets	C++
       	QtCored4.dll!QMetaObject::activate(QObject * sender=0x01aa4888, const QMetaObject * m=0x657a2374, int from_local_signal_index=2, int to_local_signal_index=3, void * * argv=0x0014b154)  Ligne 3142 + 0x15 octets	C++
       	QtGuid4.dll!QAbstractButton::clicked(bool _t1=false)  Ligne 180 + 0x17 octets	C++
       	QtGuid4.dll!QAbstractButtonPrivate::emitClicked()  Ligne 517	C++
       	QtGuid4.dll!QAbstractButtonPrivate::click()  Ligne 509	C++
       	QtGuid4.dll!QAbstractButton::mouseReleaseEvent(QMouseEvent * e=0x0014b788)  Ligne 1086	C++
       	QtGuid4.dll!QWidget::event(QEvent * event=0x0014b788)  Ligne 6064	C++
       	QtGuid4.dll!QAbstractButton::event(QEvent * e=0x0014b788)  Ligne 1048	C++
       	QtGuid4.dll!QPushButton::event(QEvent * e=0x0014b788)  Ligne 651	C++
       	QtGuid4.dll!QApplicationPrivate::notify_helper(QObject * receiver=0x01aa4888, QEvent * e=0x0014b788)  Ligne 3541 + 0x11 octets	C++
       	QtGuid4.dll!QApplication::notify(QObject * receiver=0x01aa4888, QEvent * e=0x0014b788)  Ligne 3240 + 0x2f octets	C++
       	QtCored4.dll!QCoreApplication::notifyInternal(QObject * receiver=0x01aa4888, QEvent * event=0x0014b788)  Ligne 516	C++
       	QtCored4.dll!QCoreApplication::sendSpontaneousEvent(QObject * receiver=0x01aa4888, QEvent * event=0x0014b788)  Ligne 188 + 0x38 octets	C++
      
      

      I've reinstalled the VC redistribuable. Rebuil all the project. Search if I can free QTextStream buffer but the problem persists

      Can anyone tell me what wrong et what I must to do to resolve this. Some

      Thanks in advance

      jsulm J.Hilk 2 Replies Last reply Reply Quote 0
      • jsulm
        jsulm Lifetime Qt Champion @mourad_bilog last edited by

        @mourad_bilog said in reading file line by line crashes:

        crash problem

        What kind of crash is it? Out of memory? Or something else?

        https://forum.qt.io/topic/113070/qt-code-of-conduct

        1 Reply Last reply Reply Quote 0
        • J.Hilk
          J.Hilk Moderators @mourad_bilog last edited by

          hi @mourad_bilog

          Even if this doesn't crash your application, you should check if recordData has indeed more than 3 entries!

          qDebug() << "Fin lecture d'une ligne n°" << QString::number(++nb_cmd) << record_data[3] << QTime::currentTime().toString("hh:mm") << endl ;

          besides that, I see this

          qApp->processEvents(QEventLoop::WaitForMoreEvents);

          potentially causing serious issues, do not spin the event loop manually!

          Be aware of the Qt Code of Conduct, when posting : https://forum.qt.io/topic/113070/qt-code-of-conduct

          Qt Needs YOUR vote: https://bugreports.qt.io/browse/QTQAINFRA-4121


          Q: What's that?
          A: It's blue light.
          Q: What does it do?
          A: It turns blue.

          1 Reply Last reply Reply Quote 2
          • M
            mourad_bilog last edited by

            First, Thanks for reply.

            @jsulm it's a memory crash. Below the error occured :

            Exception de première chance à 0x0fcbd5be (msvcr80d.dll) dans TheriaqueUpdater.exe : 0xC0000005: Violation d'accès lors de l'écriture à l'emplacement 0x2804e000.
            Exception non gérée à 0x0fcbd5be (msvcr80d.dll) dans TheriaqueUpdater.exe : 0xC0000005: Violation d'accès lors de l'écriture à l'emplacement 0x2804e000.
            

            @J-Hilk the crash occus on in.readLine. The recod data has 6 entries and it's shown evey time.
            I've commented the eventLoop but the application still crashes.

            Note that line data lengths can be around 1 milion carachtars.

            jsulm J.Hilk 2 Replies Last reply Reply Quote 0
            • jsulm
              jsulm Lifetime Qt Champion @mourad_bilog last edited by

              @mourad_bilog said in reading file line by line crashes:

              Note that line data lengths can be around 1 milion carachtars

              How long is the line you're reading when the app is crashing?

              https://forum.qt.io/topic/113070/qt-code-of-conduct

              1 Reply Last reply Reply Quote 0
              • J.Hilk
                J.Hilk Moderators @mourad_bilog last edited by

                @mourad_bilog said in reading file line by line crashes:

                @J-Hilk the crash occus on in.readLine. The recod data has 6 entries and it's shown evey time.

                do not base this solely on faith:

                if(record_data.size()> 3)
                qDebug() << "Fin lecture d'une ligne n°" << QString::number(++nb_cmd) << record_data[3] << QTime::currentTime().toString("hh:mm") << endl ;
                 else
                 qDebug() << "would have crashed";
                

                Be aware of the Qt Code of Conduct, when posting : https://forum.qt.io/topic/113070/qt-code-of-conduct

                Qt Needs YOUR vote: https://bugreports.qt.io/browse/QTQAINFRA-4121


                Q: What's that?
                A: It's blue light.
                Q: What does it do?
                A: It turns blue.

                1 Reply Last reply Reply Quote 2
                • M
                  mourad_bilog last edited by

                  @jsulm said in reading file line by line crashes:

                  How long is the line you're reading when the app is crashing?

                  About 1 200 000 charactars.

                  1 Reply Last reply Reply Quote 0
                  • M
                    mourad_bilog last edited by

                    @J-Hilk
                    I've commented that line but the same occurs.

                    J.Hilk 1 Reply Last reply Reply Quote 0
                    • J.Hilk
                      J.Hilk Moderators @mourad_bilog last edited by

                      @mourad_bilog alight thanks thumbs up

                      question , is this Qt4?
                      and is this a multi threaded project, and is do you read and write to that file from different places ?

                      Be aware of the Qt Code of Conduct, when posting : https://forum.qt.io/topic/113070/qt-code-of-conduct

                      Qt Needs YOUR vote: https://bugreports.qt.io/browse/QTQAINFRA-4121


                      Q: What's that?
                      A: It's blue light.
                      Q: What does it do?
                      A: It turns blue.

                      M 1 Reply Last reply Reply Quote 0
                      • M
                        mourad_bilog @J.Hilk last edited by

                        @J.Hilk
                        question , is this Qt4? yes. but the program runs correctly for some years.

                        and is this a multi threaded project, : No, it's one thread.

                        and is do you read and write to that file from different places : No, I just read from that file. I just open it in "ReadOnly" mode.

                        1 Reply Last reply Reply Quote 0
                        • M
                          mourad_bilog last edited by

                          I've read in a post how has similar problem that he has solved the problem using file.readLine no QTextStream.readLine(). What is the difference between the 2 functions ?

                          1 Reply Last reply Reply Quote 0
                          • Christian Ehrlicher
                            Christian Ehrlicher Lifetime Qt Champion last edited by

                            @mourad_bilog said in reading file line by line crashes:

                            1 200 000

                            really one line?

                            Qt has to stay free or it will die.

                            1 Reply Last reply Reply Quote 0
                            • M
                              mourad_bilog last edited by

                              @Christian-Ehrlicher there's many lines witch everyone exceed 1 milion charactars.
                              Is this a problem ? That can exceed what QTextStream buffer contains ?

                              1 Reply Last reply Reply Quote 0
                              • Christian Ehrlicher
                                Christian Ehrlicher Lifetime Qt Champion last edited by

                                At least I would consider this - it's a 2.4MB huge QString, should be ok but who knows. I would simply try QFile::readLine(), split the QByteArray and convert to QString as late as possible (if needed at all)

                                Qt has to stay free or it will die.

                                1 Reply Last reply Reply Quote 2
                                • M
                                  mourad_bilog last edited by

                                  Checking Windows events journal, it indicates that msvcr causes this crash, that's true ?

                                  Nom de l’application défaillante Updater.exe, version : 2.7.4.0, horodatage : 0x5a018f88
                                  Nom du module défaillant : MSVCR80.dll, version : 8.0.50727.9445, horodatage : 0x5a7bc74c
                                  Code d’exception : 0xc0000005
                                  Décalage d’erreur : 0x0001508e
                                  ID du processus défaillant : 0x19a0
                                  Heure de début de l’application défaillante : 0x01d4d8a5bb99dd52
                                  Chemin d’accès du module défaillant: C:\WINDOWS\WinSxS\x86_microsoft.vc80.crt_1fc8b3b9a1e18e3b_8.0.50727.9445_none_d08c58b4442ba54f\MSVCR80.dll
                                  ID de rapport : 6c470b7e-a3b2-4584-b404-3d7957545ead
                                  Nom complet du package défaillant : 
                                  ID de l’application relative au package défaillant : 
                                  

                                  I've re-installed Visual redistribuable 2005 but the problem persists. Note that I dont found msvcr80 in the system32 not like other versions msvcr71, msvcr10, etc.

                                  Does this the problem ?

                                  1 Reply Last reply Reply Quote 0
                                  • SGaist
                                    SGaist Lifetime Qt Champion last edited by

                                    Hi,

                                    Did you monitor your application RAM usage ?

                                    Interested in AI ? www.idiap.ch
                                    Please read the Qt Code of Conduct - https://forum.qt.io/topic/113070/qt-code-of-conduct

                                    1 Reply Last reply Reply Quote 1
                                    • M
                                      mourad_bilog last edited by

                                      Finally I changed reading mode to read directly from the data file. It's most faster and just I was care to encode data when reading to keep special charcatars.

                                      Thanks all for your ideas

                                      1 Reply Last reply Reply Quote 0
                                      • First post
                                        Last post