ASSERT on QString line 885 return false size by read line by line of a 125MB file.



  • Hi is this my error... the function get the correct value..
    but on big file test file i have a error from QString size...

    this file 125mb have all Unicode data to test app...

    <p/>ASSERT: "uint(i) < uint(size())" in file /Users/dev/Applications/qt5.9.1/lib/QtCore.framework/Headers/qstring.h, line 885 QString(int size, Qt::Initialization);
    Abort trap: 6

    //// count number in array mixed...
    static inline int maxscript(int arr[], int stop, int num)
    {
        int result = 0;
        for(int i=0; i<stop; i++)
        {
              if (arr[i] == num) {
                result++;
              }
        }
        return result;
    }
    //// statistic script() from QChar
    int  hit_Script_found( QString text ) {
        const int nox = text.size();
        int *array = new int[nox];
        for (int o = 0; o < text.size(); o++) {
          const QChar vox(text.at(o));
          const int typsc = (int)vox.script();
          array[o]=typsc;
         }
        std::map <int,int, std::greater < int > > mhit;
        int NrScript =2; /// start from latin & go up..
        int hitmaxnumber =0;
        do {
              NrScript++;
                  int hits = maxscript(array,nox,NrScript);
                    if (hits > 0) {
                        hitmaxnumber = qMax(hitmaxnumber,hits);
                        mhit.insert ( std::pair<int,int>(hits,NrScript) );
                    }
        }while(NrScript < (sumscritpsLang -2)); /// sum of avaiable script
        std::map<int,int>::iterator it = mhit.begin();
        for (it=mhit.begin(); it!=mhit.end(); ++it) {
            if (it->first == hitmaxnumber) {
                //// qDebug() << "hit is on by:" << it->second;
                return it->second;
              }
          }
       return -1;
       //// qDebug() << "found:" << hits << " by:" << NrScript;
    }
    
    

  • Qt Champions 2016

    @patrik08 said in ASSERT on QString line 885 return false size by read line by line of a 125MB file.:

    ASSERT: "uint(i) < uint(size())

    Yes its not very informative but it means that you ask it to go beyond its max index
    like
    QString g;
    g[888888] = "";

    Also just as a note
    int *array = new int[nox];

    Do you delete that again ?



  • @mrjj no ... not delete the array at end of use... :-(
    tanks...


  • Qt Champions 2016

    @patrik08

    You can avoid it all together with

    std::vector<int> array;
    array.reserve(nox);

    unless you have a really good reason to new it.



  • Super tanks... now 125mb file iterate line by line in 30sec...
    only delete[] array; ... i try the vector int if is faster...


  • Qt Champions 2016

    @patrik08
    It wont be faster. Its just offer other benefits, like easy to send as parameter, it knows its
    own size. Multiple way to loop over it. etc. On big lists you can use move feature to avoid copy
    so its good to just use instead of the more traditional [] array. (IMO)


  • Moderators

    Have you considered reading the file line by line and not storing the whole 125MB in a string? Or using the stream approach to reading the file? You would save loads of RAM and CPU cycles this way.



  • Hi ... oh yes i read line by line.. each line have other language to detect.. wich so much text in over 200 or more language...
    google the file name test_shuffle_1000_48_666.utf8.gz
    i tested for https://github.com/pehohlva/QCLD2 for language detection ..
    the error was unclosed array cycle that fill ram... not QString... have incremented all line... :-) & must go in assert -


Log in to reply
 

Looks like your connection to Qt Forum was lost, please wait while we try to reconnect.