[SOLVED] Qt and CLucene



  • Dear all

    I am not certain if this is a Qt problem but if anybody knows what the problem might be I would be very grateful. I have the following problem. When the application starts, the method loadIndex is called and this method tries to load an existing index. This method works fine when there is an index and when there is no index. However during the execution of the program, the method startIndexing can be called. When a new IndexWriter is created I get the following message:

    @Exception at 0x75ccc41f, code: 0xe06d7363: C++ exception, flags=0x1 (execution cannot be continued) (first chance) at c:\users\clucene-core-2.3.3.4\clucene-core-2.3.3.4\src\core\clucene\store\directory.cpp:54@

    However when I check the index directory, I see that the index is being generated by the method startIndexing. However when the execution flow reaches the statement

    @IndexReader* r = IndexReader::open(indexDirectory.toStdString().c_str());@

    the following message is shown:

    @:0: error: Exception at 0x56e37bea, code: 0xc0000005: read access violation at: 0x0, flags=0x0 (first chance)@

    I have no idea what the problem can be. I use Qt 5.1, Qt creator and MSVC 2010. I compiled the CLucene code as a DLL to which I link in the application.

    @void IndexingEngine::loadIndex(Dataset *dataset, QString indexDirectory)
    {
    QTime timer;
    timer.start();

    try{
        if ( IndexReader::indexExists(indexDirectory.toStdString().c_str()) ){
            if ( IndexReader::isLocked(indexDirectory.toStdString().c_str()) ){
                printf("Index was locked... unlocking it.\n");
                IndexReader::unlock(indexDirectory.toStdString().c_str());
            }
        }
        else
            return;
    
        dataset->setIndexDirectory(indexDirectory);
        QStringList *termsInIndex = new QStringList();
    
        IndexReader* r = IndexReader::open(indexDirectory.toStdString().c_str());
    
        TermEnum* te = r->terms();
        int32_t nterms;
        for (nterms = 0; te->next() == true; nterms++) {
            termsInIndex->append(QString::fromStdWString(te->term()->text()));
        }
    
        _CLLDELETE(te);
    
        r->close();
        _CLLDELETE(r);
    
    }catch(CLuceneError& err){
        printf("Error: %s\n", err.what());
    }catch(...){
        printf("Unknown error\n");
    }
    

    }

    void IndexingEngine::startIndexingTXT(Dataset dataset, bool clearIndex, QString indexDirectory)
    {
    int i=0;
    QTime timer;
    timer.start();
    const char
    target = indexDirectory.toStdString().c_str();

    IndexWriter* writer = NULL;
    lucene::analysis::WhitespaceAnalyzer an;
    
    try{
        if ( !clearIndex && IndexReader::indexExists(target) ){
            if ( IndexReader::isLocked(target) ){
                printf("Index was locked... unlocking it.\n");
                IndexReader::unlock(target);
            }
    
            writer = _CLNEW IndexWriter( target, &an, false);
        }else{
            writer = _CLNEW IndexWriter( indexDirectory.toStdString().c_str() ,&an, true);
        }
    
        writer->setMaxFieldLength(0x7FFFFFFFL); // LUCENE_INT32_MAX_SHOULDBE
    
        // Turn this off to make indexing faster; we'll turn it on later before optimizing
        writer->setUseCompoundFile(false);
    
        {
            QDirIterator it2 (dataset->getDatasetDirectory(), QDir::NoDotAndDotDot | QDir::Files | QDir::Dirs);
            while (it2.hasNext())
            {
                it2.next();
                QFile file;
                file.setFileName(it2.filePath());
                file.open(QIODevice::ReadOnly | QIODevice::Text);
    
                QString filename = it2.fileName();
                QString contents = file.readAll();
    
                // Re-use the document object
                Document doc;
    
    
                doc.clear();
                doc.add( *_CLNEW Field(_T("filename"), filename.toStdWString().c_str(), Field::STORE_YES | Field::INDEX_UNTOKENIZED ) );
                doc.add( *_CLNEW Field(_T("contents"), contents.toStdWString().c_str(), Field::STORE_YES | Field::INDEX_TOKENIZED) );
    
                writer->addDocument( &doc );
                i++;
    
                file.close();
            }
        }
    
        // Make the index use as little files as possible, and optimize it
        writer->setUseCompoundFile(true);
        writer->optimize();
    
        // Close and clean up
        writer->close();
        _CLLDELETE(writer);
    
        dataset->setIndexDirectory(indexDirectory);
        QStringList *termsInIndex = new QStringList();
    
        IndexReader* r = IndexReader::open(indexDirectory.toStdString().c_str());
    
        TermEnum* te = r->terms();
        int32_t nterms;
        for (nterms = 0; te->next() == true; nterms++) {
            termsInIndex->append(QString::fromStdWString(te->term()->text()));
        }
    
        _CLLDELETE(te);
    
        r->close();
        _CLLDELETE(r);
    
        int elapsedTime = timer.elapsed();
        statusNotification->setText(QString::number(i) + " reports indexed, " + QString::number(nterms) +
                                    " unique terms, " +QTime(0,0,0,0).addMSecs(elapsedTime).toString("hh:mm:ss.zzz")+" time elapsed");
        progress->setValue(100);
    
    }catch(CLuceneError& err){
        printf("Error: %s\n", err.what());
    }catch(...){
        printf("Unknown error\n");
    }
    getStats(indexDirectory.toStdString().c_str());
    
    _lucene_shutdown(); //clears all static memory
    

    }
    @


  • Lifetime Qt Champion

    Hi,

    Did you try to run it with the debugger ? That should give you more information



  • Dear Sir

    All information which I obtained with the debugger is displayed in the post ...
    Unless you have any idea for obtaining additional information ..?

    Kind regards
    Hatchi


  • Lifetime Qt Champion

    The only information ? Did you run it with a debug build ?




  • Lifetime Qt Champion

    Can you provide the backtrace ?


  • Lifetime Qt Champion

    Just a quick question, since IndexReader::open takes a const char * (and probably the other functions too) why not use something like:

    @QByteArray asciiPath = indexDirectory.toAscii(); // also used in toStdString();
    const char * path = asciiPath.constData(); << this will ensure that path will point to valid buffer.
    @
    ?



  • Dear Sir

    I tried your latest suggestion but I still received the same errors. I am pretty sure something is going wrong inside the CLucene library itself, however when I tried to put breakpoints in the CLucene source code which is in the include path of my application, the debugger doesn't go to here. I compiled CLucene from source and linked in debug modus with the debug version of the DLL. Do you have any suggestion for obtaining the call trace inside this DLL?

    Kind regards
    Hatchi


  • Lifetime Qt Champion

    You could already try to get a backtrace from the debug console in QtCreator.

    I am however surprised that you can't get in the library, are you sure that your program links agains the debug version of CLucene ?



  • Dear Sir

    This is the backtrace I got from Qt creator:

    0 QAbstractScrollArea::setHorizontalScrollBarPolicy Qt5Widgetsd 0x5ebe7bea
    1 QAbstractScrollArea::setHorizontalScrollBarPolicy Qt5Widgetsd 0x5ed03f4b
    2 QAbstractScrollArea::setHorizontalScrollBarPolicy Qt5Widgetsd 0x5ecf86c3
    3 QAbstractScrollArea::setHorizontalScrollBarPolicy Qt5Widgetsd 0x5edf4eb1
    4 IndexingEngine::startIndexingTXT indexingengine.cpp 207 0xb6a2e4
    5 TextBrowser::indexDocuments textbrowser.cpp 248 0xb62b67
    6 TextBrowser::qt_static_metacall moc_textbrowser.cpp 140 0xb90424
    7 QObject::destroyed Qt5Cored 0x5df9b210
    8 QObject::destroyed Qt5Cored 0x5df9ac41
    9 QAbstractScrollArea::setHorizontalScrollBarPolicy Qt5Widgetsd 0x5ebd862b
    10 QAbstractScrollArea::setHorizontalScrollBarPolicy Qt5Widgetsd 0x5ebd7d07
    11 QAbstractScrollArea::setHorizontalScrollBarPolicy Qt5Widgetsd 0x5ebcf411
    12 QAbstractScrollArea::setHorizontalScrollBarPolicy Qt5Widgetsd 0x5ee35a18
    13 QAbstractScrollArea::setHorizontalScrollBarPolicy Qt5Widgetsd 0x5ed37dc2
    14 QAbstractScrollArea::setHorizontalScrollBarPolicy Qt5Widgetsd 0x5ed38e82
    15 QAbstractScrollArea::setHorizontalScrollBarPolicy Qt5Widgetsd 0x5ee34ab0
    16 QAbstractScrollArea::setHorizontalScrollBarPolicy Qt5Widgetsd 0x5ec39db5
    17 QAbstractScrollArea::setHorizontalScrollBarPolicy Qt5Widgetsd 0x5ed38ca1
    18 QAbstractScrollArea::setHorizontalScrollBarPolicy Qt5Widgetsd 0x5ee35a7b
    19 QAbstractScrollArea::setHorizontalScrollBarPolicy Qt5Widgetsd 0x5ebf233e
    ... <More>

    I could get info from the library in the stack trace but as soon as execution reaches the point where it always crashes, this is the only information ...

    Would it be useful if I would send you the dll and some accompanying code to try and test it yourself?

    Hatchi


  • Lifetime Qt Champion

    One other thing you might want to try first is to build the clucene code along your application (you can have a look at fulltextsearch.pri to see what files to build.



  • Thank you for this suggestion. This solved the problem. I see that Qt uses an older version of CLucene while I was using the newest unstable version.

    Thanks!


  • Lifetime Qt Champion

    You're welcome !

    Don't forget to update the thread title to solved so other forum users will know that a solution have been found :)


Log in to reply
 

Looks like your connection to Qt Forum was lost, please wait while we try to reconnect.