QProcess have conflics on program.

[darongyi]

I run QThread when program start first.
After running network process, my process works on mainwindow function.
But when I QProcess start one command,

nw_path_close_fd Failed to close guarded necp fd 21 [9: Bad file descriptor]
--> this log show on debugging mode.
My code is below.

LOG

"-ccurl \"http://dbpedia.org/data/Haeinsa.json\""
2018-09-26 17:19:48.167799+0900 CRIClient[3645:155073] nw_path_close_fd Failed to close guarded necp fd 21 [9: Bad file descriptor]
ParseData
"Location"
"-ccurl \"http://dbpedia.org/data/Haeinsa.json\" | perl /Users/yoshimi/Downloads/dbpediaPlaceData.txt 'Haeinsa'"

**2018-09-26 17:20:13.903292+0900 CRIClient[3647:155302] nw_path_close_fd Failed to close guarded necp fd 21 [9: Bad file descriptor]**
"-ccurl \"https://en.wikipedia.org/api/rest_v1/page/media/Haeinsa\" | perl /Users/yoshimi/Downloads/WikiJSONimages.txt"
2018-09-26 17:20:22.820658+0900 CRIClient[3650:155442] nw_path_close_fd Failed to close guarded necp fd 21 [9: Bad file descriptor]
"-ccurl \"https://en.wikipedia.org/w/index.php?title=Haeinsa&action=render\" | perl /Users/yoshimi/Downloads/WikiHTMLcleaner.txt"
2018-09-26 17:20:28.689699+0900 CRIClient[3653:155502] nw_path_close_fd Failed to close guarded necp fd 21 [9: Bad file descriptor]
max processes reached! Stopping timer!

Worker (To run command)

#include "worker.h"
#include "parser_definitions.h"
#include "processutils.h"


using namespace ParserNS;
Worker::Worker(QObject *parent) : QObject(parent)

{
    m_mutex = new QMutex();
    m_timer = new QTimer();
    m_timer->setInterval(100);
    connect(m_timer, &QTimer::timeout, this, QOverload<>::of(&Worker::doWork));


}

void Worker::setDefaultValue(QString keyword, QString subtype, QString recordtype)
{
    mKeyword = keyword;
    mSubtype = subtype;
    mName    = "";
    mRecordtype =recordtype;
    no_process= 0;
    max_process= 3;
    rflag = false;

    setDefaultPath();

}

void Worker::setDefaultPath()
{

   QString basicPath = QStandardPaths::writableLocation(QStandardPaths::DownloadLocation);
   dbpediaPlaceFile  = QString("%1%2%3").arg(basicPath, QDir::separator(),"dbpediaPlaceData.txt");
   wikipediaHTMLFile = QString("%1%2%3").arg(basicPath,QDir::separator(),"WikiHTMLcleaner.txt");
   wikiJSONImagesFile = QString("%1%2%3").arg(basicPath,QDir::separator(),"WikiJSONimages.txt");
}

void Worker::start()
{
    m_timer->start();
}

void Worker::doWork()
{
    this->setAbort(false);

    ProcessUtils pu;
    QString readOut = "";
    QStringList args;
    QString progam = "/bin/sh";
    QObject::connect(& pu, SIGNAL(showErrorMessage(QString)), this, SIGNAL(showErrorMessage(QString)));


    QString command = "";

    QString keyword = ParserNS::convertTitle(mKeyword);


    if(no_process == 0 )   // bring subtype from dbpedia JSON
    {
        mName= "dbpediaplace";
       if(mSubtype.isEmpty())
          command  = QString("curl \"http://dbpedia.org/data/%1.json\"").arg(keyword);
        else
           no_process++;

       args << "-c" << command;

       readOut = pu.runCommand(progam,args);
       copyvalue(readOut);
        no_process++;
    }

    if(no_process == 1)
    {
        args.clear();
         if(mRecordtype == ParserNS::JsonItem::Recordtype::THING)
        {
           mName =  QFileInfo(dbpediaPlaceFile).fileName().toLower();
           command  = QString("curl \"http://dbpedia.org/data/%1.json\""
                         " | perl %2 '%3'").arg(keyword).arg( dbpediaPlaceFile).arg(mKeyword);

         args << "-c" << command;
          readOut = pu.runCommand(progam,args);
          copyvalue(readOut);
          no_process++;


         }
        else if(mRecordtype == ParserNS::JsonItem::Recordtype::MEDIUM
                || mRecordtype == ParserNS::JsonItem::Recordtype::EVENT)

        {
            no_process++;
        }

   }

    if(no_process ==2)
   {
        args.clear();
       mName =  QFileInfo(wikiJSONImagesFile).fileName().toLower();
       command = QString("curl \"https://en.wikipedia.org/api/rest_v1/page/media/%1\""
                         " | perl %2").arg(keyword).arg(wikiJSONImagesFile);

       args << "-c" << command;
       readOut = pu.runCommand(progam,args);
       copyvalue(readOut);
       no_process++;


   }
   else if(no_process == 3)
   {
       args.clear();
       mName =  QFileInfo(wikipediaHTMLFile).fileName().toLower();
       command = QString("curl \"https://en.wikipedia.org/w/index.php?title=%1&action=render\""
                          " | perl %2").arg(keyword).arg(wikipediaHTMLFile);

       args << "-c" << command;
       readOut =pu.runCommand(progam,args);
       copyvalue(readOut);


    }

    if (no_process <= max_process && (rflag == true))
   {
       qDebug() << "max processes reached! Stopping timer!";
       no_process = 0;
       m_timer->stop();


     }
}

void Worker::copyvalue(QString read)
{
    const QString resultString = read;

    QString nKind = mName;

    if(nKind.startsWith("dbpediaplace"))
    {
        if(no_process == 0 && mSubtype.isEmpty())
          result.strSubtype = parseData(resultString);
        else
          result.strdbpediaPlace = resultString;

        rflag = false;

    }
    else if(nKind.startsWith("wikihtmlcleaner"))
    {
        result.strWikipedia = resultString;
        rflag = true;
    }
    else
    {
        result.strWikiJson = resultString;
        rflag = false;
    }



}

ProcessUtils (run process)

QString ProcessUtils::runCommand(QString cmd,QStringList commands)
{
    QProcess p;

    qDebug() << commands.join("");
    p.start(cmd,commands);

    bool finished = p.waitForFinished(-1);

    this->setStdOut(p.readAllStandardOutput());
    this->setStdErr(p.readAllStandardError());
    this->setExitCode(p.exitCode());

    QStringList report;
    if(!finished && p.error() == QProcess::Timedout)
    {
        report << "Command failed to finish:"
               << "\"" + cmd + "\"";
        p.kill();
        if(exitCode() == 0)
            this->setExitCode(-1);
    }
    else if(!m_stdErr.isEmpty() || p.exitCode() != 0 || !finished)
    {
        report << "Command exited with errors:"
               << "\"" + cmd + "\""
               << "exit code: " + QString::number(p.exitCode())
               << "error string: " + p.errorString()
               << "stdErr: " + m_stdErr;
        if(exitCode() == 0)
            this->setExitCode(-1);
    }
//    if(report.size() > 0)
//        emit showErrorMessage(report.join("\n\n"));

    return m_stdOut;
}

Mainwindow

void MainWindow::runProcessWorker(RecordItem *mRecordItem)
{

    Worker *worker = new Worker;

    worker->setDefaultValue(mRecordItem->keyword(),
                            mRecordItem->subtype(), mRecordItem->recordtype());


    // move thread from CRIClientThread
    worker->moveToThread(criclientObj->getThread());

    worker->start();
    connect(&workerThread, SIGNAL(finished()), this, SLOT(onWorkFinished()));
    connect(&workerThread, SIGNAL(finished()), worker, SLOT(deleteLater()));

    workerThread.start();
}

[JonB]

@darongyi
I don't know where exactly the error message emanates from or what you are saying. But I don't know if it's good to move your QProcess over to another thread and then have the original main thread execute the calls to wait for it to finish etc.

In any case, what is the logic of what you are doing here, what are you trying to achieve? There seems no point in moving the process starter into its own thread if the calling main thread then just waits for it to finish!

A process spawned by QProcess is already running in its own independent process anyway, so what is the point in pushing it into its own thread?

Assuming your intention is something like "run the process without blocking the main UI thread and collect its output", why not get rid of your synchronous waiting calls waitForBytesWritten() & waitForFinished() and instead use QProcess's various signals/slots to execute asynchronously and receive the output as & when it arrives? That is the usual way to write this kind of thing.

P.S.
Is your command of /bin/sh -c ls just a placeholder for the command you actually want to issue? Because if it's the real command, to get the contents of a directory you don't want to issue an OS ls command, you're much better just using http://doc.qt.io/qt-5/qdir.html#entryInfoList-1 to achieve this....

[darongyi]

I think my mac have problem. I change code.