Important: Please read the Qt Code of Conduct - https://forum.qt.io/topic/113070/qt-code-of-conduct

after deploying on RPI3 , UploadCSV file take very long time



  • Hello ,
    I tried to load a CSV file in my App . So I connected a button signal(clicked) to Slot which load a CSV file .
    After deploying my App on Rpi3 target .

    when I click on the button , it takes very long time to load the CSV (about 6 minutes) .
    but finally it load the csv correctly !

    Is it software problem ?

    thank you


  • Lifetime Qt Champion

    Hi,

    You have to give more information:

    • which size is that file ?
    • are you doing any processing on the data ?
    • what are you doing with the data ?
    • which version of Qt are you using ?
    • did you do any benchmarking of your code ?
    • what does the Pi do when you load the file ?


  • @SGaist I think the problem is caused by doing processing on the data .
    this is my code :

     while(!inFile.atEnd()){
            QString readedLine =inFile.readLine();
            QStringList listValue = readedLine.split(";");
            cter +=1;
    
            ui->tablewidget2->setRowCount(cter);
    
            ui->tablewidget3->setRowCount(cter);
    
            ui->tablewidget->setRowCount(cter);
            ui->tablewidget4->setRowCount(cter);
            ui->tablewidget5->setRowCount(cter);
    
    
    
            for(int k=0;k<listValue.size();k++){
                QTableWidgetItem     *Mytable = new QTableWidgetItem(listValue[k]);
                ui->tablewidget->setItem(cter-1,k,Mytable);
                ui->tablewidget->hideColumn(0);
                ui->tablewidget->setItem(cter-1, 8, new QTableWidgetItem(""));
                ui->tablewidget->setItem(cter-1, 9, new QTableWidgetItem(""));
                if(k<9)   Mytable->setFlags(Mytable->flags() ^ Qt::ItemIsEditable);// modif la valeur du k pour rendre le champ editable
    
    
               //setpoint table
                for(int t=10;t<551;t++){
                    QTableWidgetItem     *Mysetpoint = new QTableWidgetItem(listValue[k]);
    
                    ui->tablewidget2->setItem(cter-1,k,Mysetpoint);
                    ui->tablewidget2->hideColumn(0);
                    ui->tablewidget2->hideRow(t);
                    ui->tablewidget2->setItem(cter-1, 8, new QTableWidgetItem(""));
                    ui->tablewidget2->setItem(cter-1, 9, new QTableWidgetItem(""));
                    if(k<9)   Mysetpoint->setFlags(Mysetpoint->flags() ^ Qt::ItemIsEditable);
    
    
                }
                //setpoint table
    
                for(int a=341;a<549;a++){
                    QTableWidgetItem     *Mytableconfig = new QTableWidgetItem(listValue[k]);
    
                    ui->tablewidget3->setItem(cter-1,k,Mytableconfig);
                    ui->tablewidget3->hideColumn(0);
                    ui->tablewidget3->hideRow(a);
    
                    for(int x=0;x<10;x++){
                        ui->tablewidget3->hideRow(x);
    
                    }
    
    
                    ui->tablewidget3->setItem(cter-1, 8, new QTableWidgetItem(""));
                    ui->tablewidget3->setItem(cter-1, 9, new QTableWidgetItem(""));
                    if(k<9)   Mytableconfig->setFlags(Mytableconfig->flags() ^ Qt::ItemIsEditable);
    
                    //config table
                }
                //calib table
                //for(int m=0;m<342;m++){ for(int y=500;y<549;y++){
                QTableWidgetItem     *Mysetpoint = new QTableWidgetItem(listValue[k]);
    
                ui->tablewidget4->setItem(cter-1,k,Mysetpoint);
                ui->tablewidget4->hideColumn(0);
                for(int m=0;m<341;m++){
                    ui->tablewidget4->hideRow(m);}
                for(int p=500;p<549;p++){
                    ui->tablewidget4->hideRow(p);}
                ui->tablewidget4->setItem(cter-1, 8, new QTableWidgetItem(""));
                ui->tablewidget4->setItem(cter-1, 9, new QTableWidgetItem(""));
                if(k<9)   Mysetpoint->setFlags(Mysetpoint->flags() ^ Qt::ItemIsEditable);
                //calib table
                for(int n=0;n<525;n++){
                    QTableWidgetItem     *Mysetpoint = new QTableWidgetItem(listValue[k]);
    
                    ui->tablewidget5->setItem(cter-1,k,Mysetpoint);
                    ui->tablewidget5->hideColumn(0);
                    ui->tablewidget5->hideRow(n);
                    ui->tablewidget5->setItem(cter-1, 8, new QTableWidgetItem(""));
                    ui->tablewidget5->setItem(cter-1, 9, new QTableWidgetItem(""));
                    if(k<9)   Mysetpoint->setFlags(Mysetpoint->flags() ^ Qt::ItemIsEditable);
                }
                //calib table
    
    
            }
    

    So who can help me to optimize this code !
    I have one CSV file . I will extract 4 table from it

    thank you



  • I suspect that updating the UI might be slow. Your widget should, AFAIK, have setUpdatesEnabled(bool) method you can use to turn updating off during the process.



  • @mvuori No I tried your solution but stiil take long time :/

    please ho can suggest a better way to do this :/



  • @dziko147
    If I read well I see for every cells of every line of yor csv I see:

    • 3 QTableWidgetItem
    • 1623 QTableWidgetItem ( setpoint table )
    • 594 QTableWidgetItem ( second setpoint table )
    • sorry but I lose the computation of objects but you must add other several thousand of QTableWidgetItem

    so only for one line on your csv there are several thousand of QTableWidgetItem, multiply everything for every line of CSV
    It is a huge number of items.

    First step I'd try set ui->tablewidget2->setUpdatesEnabled( false )
    for every QTableWidget and then the fill operation
    I'd try set ui->tablewidget2->setUpdatesEnabled( true ), alway for every QTableWidget.

    setUpdatesEnabled(false) disables update of QTableWidget when you add a QTableWidgetItem, for every add operation



  • @CP71 I resume what i want to do .

    So i load a csv file(compouned of 600 row and 7 column).
    this csv is an Input for my Application. it contains variable and their types .

    So I would like to get every type in a single tablewidget .



  • @dziko147
    Ok, try to call set Updates Enabled( false ) for every QTableWidget before your fill loop. I think could improve the performance.
    It is the first idea when I see your code



  • @CP71 like this ?

     while(!inFile.atEnd()){
            QString readedLine =inFile.readLine();
            QStringList listValue = readedLine.split(";");
            cter +=1;
    
            ui->tablewidget2->setRowCount(cter);
    
            ui->tablewidget3->setRowCount(cter);
    
            ui->tablewidget->setRowCount(cter);
            ui->tablewidget4->setRowCount(cter);
            ui->tablewidget5->setRowCount(cter);
    
    ui->tablewidget->setUpdatesEnabled(false);
    ui->tablewidget2->setUpdatesEnabled(false);
    ui->tablewidget3->setUpdatesEnabled(false);
    ui->tablewidget4->setUpdatesEnabled(false);
    ui->tablewidget5->setUpdatesEnabled(false);
    
    
            for(int k=0;k<listValue.size();k++){
                //csv global
                QTableWidgetItem     *Mytable = new QTableWidgetItem(listValue[k]);
                ui->tablewidget->setItem(cter-1,k,Mytable);
                ui->tablewidget->hideColumn(0);
                ui->tablewidget->setItem(cter-1, 8, new QTableWidgetItem(""));
                ui->tablewidget->setItem(cter-1, 9, new QTableWidgetItem(""));
                if(k<9)   Mytable->setFlags(Mytable->flags() ^ Qt::ItemIsEditable);// modif la valeur du k pour rendre le champ editable
                ui->tablewidget2->setUpdatesEnabled(true);
                  //csv global
    
                //setpoint table
                for(int t=10;t<551;t++){
                    QTableWidgetItem     *Mysetpoint = new QTableWidgetItem(listValue[k]);
    
                    ui->tablewidget2->setItem(cter-1,k,Mysetpoint);
                    ui->tablewidget2->hideColumn(0);
                    ui->tablewidget2->hideRow(t);
                    ui->tablewidget2->setItem(cter-1, 8, new QTableWidgetItem(""));
                    ui->tablewidget2->setItem(cter-1, 9, new QTableWidgetItem(""));
                    if(k<9)   Mysetpoint->setFlags(Mysetpoint->flags() ^ Qt::ItemIsEditable);
                    ui->tablewidget2->setUpdatesEnabled(true);
    
    
                }
                //setpoint table
    
                //config table
                for(int a=341;a<549;a++){
                    QTableWidgetItem     *Mytableconfig = new QTableWidgetItem(listValue[k]);
    
                    ui->tablewidget3->setItem(cter-1,k,Mytableconfig);
                    ui->tablewidget3->hideColumn(0);
                    ui->tablewidget3->hideRow(a);
    
                    for(int x=0;x<10;x++){
                        ui->tablewidget3->hideRow(x);
    
                    }
    
    
                    ui->tablewidget3->setItem(cter-1, 8, new QTableWidgetItem(""));
                    ui->tablewidget3->setItem(cter-1, 9, new QTableWidgetItem(""));
                    if(k<9)   Mytableconfig->setFlags(Mytableconfig->flags() ^ Qt::ItemIsEditable);
                    ui->tablewidget3->setUpdatesEnabled(true);
    
                    //config table
                }
                
                //calib table
                
                QTableWidgetItem     *Mytablecalib = new QTableWidgetItem(listValue[k]);
    
                ui->tablewidget4->setItem(cter-1,k,Mytablecalib);
                ui->tablewidget4->hideColumn(0);
                for(int m=0;m<341;m++){
                    ui->tablewidget4->hideRow(m);
                }
                for(int p=500;p<549;p++){
                    ui->tablewidget4->hideRow(p);
                }
                ui->tablewidget4->setItem(cter-1, 8, new QTableWidgetItem(""));
                ui->tablewidget4->setItem(cter-1, 9, new QTableWidgetItem(""));
                if(k<9)   Mytablecalib->setFlags(Mytablecalib->flags() ^ Qt::ItemIsEditable);
                ui->tablewidget4->setUpdatesEnabled(true);
                //calib table
                
                //param table
                for(int n=0;n<525;n++){
                    QTableWidgetItem     *Myparamtable = new QTableWidgetItem(listValue[k]);
    
                    ui->tablewidget5->setItem(cter-1,k,Myparamtable);
                    ui->tablewidget5->hideColumn(0);
                    ui->tablewidget5->hideRow(n);
                    ui->tablewidget5->setItem(cter-1, 8, new QTableWidgetItem(""));
                    ui->tablewidget5->setItem(cter-1, 9, new QTableWidgetItem(""));
                    if(k<9)   Myparamtable->setFlags(Myparamtable->flags() ^ Qt::ItemIsEditable);
                    ui->tablewidget5->setUpdatesEnabled(true);
    
                }
                //param table
    


  • @dziko147
    before while

    ui->tablewidget->setUpdatesEnabled(false);
    ui->tablewidget2->setUpdatesEnabled(false);
    ui->tablewidget3->setUpdatesEnabled(false);
    ui->tablewidget4->setUpdatesEnabled(false);
    ui->tablewidget5->setUpdatesEnabled(false);

    while (...)
    {
    ...
    }

    after while
    ui->tablewidget->setUpdatesEnabled(true);
    ui->tablewidget2->setUpdatesEnabled(true);
    ui->tablewidget3->setUpdatesEnabled(true);
    ui->tablewidget4->setUpdatesEnabled(true);
    ui->tablewidget5->setUpdatesEnabled(true);



  • @CP71 unfortunately it still takes long time :/



  • @dziko147
    no other idea at the moment!
    Sorry but I must go.
    See you on monday



  • @CP71 Ok thank you bye :)


  • Lifetime Qt Champion

    First thing: why do you have one CSV file that seems to contain data for radically different elements ?
    Wouldn't it make more sense to have one file for each ?
    You code will break as soon as you add a line somewhere in the middle of the file.

    That said as @CP71 wrote, you are creating a ridiculous amount of items, that you hide as soon as they are created doing many operations for nothing.

    The file being pretty small, you should load it at once and then process the lines that are specific to each table widget by group. There's no need to do anything for your table widget number one when the lines do not concern it.


Log in to reply