after deploying on RPI3 , UploadCSV file take very long time

[dziko147]

Hello ,
I tried to load a CSV file in my App . So I connected a button signal(clicked) to Slot which load a CSV file .
After deploying my App on Rpi3 target .

when I click on the button , it takes very long time to load the CSV (about 6 minutes) .
but finally it load the csv correctly !

Is it software problem ?

thank you

[SGaist]

Hi,

You have to give more information:

• which size is that file ?
• are you doing any processing on the data ?
• what are you doing with the data ?
• which version of Qt are you using ?
• did you do any benchmarking of your code ?
• what does the Pi do when you load the file ?

[dziko147]

@SGaist I think the problem is caused by doing processing on the data .
this is my code :

 while(!inFile.atEnd()){
        QString readedLine =inFile.readLine();
        QStringList listValue = readedLine.split(";");
        cter +=1;

        ui->tablewidget2->setRowCount(cter);

        ui->tablewidget3->setRowCount(cter);

        ui->tablewidget->setRowCount(cter);
        ui->tablewidget4->setRowCount(cter);
        ui->tablewidget5->setRowCount(cter);



        for(int k=0;k<listValue.size();k++){
            QTableWidgetItem     *Mytable = new QTableWidgetItem(listValue[k]);
            ui->tablewidget->setItem(cter-1,k,Mytable);
            ui->tablewidget->hideColumn(0);
            ui->tablewidget->setItem(cter-1, 8, new QTableWidgetItem(""));
            ui->tablewidget->setItem(cter-1, 9, new QTableWidgetItem(""));
            if(k<9)   Mytable->setFlags(Mytable->flags() ^ Qt::ItemIsEditable);// modif la valeur du k pour rendre le champ editable


           //setpoint table
            for(int t=10;t<551;t++){
                QTableWidgetItem     *Mysetpoint = new QTableWidgetItem(listValue[k]);

                ui->tablewidget2->setItem(cter-1,k,Mysetpoint);
                ui->tablewidget2->hideColumn(0);
                ui->tablewidget2->hideRow(t);
                ui->tablewidget2->setItem(cter-1, 8, new QTableWidgetItem(""));
                ui->tablewidget2->setItem(cter-1, 9, new QTableWidgetItem(""));
                if(k<9)   Mysetpoint->setFlags(Mysetpoint->flags() ^ Qt::ItemIsEditable);


            }
            //setpoint table

            for(int a=341;a<549;a++){
                QTableWidgetItem     *Mytableconfig = new QTableWidgetItem(listValue[k]);

                ui->tablewidget3->setItem(cter-1,k,Mytableconfig);
                ui->tablewidget3->hideColumn(0);
                ui->tablewidget3->hideRow(a);

                for(int x=0;x<10;x++){
                    ui->tablewidget3->hideRow(x);

                }


                ui->tablewidget3->setItem(cter-1, 8, new QTableWidgetItem(""));
                ui->tablewidget3->setItem(cter-1, 9, new QTableWidgetItem(""));
                if(k<9)   Mytableconfig->setFlags(Mytableconfig->flags() ^ Qt::ItemIsEditable);

                //config table
            }
            //calib table
            //for(int m=0;m<342;m++){ for(int y=500;y<549;y++){
            QTableWidgetItem     *Mysetpoint = new QTableWidgetItem(listValue[k]);

            ui->tablewidget4->setItem(cter-1,k,Mysetpoint);
            ui->tablewidget4->hideColumn(0);
            for(int m=0;m<341;m++){
                ui->tablewidget4->hideRow(m);}
            for(int p=500;p<549;p++){
                ui->tablewidget4->hideRow(p);}
            ui->tablewidget4->setItem(cter-1, 8, new QTableWidgetItem(""));
            ui->tablewidget4->setItem(cter-1, 9, new QTableWidgetItem(""));
            if(k<9)   Mysetpoint->setFlags(Mysetpoint->flags() ^ Qt::ItemIsEditable);
            //calib table
            for(int n=0;n<525;n++){
                QTableWidgetItem     *Mysetpoint = new QTableWidgetItem(listValue[k]);

                ui->tablewidget5->setItem(cter-1,k,Mysetpoint);
                ui->tablewidget5->hideColumn(0);
                ui->tablewidget5->hideRow(n);
                ui->tablewidget5->setItem(cter-1, 8, new QTableWidgetItem(""));
                ui->tablewidget5->setItem(cter-1, 9, new QTableWidgetItem(""));
                if(k<9)   Mysetpoint->setFlags(Mysetpoint->flags() ^ Qt::ItemIsEditable);
            }
            //calib table


        }

So who can help me to optimize this code !
I have one CSV file . I will extract 4 table from it

thank you

[mvuori]

I suspect that updating the UI might be slow. Your widget should, AFAIK, have setUpdatesEnabled(bool) method you can use to turn updating off during the process.

[dziko147]

@mvuori No I tried your solution but stiil take long time :/

please ho can suggest a better way to do this :/

[CP71]

@dziko147
If I read well I see for every cells of every line of yor csv I see:

• 3 QTableWidgetItem
• 1623 QTableWidgetItem ( setpoint table )
• 594 QTableWidgetItem ( second setpoint table )
• sorry but I lose the computation of objects but you must add other several thousand of QTableWidgetItem

so only for one line on your csv there are several thousand of QTableWidgetItem, multiply everything for every line of CSV
It is a huge number of items.

First step I'd try set ui->tablewidget2->setUpdatesEnabled( false )
for every QTableWidget and then the fill operation
I'd try set ui->tablewidget2->setUpdatesEnabled( true ), alway for every QTableWidget.

setUpdatesEnabled(false) disables update of QTableWidget when you add a QTableWidgetItem, for every add operation

[dziko147]

@CP71 I resume what i want to do .

So i load a csv file(compouned of 600 row and 7 column).
this csv is an Input for my Application. it contains variable and their types .

So I would like to get every type in a single tablewidget .

[CP71]

@dziko147
Ok, try to call set Updates Enabled( false ) for every QTableWidget before your fill loop. I think could improve the performance.
It is the first idea when I see your code

[dziko147]

@CP71 like this ?

 while(!inFile.atEnd()){
        QString readedLine =inFile.readLine();
        QStringList listValue = readedLine.split(";");
        cter +=1;

        ui->tablewidget2->setRowCount(cter);

        ui->tablewidget3->setRowCount(cter);

        ui->tablewidget->setRowCount(cter);
        ui->tablewidget4->setRowCount(cter);
        ui->tablewidget5->setRowCount(cter);

ui->tablewidget->setUpdatesEnabled(false);
ui->tablewidget2->setUpdatesEnabled(false);
ui->tablewidget3->setUpdatesEnabled(false);
ui->tablewidget4->setUpdatesEnabled(false);
ui->tablewidget5->setUpdatesEnabled(false);


        for(int k=0;k<listValue.size();k++){
            //csv global
            QTableWidgetItem     *Mytable = new QTableWidgetItem(listValue[k]);
            ui->tablewidget->setItem(cter-1,k,Mytable);
            ui->tablewidget->hideColumn(0);
            ui->tablewidget->setItem(cter-1, 8, new QTableWidgetItem(""));
            ui->tablewidget->setItem(cter-1, 9, new QTableWidgetItem(""));
            if(k<9)   Mytable->setFlags(Mytable->flags() ^ Qt::ItemIsEditable);// modif la valeur du k pour rendre le champ editable
            ui->tablewidget2->setUpdatesEnabled(true);
              //csv global

            //setpoint table
            for(int t=10;t<551;t++){
                QTableWidgetItem     *Mysetpoint = new QTableWidgetItem(listValue[k]);

                ui->tablewidget2->setItem(cter-1,k,Mysetpoint);
                ui->tablewidget2->hideColumn(0);
                ui->tablewidget2->hideRow(t);
                ui->tablewidget2->setItem(cter-1, 8, new QTableWidgetItem(""));
                ui->tablewidget2->setItem(cter-1, 9, new QTableWidgetItem(""));
                if(k<9)   Mysetpoint->setFlags(Mysetpoint->flags() ^ Qt::ItemIsEditable);
                ui->tablewidget2->setUpdatesEnabled(true);


            }
            //setpoint table

            //config table
            for(int a=341;a<549;a++){
                QTableWidgetItem     *Mytableconfig = new QTableWidgetItem(listValue[k]);

                ui->tablewidget3->setItem(cter-1,k,Mytableconfig);
                ui->tablewidget3->hideColumn(0);
                ui->tablewidget3->hideRow(a);

                for(int x=0;x<10;x++){
                    ui->tablewidget3->hideRow(x);

                }


                ui->tablewidget3->setItem(cter-1, 8, new QTableWidgetItem(""));
                ui->tablewidget3->setItem(cter-1, 9, new QTableWidgetItem(""));
                if(k<9)   Mytableconfig->setFlags(Mytableconfig->flags() ^ Qt::ItemIsEditable);
                ui->tablewidget3->setUpdatesEnabled(true);

                //config table
            }
            
            //calib table
            
            QTableWidgetItem     *Mytablecalib = new QTableWidgetItem(listValue[k]);

            ui->tablewidget4->setItem(cter-1,k,Mytablecalib);
            ui->tablewidget4->hideColumn(0);
            for(int m=0;m<341;m++){
                ui->tablewidget4->hideRow(m);
            }
            for(int p=500;p<549;p++){
                ui->tablewidget4->hideRow(p);
            }
            ui->tablewidget4->setItem(cter-1, 8, new QTableWidgetItem(""));
            ui->tablewidget4->setItem(cter-1, 9, new QTableWidgetItem(""));
            if(k<9)   Mytablecalib->setFlags(Mytablecalib->flags() ^ Qt::ItemIsEditable);
            ui->tablewidget4->setUpdatesEnabled(true);
            //calib table
            
            //param table
            for(int n=0;n<525;n++){
                QTableWidgetItem     *Myparamtable = new QTableWidgetItem(listValue[k]);

                ui->tablewidget5->setItem(cter-1,k,Myparamtable);
                ui->tablewidget5->hideColumn(0);
                ui->tablewidget5->hideRow(n);
                ui->tablewidget5->setItem(cter-1, 8, new QTableWidgetItem(""));
                ui->tablewidget5->setItem(cter-1, 9, new QTableWidgetItem(""));
                if(k<9)   Myparamtable->setFlags(Myparamtable->flags() ^ Qt::ItemIsEditable);
                ui->tablewidget5->setUpdatesEnabled(true);

            }
            //param table

[CP71]

@dziko147
before while

ui->tablewidget->setUpdatesEnabled(false);
ui->tablewidget2->setUpdatesEnabled(false);
ui->tablewidget3->setUpdatesEnabled(false);
ui->tablewidget4->setUpdatesEnabled(false);
ui->tablewidget5->setUpdatesEnabled(false);

while (...)
{
...
}

after while
ui->tablewidget->setUpdatesEnabled(true);
ui->tablewidget2->setUpdatesEnabled(true);
ui->tablewidget3->setUpdatesEnabled(true);
ui->tablewidget4->setUpdatesEnabled(true);
ui->tablewidget5->setUpdatesEnabled(true);

[dziko147]

@CP71 unfortunately it still takes long time :/

[CP71]

@dziko147
no other idea at the moment!
Sorry but I must go.
See you on monday

[dziko147]

@CP71 Ok thank you bye :)

[SGaist]

First thing: why do you have one CSV file that seems to contain data for radically different elements ?
Wouldn't it make more sense to have one file for each ?
You code will break as soon as you add a line somewhere in the middle of the file.

That said as @CP71 wrote, you are creating a ridiculous amount of items, that you hide as soon as they are created doing many operations for nothing.

The file being pretty small, you should load it at once and then process the lines that are specific to each table widget by group. There's no need to do anything for your table widget number one when the lines do not concern it.

[dziko147]

@SGaist ok your solution is clear . and thank you for your reply .
but can you suggest any example ?

[dziko147]

Finally , i solve this problem :D .

this is source code for those you need :D

//here i fill data to tablewidget
while(!inFile.atEnd()){
        QString readedLine =inFile.readLine();
        QStringList listValue = readedLine.split(";");
        cter +=1;
  ui->tablewidget->setRowCount(cter);
 for(int k=0;k<listValue.size();k++){
            //csv global
            QTableWidgetItem     *Mytable = new QTableWidgetItem(listValue[k]);
            ui->tablewidget->setItem(cter-1,k,Mytable);
            ui->tablewidget->hideColumn(0);
            ui->tablewidget->setItem(cter-1, 8, new QTableWidgetItem(""));
            ui->tablewidget->setItem(cter-1, 9, new QTableWidgetItem(""));
            if(k<9)   Mytable->setFlags(Mytable->flags() ^ Qt::ItemIsEditable);// modif la valeur du k pour rendre le champ editable
              //csv global
  }
}

then when button clicked I extract some data from tablewidget :

void UploadCSV::on_Corebtn_clicked()
{      ui->tablewidget->setFixedSize(950,450);

      // ui->tablewidget->setVisible(true);
       for (int i=0;i<ui->tablewidget->rowCount() ;i++ ) {
           QTableWidgetItem *cell7 = ui->tablewidget->item(i,7);
           QVariant myData7 = cell7->data(Qt::DisplayRole);
           if(myData7!="Core_setpoint"){
               ui->tablewidget->hideRow(i);
           }
       }

}

then when button is released should Show the row hided :D

void UploadCSV::on_Corebtn_released()
{
    ui->tablewidget->setFixedSize(950,450);
    for (int i=0;i<ui->tablewidget->rowCount() ;i++ ) {
  ui->tablewidget->showRow(i);
    }
}

thank you all for your help :)