Plotting a very long graph using QCustomPlot

[lukutis222]

Hello. I use QCustomPlot to plot a graph. The graph represents current and voltage and is intended to monitor battery charge/discharge.

I have current/voltage sensor that is sending data every 200ms. When my QT application receives data, the following functions are called:

          sample_counter ++;

          addPoint(sample_counter,current);
          plot(ui->customPlot);

Where addPoint and plot functions are :

void MainWindow::addPoint(double x, double y)
{
    qv_x.append(x);
    qv_y.append(y);

}

void MainWindow::plot(QCustomPlot *customPlot)
{
    customPlot->graph(0)->setData(qv_x,qv_y);
    customPlot->replot();
    customPlot->update();

}

As you can see addPoint function appends a value to a vectors x and y. Those are private MainWindow class variables.

QVector<double> qv_x,qv_y;

My graph looks fine :

I notice that after 24hours or more, my QT application just crashes and turns off. I wonder if this can be caused due to so many points collected in x and y vectors that the application cannot keep up? If I am sending x and y samples every 200ms, after 24hours there will be 432000 samples on the graph.

Ideally, I would like to be able to keep the QT application running and collecting samples for up to a week.
I would appreciate any help/tips on how to debug and optimise this.

[JonB]

@lukutis222
I don't know anything about QCustomPlot and I don't suggest your code should "crash", but after one week you would have 3 million samples to plot. That is surely way too many to be of much use to the user. The usual with this kind of thing is to "prune"/"aggregate" the samples to something useful when there are so many, have you considered that?

[lukutis222]

@JonB
Why would that be too many for the user? Surely the more samples the better it is for the user and if the application can support this I dont see any problem. He would be able to zoom in/zoom out at any time within last 7 days and see what exactly was happening to the current and voltage at any given time which I think is super cool.

Could you please clarify what do you mean regarding prune/agregate. I am not too sure what that means. I would consider anything if that makes my application run stable without crashes and produce some decent results.

[JonB]

@lukutis222 said in Plotting a very long graph using QCustomPlot:

Surely the more samples the better it is for the user and if the application can support this I dont see any problem.

Then why does it crash?

The user cannot see 3 million points in a plot. There are (probably) not this many pixels on the screen even if it filled every one.

Pruning/aggregating means reducing the number of individual points. For example, taking the average of every 300 samples and just plotting one point for that still means the user can see a figure for every one minute over a week while reducing the total points plotted to 10,000.

If you don't want to do that it's up to you. I do not know whether the large number of points is causing instability/crashes, you could try the reduction and see.

[lukutis222]

@JonB
I dont know why it crashes. It is possible due to too many points. That is the exact reason why I post this question on this forum. I thought someone had some experience with QCustomPlot and might know if that is possible ir not. If you do not know - that is not a problem :) Thank you for your suggestion, I will try to send the data with half frequency (400ms instead of 200ms) and see if I can get the application to run for 48 hrs instead - that would be a good test to see if it is causes the issues.

@JonB said in Plotting a very long graph using QCustomPlot:

The user cannot see 3 million points in a plot. There are (probably) not this many pixels on the screen even if it filled every one.

User is able to see as many points in the graph as he likes because QCustomPlot graph is interractable which means user can drag the plot left/right, up/down, zoom in/zoom out hence the more points on the graph the better it is. Even though there might be 3 million points in the graph, user could zoom in to a single point on the graph if he wanted to and see what was the value at a particular second

[jsulm]

@lukutis222 said in Plotting a very long graph using QCustomPlot:

I dont know why it crashes

Then you should run through debugger and check the stack trace

[JoeCFD]

@lukutis222 think about if vector is used to store data, copy may be applied when the capacity of a vector has to be raised. As the data size increases, copy may take long and then new data comes in. For large data set, special care has to be taken.

[SimonSchroeder]

@lukutis222 said in Plotting a very long graph using QCustomPlot:

If I am sending x and y samples every 200ms, after 24hours there will be 432000 samples on the graph.

Just create vectors with 432000 entries of random data (or a constant value). See if QCustomPlot will happily display this first. There could be other things going on when you slowly add one sample after another.

It might also take more time than 200ms to replot the graph. This would mean that all your calls to MainWindow::plot would gather up inside the event loop. Unfortunately, Qt does not provide anything (to my knowledge) that would help merge all slots into a single call.

Just to be sure: You are only using a single thread? Or is the collection of the data samples done in a separate thread?

[lukutis222]

@SimonSchroeder Just a single thread. I am thinking to reduce the sampling frequency to 1second instead of 200ms when I am about to do long graphs ( more than 24 hrs). If I am measuring a couple of hours, then 200ms is good.

@SimonSchroeder said in Plotting a very long graph using QCustomPlot:

Just create vectors with 432000 entries of random data (or a constant value). See if QCustomPlot will happily display this first. There could be other things going on when you slowly add one sample after another.

I tried just creating 3000000 samples on a graph:

    for(int i = 0; i <3000000;i++){
     addPoint_test(i,1);
    }
     plot_test(ui->customPlot);

The graph was drawn but the whole application was lagging a lot!

[JonB]

@lukutis222
Thought for speed/efficiency: your current code calls replot/update for each individual point added. That is 5 times per second. Surely user does not need to see this many separate updates in real time? So even if you decide not to reduce your points/samples plotted, so that user can eventually see every one, you could e.g. only replot/update once per second. Even better if QCustomPlot has a method to add multiple points in one call, but even if not still reduces redraws by factor of 5, allowing time for redraws to complete/reducing load?

[lukutis222]

@JonB
Do you mean that instead of plotting 1 point every 200ms just plot 5 points every 1 second? That might be good idea to reduce the number of draws! Thanks

[JonB]

@lukutis222
Yes. Even if QCustomPlot does not offer a single call to plot 5 points it will still be the replot/update() which is "expensive". Calling that only once per second after adding, say, 5 points should be an improvement. It is not clear whether you can afford to add the 5 points in 5 separate calls as they arrive or whether you should "buffer" those and add them in a for loop --- depends on their implementation of adding points one at a time --- so you might have to test the buffering approach to be sure.