Deleting QChart Causes 30+ Second Application Hang!

FleetingMemory

A little more testing - I used release mode, removed all series from QChart in a QChartView after a new query is run. During this test I re-used the same QChart -- after running removeAllSeries(), I used the points from the query to generate new series and add them to the chart. Points are formatted <millis since epoch, value>. Series have OpenGL turned on in this test.

1: A Query is run, Qlist<Qlist<QPointF>> generated. I return the Query Count
2: If QChart Exists, remove all series (this I timed)
3: for each Qlist<QPointF> generate a new series with replace(), append Axes and insert into chart (this I timed)

Query Size 450,000 points. Spread across 12,000 series.
populating 12,000 series after query takes 2.964 s
Gets stuck on remove all 12,000 series (freezes main thread), takes 35.774 seconds to remove

Query Size: 332,000 points
populating 10,600 series: 3.3s
populating 2nd time diff data: 3.4s
removing all series: 25.35s
removing all (2nd time diff data): 25.6s

Query Size: 247,900 points
populating 9300 series: 2.5s
removing all series: 19s

Query Size 250,000
Populating Series: 1.32 s
removing all 8,050 series: 14.33s
removing all 8,050 series (2nd run diff data): 13.4s

Query Size 223,200
Populating 8600 series: 2.4s
populating 8600 series(2nd run diff data): 2s
removing all series: 15.6s
removing all series (2nd run diff data): 15.6s

Query Size: 195,600
populating series: 1.7s
Removing 7820 series: 12.5s
removing all 7820 series (2nd run diff data): 12.9s

Query Size 158,543
populating the series after query: 0.7s
removing all 4,859 series: 4.9s

Query Size 21700
Removing 1240 series: 0.395s
populating series: 0.1s

Query Size: 9,818
populating series: 0.04s
removing all 735 series: 0.045s
removing all 735 series (2nd run diff data): 0.044s

Performance seems to get worse pretty quickly at a non-linear rate

If it helps, I am compiling this as a plugin loaded in a simple main application (though that shouldn't affect things).
I have tried with useOpenGL both on and off. As far as I can tell, it makes little difference.
The fact the removeAll/delete freezes the main thread, which essentially locks up the entire application it is running in is a huge issue for me...

JoeCFD

@FleetingMemory
look at source code here.
https://codebrowser.dev/qt5/qtcharts/src/charts/qchart.cpp.html
removeAllSeries func has a loop to call removeSeries(QAbstractSeries *series)
I guess it will trigger update for each removal. This is not efficient.

JonB

@FleetingMemory
Why can't you test exactly these figures, with whatever (hopefully tiny) code you use, against the skeleton I provided? You only have to alter the two loop counters. Then people have actual code they can test, so far as I know we really don't know for sure what your code reads. I seem to have been saying/suggesting this all along. If you want people to help, or even to report a bug, we have to have a complete, minimal, reproducible example.

FleetingMemory

@JonB I just tried your loop. I get similar results when increasing series count.

JoeCFD

@FleetingMemory I tried Jon's code with 10000 series + 10 points and tried different things:

disable update of QChartView
disable QChartView and QChart
hide QChartView and QChart
before chart is deleted. Nothing helps.

However, I am able to get quick clean-up of qchartview with

    chartView->setChart( new QChart() );
    chart->deleteLater();

Not sure if this helps you. If this piece of code does not help, you may try to delete the chart in a thread while charview is available for other uses. I guess your app will have issue at exit.

FleetingMemory

This post is deleted!

FleetingMemory

@JoeCFD I tried that as well, it is marginally faster to delete this way than removeAll. I have tried threading but because QChart calls GUI functions, I was not able to get it to work. I tried threading the delete as well as the removal of Series. Neither works. I usually run into errors like -- widget cannot be moved to a new thread -- accessing widget from wrong thread (usually pertains to the QChartData) -- sending signal accross thread, which results in an immediate crash. This is because the removeALLSeries and delete call on update()

SGaist

Hi,

Did you try calling setUpdateEnabled(false) on the chart ?

FleetingMemory

@SGaist QChart does not have updatesEnabled(false) only QChartView. Doesn't seem to make a difference. I also tried blocking signals.

JoeCFD

@FleetingMemory My results with Jon's code:

use deleteLater
Starting to delete
Finished deleting 771
original deletion
Starting to delete
Finished deleting 11663

Qt 6.8.2 + Ubuntu 22.04

FleetingMemory

@JoeCFD how many series? It still locks the main thread though... I did not see a noticeable difference between delete and deleteLater performance on my side

JoeCFD

@FleetingMemory 10000 series + 10 points

Jon's code and chartView is not deleted. Series are cleared quickly.

#include "chart.h"

// using namespace QtCharts;

Widget::Widget(QWidget *parent)
    : QWidget(parent)
{
    setGeometry(100, 100, 800, 600);
    setLayout(new QVBoxLayout);

    this->chart = new QChart();
    this->chartView = new QChartView(chart, this);
    layout()->addWidget(chartView);
    chartView->setSizePolicy(QSizePolicy::MinimumExpanding, QSizePolicy::MinimumExpanding);

    for (int i = 0; i < 10000; i++)
    {
        QLineSeries* series = new QLineSeries();
        for (int x = 0; x < 10; x++)
            series->append(x, x * (i + 1));
        chart->addSeries(series);
    }
    
    chart->createDefaultAxes();
    chart->axes(Qt::Horizontal).first()->setTitleText("X Axis");
    chart->axes(Qt::Vertical).first()->setTitleText("Y Axis");
    QTimer::singleShot(5000, this, &Widget::deleteChart);
}

void Widget::deleteChart()
{
    QElapsedTimer elapsed;
    qDebug() << "Starting to delete";
    elapsed.start(); 

    chartView->setChart( new QChart() );
    chart->deleteLater();

    layout()->removeWidget(chartView);
    //chart->removeAllSeries();
    //delete chart;
    //chart = nullptr;
    //delete chartView;
    //chartView = nullptr;
    qDebug() << "Finished deleting" << elapsed.elapsed();
}

Widget::~Widget() {}

FleetingMemory

@JoeCFD Going back to my original code, I tried moving setchart(new QChart()) and deletelater() to before I populate the new chart with new series. Previously I was populating a new chart then setting it, then deleting a pointer to the old chart. Now, if I watch the memory, deleteLater still doesn't actually call delete until after the new chart is populated fully. So it looks like it deleted quickly but it still hangs after the new chart pops in with data. Doing it this way, however does take slightly less time to delete.

JoeCFD

@FleetingMemory Maybe try to delete it in a thread. Also a splash widget may be needed at exit.

FleetingMemory

@JoeCFD Can't delete in a thread, it calls on GUI functions because it loops across removeSeries(), which calls update. This causes a cross-thread exception in Qt. DeleteLater() doesn't actually delete immediately within your timer bounds in the code you posted above (that is just timing QT scheduling it), so it still freezes the program later on.

JonB

@FleetingMemory
As you have now discovered, it is the deleting of each series which is so time consuming (for unknown reasons). There is a huge difference between 10 series with 10,000 points each versus 10,000 series with 10 points each. (In fact, in latter case timing is same even if all series are empty/have no points.) There seems to be little you can do to affect that timing. I see no evidence it has anything to do with whether the view/chart/series are shown or not, have signals disabled or not, etc.

Which takes me back to my question as to why you need 10,000 series or any such figure, which you did not answer. There is no chance the user can view or appreciate such a large number, so what is the point of adding that many onto the chart in the first place? That is where I would look for efficiency improvement.

If you really are not prepared to reduce that, then: I presume any deleteLater() won't help as that just delays when the delay comes to a bit later. I assume you are right that series cannot or should not be deleted from a thread as they are some kind of UI object. Which would leave you with one possible "hack" to distribute the deletion time "nicely". Follow @JoeCFD's code to assign a new, blank chart to the view (chartView->setChart( new QChart() );). But do not immediately delete the original chart there: instead, keep it around in a suitable variable. In UI code thread set off a repeating timer every so often: in each timeout use removeSeries() to delete a few of the 10,000 series, example perhaps 100 at a time. You can do this because you are in the main/UI thread. Over time you will have deleted all the series from the old QChart object, at that point you can delete the (empty) chart and terminate the timer. Seems to me reducing the number of series in the first place is preferable, but there you are if you really want to do this.

FleetingMemory

@JonB said in Deleting QChart Causes 30+ Second Application Hang!:

Which takes me back to my question as to why you need 10,000 series or any such figure, which you did not answer. There is no chance the user can view or appreciate such a large number, so what is the point of adding that many onto the chart in the first place? That is where I would look for efficiency improvement.

The chart is to be used for displaying & isolating specific data/time frames, thus all data/lines need to be present, and each series is a line being displayed. By manipulating the X-Axis the user can zoom into/out of to a specific time frame. The idea of deleting a few at a time is something I have definitely been exploring as a possible solution however it doesn't fix the fact it will still hang the whole application when the window is closed or application exited as well.

JoeCFD

@FleetingMemory that is why I wrote earlier that you need a splash widget with some nice animation at exit and make your app wait for these series to be cleared.

You can also file a bug with Jon's test code to Qt for optimization in removeAllSeries call. I guess one update may be enough instead of one update after one deletion.

Christian Ehrlicher

I've tested @JonB 's code with 12000 series / 40 data points each in debug mode:

Starting to create
Finished creating 32858
Starting to delete
Finished deleting 195421

Digged a little bit around to see what happens during destruction. The most time the legend gets updated... so:
I added this before delete chart;:
delete chart->legend();

-->
Starting to create
Finished creating 33861
Starting to delete
Finished deleting 27167

The best thing I found is to add d_ptr->m_dataset->disconnect(); in QChart dtor. I doubt this will go in into official QtCharts since it's in maintainance mode only.

-->
Starting to create
Finished creating 31901
Starting to delete
Finished deleting 16988

JonB

@Christian-Ehrlicher
All I saw was all the time being spent in removeAllSeries(), or whichever you remove series if one by one. I looked at remoceSeries() source code online but I don't have sources to dig further. I don't know how that is affected by legend. Nothing struck me except however it is storing its model data. That could relate to your d_ptr->m_dataset. And d_ptr requires OP to change and compile Qt sources, right?

It's taking 30 seconds to get rid of 10,000 series which it never took 30 seconds to create, it is a lot of time per series.

P.S.
Of course, as you say QtCharts is going so realise nothing is going to be done about sources.