QT charts extremely slow - QLineSeries

[Witc]

@JonB I had mistake when calling the signal, Now my self.sigDatasetChanged.emit(mySerie) is emited (renamed to sigPlotData), but no line is plotted - have no idea how to pass the mySerie to Chart
main.py repaired for emitting signal with mySerie:

# This Python file uses the following encoding: utf-8
import os
import sys
from pathlib import Path

from PySide6.QtCore import QObject, Slot, Signal
from PySide6.QtQml import QQmlApplicationEngine
from PySide6.QtWidgets import QApplication 
from PySide6.QtCharts import QLineSeries


class AppWindow(QObject):

    # Signals from python to QML
    sigPlotData = Signal(QLineSeries)
    
    def __init__(self):
        super().__init__(None)


    def plotData(self):
       
        x = [0,1,2,3,4,5,6,7,8,9]
        y = [10,20,10,5,23,15,12,2,9,21]

        mySerie = QLineSeries()
        #mySerie.attachAxis(self.xSigVal)
        #mySerie.attachAxis(self.ySigVal)
        
        for i in x:
            mySerie.append(i, y[i])
        
        # pass the data to the QML chart
        self.sigPlotData.emit(mySerie)
        QApplication.processEvents()
            

if __name__ == "__main__":
    app = QApplication(sys.argv)
    engine = QQmlApplicationEngine()

    appWin = AppWindow()
    engine.rootContext().setContextProperty("backend", appWin)

    engine.load(os.fspath(Path(__file__).resolve().parent / "main.qml"))
    if not engine.rootObjects():
        sys.exit(-1)

    appWin.plotData()

    sys.exit(app.exec())

[Witc]

@JonB BTW this is why I started this thread - I do not know how to pass the QlineSerie to QtChart from python

[JonB]

@Witc

@JonB said in QT charts extremely slow - QLineSeries:

Have you verified function onSigDatasetChanged(serie) gets called at all?

                    var serie1 = myChart.createSeries(ChartView.SeriesTypeLine, "signal", axisX, axisY);

                    serie1 = serie

I don't know, but this does not look right. What is the point of creating a series and assigning it to a variable if you then overwrite that variable on the next line? Aren't you supposed to do something like serie1.append(serie) or myChart.addSeries(serie) to put data points or a series onto a chart?

[Witc]

@JonB Just tried myChart.addSeries(serie) and also serie1.append(serie) - unfortunatelly no data are plotted.
When I created myChart.createSeries(ChartView.SeriesTypeLine, "signal", axisX, axisY); I also told that I want to SeriesTypeLine type ...

[JonB]

@Witc said in QT charts extremely slow - QLineSeries:

I also told that I want to SeriesTypeLine type

Isn't that irrelevant when the series you have created and wish to add is already a QLineSeries?

Last time of asking:

Have you verified function onSigDatasetChanged(serie) gets called at all?

When that is working, print out what is in the serie parameter passed into it. Does it indeed contain all the datapoints you added into it? In which case I would have thought myChart.addSeries(serie) is indeed the right thing to do.

[Witc]

@JonB I know the function onSigPlotData(serie) is called because in that function I added console.log("We are going to plot") and I cann see this text in console

You said print out what is in the serie - I tried like that: console.log(serie) - it only printed out QLineSeries(0x265e5c1c270) to my console

[JonB]

@Witc said in QT charts extremely slow - QLineSeries:

I added console.log("We are going to plot") and I cann see this text in console

That is good but you didn't show/say so previously, so now we know.

I tried like that: console.log(serie) - it only printed out QLineSeries(0x265e5c1c270) to my console

So what can you do about this? Don't you think e.g. console.log(serie.count()) would give better information?

[Witc]

@JonB When add console.log(serie.count()) it prints out again just some addres,
when I type mySerie.count() in python (before emitting the signal) the output is 10 - so guess the X and Y data are presented in the serie

[JonB]

@Witc said in QT charts extremely slow - QLineSeries:

When add console.log(serie.count()) it prints out again just some addres,

That does not sound good. I know your series is OK at the Python side, the question is how to get it at the QML side.

I said I know nothing about QML. At this point I think you should read e.g. https://stackoverflow.com/questions/54459713/pass-c-lineseries-to-qml-charts

You cannot pass QLineSeries to QML since ChartView has no possibility to add series. There is ChartView.createSeries() only. So the only way is to create a series in QML and pass array of points from C++ so you could add the points to the series using XYSeries.append(x,y).

[Witc]

@JonB BTW when I call myChart.addSeries(serie)in qml file - then the output is
TypeError: Property 'addSeries' of object DeclarativeChart(0x17574328e80) is not a function

[Witc]

@JonB OK, so they say I should pass array of points from python to QML (It looks like we are getting to my first post here). I can pass X and Y point by point, but have no idea how to pass all my array or list

x = [0,1,2,3,4,5,6,7,8,9]
y = [10,20,10,5,23,15,12,2,9,21]

[JonB]

@Witc
My final word: I think QML ChartView/XYSeries etc. are not the same thing as their Qt QChartView/QXYSeries etc. counterparts. So when you said

I found similar problem on the forum - they solved it (Creating the series first, and adding it to the Chart after all of the appends solved the problem)

but they use C instead of python - Is it possible to bring the solution to python? I could not find a solution :-(

I don't think it has anything to do with C++/Python, you picked a Qt widgets issue/solution which cannot be applied to QML....

Which means I only see QML XYSeries.append(real x, real y) method to add one point at a time (I do not see a way to create a series on its own and then add it to a QML ChartView). Which you can call in a loop given the two arrays of points.

If that is "too slow", I don't know what you can do. Maybe you can hide the chart view and re-show it after all points have been added, maybe that would make it faster.

Otherwise have you read e.g. https://stackoverflow.com/questions/38467769/add-c-qabstractseries-to-qml-chartview ? Does that help for C++ -> QML?

[Bob64]

I only have experience with QML charts and a C++ back end, but the approach I have taken is to create the series in QML and expose functions from C++ that accept a series so that it can be filled from the C++ side.

I don't know how this translates to Python. In particular I do not know what the mechanism is for exposing Python-side functions into QML. I presume there is some sort of equivalent way of doing this (in C++ it is necessary to declare a function as Q_INVOKABLE in a QObject-derived class). Also I do not know how efficient this is going to be as there will be additional marshalling between Python and the underlying C++ layer.

This might be one of those cases where it is necessary to bite the bullet and drop down to C/C++ to handle this from the Python.

[fcarney]

I don't know how to do this in Python, but this is how I solve this issue with QML charts. I create a function that updates all the data in the series at once that is callable from QML:

class PointUpdater : public QObject
{
    Q_OBJECT
public:
    PointUpdater(QObject* parent=nullptr)
        : QObject(parent)
    {
    }

public slots:
    void update(QAbstractSeries *series, QVariantList points){
        if(!series){
            return;
        }
        QVector<QPointF> newpoints;
        for(QVariant point: points){
            auto list = point.toList();
            newpoints.push_back(QPointF(list.at(0).toDouble(), list.at(1).toDouble()));
        }

        QXYSeries *xySeries = static_cast<QXYSeries *>(series);
        xySeries->replace(newpoints);
    }
};

[Witc]

@fcarney looks nice, but also do not know how to conver it to the python - you are passing pointer of *series (which was created in QML)
to C++ and then fill it - in python maybe impossible?!?

[Witc]

@JonB @fcarney @Bob64
Probably with my best solution below, it takes about 5 seconds to plot 5000 values

Python file - the slot fillSigChart is called from QML

@Slot(QLineSeries)
    def fillSigChart(self,serie1):
       
        start = time.time()
        for i in self.x:
            serie1.append(i, self.y[i])   # filling serie with my prepared data
        end = time.time()
        print(end - start)

In QML file: I call the slot above

Connections{ 
  target: backend
  function onSigPlotData(x,y){
    myChart.removeAllSeries();
    myChart.zoomReset()
    var serie1 = myChart.createSeries(ChartView.SeriesTypeLine, "Signal ", axisX, axisY);
    serie1.useOpenGl = true
            
    backend.fillSigChart(serie1)       // pass serie1 to python and then fill it with data
   } 
 }

[JonB]

@Witc
OK, so at least we understand how to/that you can go from QML to Python and back passing a series around.

It might be that this is as good as it gets. However, you are still appending points one by one and this may be "expensive", we don't know.

There is an overload of append() which accepts a pre-built list of points: PySide2.QtCharts.QtCharts.QXYSeries.append(points). In C++ this is void QXYSeries::append(const QList<QPointF> &points).

You should try this. So instead of serie1.append(i, self.y[i]) you want to first build all the points into a list in the loop and then append them in one go afterwards. I would guess something like:

points = []
for i in self.x:
    points.append(QPointF(i, self.y[i]))   # filling points with my prepared data
serie.append(points)  # append list of points in one call

Does that make it any faster?

Even better, given that you start with no points you need to retain and just want the newly created points, might be serie.replace(points)

Note: This is much faster than replacing data points one by one, or first clearing all data, and then appending the new data. Emits QXYSeries::pointsReplaced() when the points have been replaced.

[Bob64]

I concur with @JonB regarding appending multiple points at once, or replacing the data vs appending points one at a time. I found it made a massive difference.

One point at a time is slow in C++, but you will be paying a double penalty in Python as each individual call has to go through a wrapper layer. The Python overhead of the calls taking a list will mainly be the conversion of the Python list to the QList. I would hope that conversions like Python list to QList are optimised as much as they can be as they will be pervasive operations in a Python Qt application.

[Witc]

@JonB

Your idea to serie1.append(points) had no improvement - it took also about 5 seconds, but your second idea with serie1.replace(points) has great result!

Solution in python:

start = time.time()
points = []
for i in self.x:
     points.append(QPointF(i, self.y[i]))   # filling points with my prepared data 
serie1.replace(points)  # fill list of points in one call
end = time.time()
print(end - start)

for 50000 values it takes only 126 ms, for 5000 values it takes 13 ms

I think we can consider this thread as solved - or any other ideas?
Thank you all @JonB @Bob64 @fcarney for help!

[JonB]

@Witc
:) I guess the append() maybe just appends one at a time, even if it's empty. The replace() presumably does not have to do that.