Slow QGraphicsView, slow "40000 Chips example"

Zdenek

I have a simple problem. I need to plot a graphics scene with ~1500 simple QGraphicsItems (reimplemented QGraphicsItem::paint method draws single circle, also QGraphicsItem::shape is simple, it returns an empty shape). I need to repaint the scene as fast as possible (clear all items, add new items - in a loop). Unfortunately many items are overlapping and it probably slows down QGraphicsView.

It was recommended in many comments to look at 40000 chips example. Yes, it works, the chips (graphic items) are not overlapping. But when the source is changed in the following way:

void MainWindow::populateScene()
{
    QImage image(":/qt4logo.png");

    // Populate scene
    int xx = 0;
    for (int i = -11000; i < 11000; i += 110) {
        ++xx;
        int yy = 0;
        for (int j = -7000; j < 7000; j += 70) {
            ++yy;
            qreal x = (i + 11000) / 22000.0;
            qreal y = (j + 7000) / 14000.0;

            QColor color(image.pixel(int(image.width() * x), int(image.height() * y)));
            QGraphicsItem *item = new Chip(color, xx, yy);
            ////item->setPos(QPointF(i, j)); // original line
            item->setPos(QPointF(0, 0));  // modified line: ALL ITEMS (CHIPS) HAVE THE SAME POSITION, THEY OVERLAP
            scene->addItem(item);
        }
    }
}

the example does not work anymore! I realized (in my application) that the bottleneck is time needed to populate the scene, once it is populated, it works fine (panning, zooming, etc.) Does have anybody idea how make the populating of the scene more efficient?

jsulm

@Zdenek said in Slow QGraphicsView, slow "40000 Chips example":

clear all items, add new items - in a loop

Just to understand: you clear all items and add new items on every paint event? Why do you need to remove all items and add new very time?

Zdenek

@jsulm I'm not sure if on every paint event (paint events are generated internally by Qt, aren't they?). I have a loop in my application, it reads data from a file, and every time when the block of data is read from the file, all items in QGraphicsScene are removed and added (different) again. I see clearly that this "population step" (when QGraphics::addItem method is called 1000+ times) takes long time. I see clearly when all items are created with the same coordinates, it takes longer.

Yes, I need to remove all items and to create new ones based on new data.

JonB

@Zdenek said in Slow QGraphicsView, slow "40000 Chips example":

I see clearly when all items are created with the same coordinates, it takes longer.

To be clear: you are saying that adding lots of items, either initially with overlapping coordinates or after setPos(), is slower than adding same with non-overlapping coordinates? Have you timed that just adding to scene excluding any drawing at all from QGraphicsView? I am trying to imagine why QGraphicsScene cares at add time whether items are or are not overlapping....

Asperamanca

@Zdenek
Two points:

On QGraphicsScene, set the item index method to "NoIndex" (QGraphicsScene::setItemIndexMethod)
Do NOT remove and add items. Instead, recycle them:
- Create the items you need initially
- When the next cycle comes up, only hide the items and add them to a "reuse list", either one globally or several e.g. by item category or type
- When you want to add items now, first check the fitting "reuse list" and take an item from there
- Create new items only when you run out of items to reuse

JonB

@Asperamanca
Have you timed whether re-using, with its associated hides & shows, is really that much faster than remove & add? It may be, but there are also overheads for hidden items.

Zdenek

@JonB : I'm not sure what you meant by "excluding any drawing at all from QGraphicsView". It would be difficult to remove all other graphics items from my application, but the described behavior is well visible using 40000 Chips examle (see my initial comment, just small modification and Qt example does not work). I guess if coordinates of all (many) items are same (all items are overlapping), it is necessary to estimate the exact overlapping area and it takes time.

JonB

@Zdenek
I merely meant doing the timing only on the loop which does 1,000 addItem()s excluding any time the QGraphcisView might take to render it. Which probably means timing your call to populateScene() without anything else in the timing.

@Asperamanca may be right, I did not say he was not. You might compare remove/add against show/hide/change-pos on a test. If there is a noticeable difference and it can be applied to your case you right refactor your code to work that way.

I might take a look at the example code but I do not have time right now.....

Zdenek

@Asperamanca Thank you, it is my current plan. However, let assume "40000 chips" example, the scene is populated once only, recycling items does not help. I see clearly the dependency between overlap and slowness of Qt, I tried to modify "400000 chips" example as follows:

            double coef = 1.0;
            ////item->setPos(QPointF(i, j)); // original line
            item->setPos(QPointF(coef*i, coef*j));  // modified line: ALL ITEMS (CHIPS) HAVE THE SAME POSITION, THEY OVERLAP

'coef = 1.0' gives the same results are original Qt code. 'coef = 0.0' is full overlap. With 'coef = 0.1' the slowness the example is obvious, with 'coef = 0.05' it takes few seconds to populate the scene.

JonB

@Zdenek
This confirms my query/suspicion. It is not the add/remove of QGraphicsItems which is "slow", rather it is apparently whatever needs to deal with "overlapping" at the QGraphicsScene level. Still not sure what that would be. FWIW, did you try @Asperamanca's

On QGraphicsScene, set the item index method to "NoIndex" (QGraphicsScene::setItemIndexMethod)

BTW, while we are here: (a) just how "slow" it is (how long for how many items) and (b) how often (e.g. per second) are you needing to repopulate?

P.S.
Another possibility, while I think of it. As it stands every time you add (or remove or move) a QGraphicsItem from/to/on the scene the QGraphicsView will receive some signal and do some work. Have you tried: detach QGraphicsView from the scene, repopulate the scene then re-attach the QGraphicsView? If that shows much faster timing we can work from that i think.

Zdenek

@JonB thank you for you fast and valuable responses, I'm a new user of this forum and therefore I can add new comment every 10 minutes.

You are right, the population (addItem) is fast, it takes long time to render the scene for the first time(???) (I did the timing using "40000 chips" example)

On QGraphicsScene, set the item index method to "NoIndex" (QGraphicsScene::setItemIndexMethod)

It did not help.

JonB

@Zdenek
You should also check the detach/re-attach the view to the scene I suggested in previous post.

Let's eliminate one other possibility. Let's say we cannot make the QGraphicsItem approach any faster, for whatever reason. Do you actually need QGraphicsItems at all? You do if the user is going to interact with the items or if you going to move them around in code, etc. However, if all you want is just to draw thousands of "shapes" you can do that via QGraphicsScene::drawBackground/Foreground() and that should be "miles" faster?

Asperamanca

@JonB said in Slow QGraphicsView, slow "40000 Chips example":

@Asperamanca
Have you timed whether re-using, with its associated hides & shows, is really that much faster than remove & add? It may be, but there are also overheads for hidden items.

I learned that once in an Advanced Graphics View training (2010, but GraphicsView hasn't changed much since then)

Asperamanca

@Zdenek I believe the issues with overlapping items mostly come from the ItemIndexMethod. Here, it won't matter whether you remove/add or reuse items, because the BSP tree has to update whenever an item geometry changes, no matter the reason.

The "NoIndex" should remove that overhead. It will make hit testing (potentially) much slower.

There could be additional performance issues, though.

Asperamanca

You could try changing setCacheMode on QGraphicsView. Since you don't re-paint the same items often, caching would incur an overhead without gain.

Asperamanca

Further things to play around with:

QGraphicsView::setViewportUpdateMode
QGraphicsView::setOptimizationFlags
QGraphicsItem::setFlags (esp. ItemDoesntPropagateOpacityToChildren and ItemIgnoresParentOpacity)

Shuyunyun

@Zdenek I encounter the same problem as you, don't know how to solve it yet, i am continue watching on this.

SamiV123

This code is so bad, first off calling virtual paint on every time an item is painted is a bad way of doing things. (data caches will be trashed with bad locality of data)

Besides that, your 'new' will be problematic.

You should at least allocate your graphics items in a single chunk.