"corrupted size vs. prev_size while consolidating" or "double free or corruption (!prev)" from thread

DieterV

Hi,

Reasonable amount of Python experience, reasonably new to PyQt, completely new to using threading.

My use case is a piece of software that is used at the entrance of events. People come in, their invitation (qr code) is scanned, matched to an invitation in a database and then a badge gets printed.

This has been working well, except that generating the pdf and sending it to the printer takes a relatively long time (easily 500ms). During this time, the gui is unresponsive which the hostesses indicate as bothering them.

So I've been trying to improve this. I thought I could put all of the pdf generating (using PIL and pdf2image) and printing (using QPrinter) code into a separate class (subclassed from QObject).

The relevant code where I create thread is:

self.thrd_badge_printer = QtCore.QThread()
self.badge_printer = BadgePrinter(
            config=self.config,
            base_path=base_path,
        )
self.badge_printer.moveToThread(self.thrd_badge_printer)
self.badge_printer.sgn_printing_finished.connect(self.thrd_badge_printer.quit)
self.sgn_add_badge_to_queue.connect(self.badge_printer.add_badge_to_queue)
self.thrd_badge_printer.start()

After scanning a badge and doing the database stuff, I emit the self.sgn_add_badge_to_queue signal with the data that is needed to generate the badge (user name/lastname/..., placement data for the badge layout, ...).

The pdf gets generated correctly, but unless I comment out the last line (painter.end() at the end of printing), the whole application just shuts down. No additional logs indicating the issue, but I get either "corrupted size vs. prev_size while consolidating" or "double free or corruption (!prev)" error message printed to the terminal.

Ideally I'd like to learn a bit on how to troubleshoot this myself, but I might just not have enough knowledge yet.

I have a couple of questions:

Given that I'm very much still learning, I heavily rely on logging to troubleshoot my own code. I find I usually get no logs for issues when using threading. Is there a good way to "work around" this or get insight in what is going on?
I think I did the thread creation as it should, first creating the QThread, then the "worker" (subclassed from QObject), then .moveToThread, then connecting the signals, then start the thread. Right?
I try to keep everything in the worker separate as much as possible. I don't call any of its methods (except through a connected signal as stated above). The only object I'm passing when emitting the signal is the object of the class I created that has all the information needed to generate the badge. As far as I know, nothing is "changed" in this object in the "worker". Apart from that (and the fact that the worker uses PyQt6 objects), there are no shared objects. At least, I think :).
Obviously I'm doing something wrong. I read somewhere you can't do any operations on pixmaps, maybe there are other things I simply don't know about. Anything "obvious" I might have missed?

Thanks!
Dieter

JonB

@DieterV
Qt UI applications require that no secondary thread can access any UI (like widgets) objects. All UI operations must only be done from the main thread. Main and secondary threads can communicate via signals/slots, but the secondary threads must not read or write in the UI.

I don't know how this relates to printing. It may be that neither QPrinter nor QPaintDevices must be accessed in a secondary thread. QPainter is in the Qt GUI namespace so that may be forbidden in a thread. You also mention "operations on pixmap". IIRC QPixmap counts as a GUI element and also must not be manipulated in threads. The only "picture" class which can be used in a thread is QImage.

I don't know quite what you should do with this information, but it might be the cause of your problems.

hualas

This post is deleted!

DieterV

@JonB

Thanks. Unless someone else objects (or "corrects me" I mean), I'll take that as:

"don't use any classes from the PyQt6.QtGui namespace in a separate thread"

Those include QPageLayout, QPageSize as well as QPainter, all of which I use during printing.

Strange thing is that I haven´t seen such warnings anywhere. To be fair, I haven't found examples proving the opposite (using QtGui namespace in a separate thread). I guess it just isn't as simple as that but might work as a "rule of thumb" until I understand better. If you -by any chance- could recommend a book providing more clarification about this, I'd gladly order it.

I could probably rewrite that whole part to not use any PyQt, but frankly I'm glad I got it working up to this point so I'll leave it this way for now.

SGaist

Hi,

You can paint on a QImage in a different thread.
However, everything that is a QWidget cannot be. It's an old established rule that everything GUI related has to happen in the GUI thread.

DieterV

Thanks again for your reply.

In that case (and unless I'm understanding you wrong), I'm not sure what I'm doing wrong.

These are my imports in the second thread:

import pdf2image
import PIL.Image
import PIL.ImageQt
import PyQt6.QtCore as qtc
from PyQt6.QtCore import Qt
from PyQt6.QtGui import QPageLayout, QPageSize, QPainter
from PyQt6.QtPrintSupport import QPrinter
from PyQt6.QtSvg import QSvgRenderer

I'm not painting anything GUI related. I'm generating a pdf, paint it to the printer, then create an svg, paint it to the printer as well, generate a second page on the printer and paint another pdf.

It seems that, if I comment out the final painter.end(), no crash is caused. But then, obviously, nothing gets printer either :).

I do realize some example code would probably help, if needed I can try to write a compact example that exhibits the same issues to demonstrate what I'm doing.

Anyway, two specific questions:

@SGaist said in "corrupted size vs. prev_size while consolidating" or "double free or corruption (!prev)" from thread:

It's an old established rule that everything GUI related has to happen in the GUI thread.

I'm pretty new to PyQt, so not aware of "old established rules". Is there anywhere I can learn about these rules? Would probably help avoid issues as well as forum posts asking for help :)
What is the best way to troubleshoot these kind of issues when using threads where the application just crashes with almost no output?

Once again, thanks for your help...

Anonimista

@DieterV said in "corrupted size vs. prev_size while consolidating" or "double free or corruption (!prev)" from thread:

I can try to write a compact example that exhibits the same issues to demonstrate what I'm doing

That would be nice