Why don't the signals emit from QThread ?

BjornW

I don't really understand your code, but it is rarely a good idea to run Qt stuff inside a while(true) kind of construction.

kshegunov

I'm pretty sure your problem is your ConsoleStream object's in the main thread so it doesn't get the slots delivered. It's created here.
However, I share @SGaist's sentiment - you need to provide MREs if you expect people to take their time. No one wants to sift through dozens of classes to try to make sense of your code.

daegontaven

@BjornW @SGaist @kshegunov Okay I did what you guys wanted and added an MVCE so you guys can easily test the code. I hope this helps. Thank you so much for looking into this. I just saw you replies because I had given up on the project. But I feel now my efforts may not yet be in vain.

Here is the updated stackoverflow question. Here is a gist of main.ui and main.py so it's easier to highlight lines. Thank you once again.

^_^

daegontaven

@kshegunov said in Why don't the signals emit from QThread ?:

I'm pretty sure your problem is your ConsoleStream object's in the main thread so it doesn't get the slots delivered. It's created here.
However, I share @SGaist's sentiment - you need to provide MREs if you expect people to take their time. No one wants to sift through dozens of classes to try to make sense of your code.

I'll be referencing the gist from now on, so it's easier to locate where everything is.

Okay but PythonInterpreter is already inside it's own thread as shown here. Does it not apply to self.stream because of the multiple inheritance ? Or am I missing something ?

BjornW

Please provide a minimal example. There is too much, too confusing code :/

daegontaven

@BjornW What's confusing about the code I refactored ? There are no extra widgets, no unnecessary styling, no optimization of any sorts, lots of docstrings and just a few classes. Please point out what's confusing instead of saying it just is. Did you run the program ? It is literally two files and just 240 lines. I don't think it's easy to make a console for a Python interpreter under 160 lines using threads(Interpreter variables need to be accessed) . Ofcourse if it was that easy, I would have solved it through looking at documentation. This is the absolute minimal version. I only used parts detrimental to the functionality of a console.

Sorry. My temper got the best of me.. I've been at this too long and no one seems to know the answer. The simplest version which was 160 lines didn't even have a buffer and couldn't run a while loop. I keep telling people it's not possible. They keep telling me MCVE. Can't take the engine out and call it a car. Can't the take the chassis out and try to drive it either. Tell me what I can do to make this better instead of just saying MCVE. What do I MCVE.. Is there even anything left ?

I really need some coffee.

BjornW

@daegontaven

Its cool. I'm not saying your code is bad, I'm just saying it is not very easy to jump into someone elses code, regardless how good it is.

Now, if you create a thread, and by that i mean just a thread. No buffers and stuff, and create some object to emit a signal, is it "late"? I suppose not, after that, just keep adding stuff until you can actually reproduce the "late" behavior with the absolutely least amount of code. It is a good idea, even for your own sake, to try to resolve the issue like that. It is very easy to get lost in other workings of your code.

I'd like to try the following:

In your DelayedBuffer you create a timer like so:

L50: self.timer = QTimer()

This timer is not a child of the DelayedBuffer and therefore will not move with the buffer into a new thread. This means it will be executing in the thread where it was created (?). Try changing to

L50: self.timer = QTimer(self)

Otherwise:
Have you tried flushing your outputs?
Why do you call processEvents?

kshegunov

@daegontaven said in Why don't the signals emit from QThread ?:

Okay but PythonInterpreter is already inside it's own thread as shown here. Does it not apply to self.stream because of the multiple inheritance ? Or am I missing something ?

That a QObject belongs to a thread is something Qt introduces. In regular C++ there's no such thing - the data (object) is separate from the thread the methods are run in. In your case the constructor of PythonInterpreter is run in the same thread as the one the calling function is run in, meaning everything you do in that constructor is again in the same thread. As you're creating an object in said constructor that object is going to be in the same thread, understand where I'm getting with this? If you later call moveToThread that applies only to the one object, not to all the objects created in the constructor. So to make this work properly Qt has introduced object ownership - i.e. one object is a child to another object. If an object's moved to a thread all its children are moved to the same thread. However you don't set a parent to the object you create in the constructor so those objects have no parent, thus are not children to the PythonInterpreter instance, and finally are not going to be moved to the PythonInterpreter's object's thread. There are 2 things that can be done to solve this:

Always set the parent for any QObject instance you create, so you have a nice tidy and predictable object tree.
Move all the objects manually to the required thread, which is quite cumbersome.

More on ownership and trees you can find here and QObject::moveToThread documentation specifically refenreces which objects are moved and what are the limitations of this function.

daegontaven

@kshegunov I updated the gist. Is this how you set a parent ?

Or do I have to call setParent ? Which object do I call that method on ?

kshegunov

self.stream = ConsoleStream(this)

would be my guess (I don't know python, so you need to adjust that line accordingly).

Which object do I call that method on ?

For each QObject derived class' instance you want to be moved with the parent to the thread. As is now, the PythonInterpreter instance is moved to a thread, here:

self.interpreter.moveToThread(self.thread)

but the objects you create in its constructor, i.e:

self.stream = ConsoleStream()

are not, as they have no parent associated with them.

daegontaven

@kshegunov Thank you so much for responding. I tried your method(setting a parent). It actually improved the response time significantly. Much less lag on windows machines now. But the blocking issue still persists. It would appear this issue has never been solved in all these years. So I've decided to abandon the QPlainTextEdit strategy. Is it possible to use QListView instead? How would I go about changing the code so it would work as close as possible to QPlainTextEdit?

SGaist

It's not a question of unsolved issue, QPlainTextEdit has never been designed to handle console like behaviour. AFAIK, what people do is to use a heavily styled QListView + QStyleItemDelegate combo with a custom model to allow more fine grained updates and avoid performance issue by using a rolling window over the data.

daegontaven

@SGaist Yes, that's what I meant to say. I wish it was built for faster processing of live data too. I think the issue with the signal from the buffer was that the emits are being queued in the event loop and unfortunately Python has a GIL which might be blocking the concurrent execution. So it might be an issue inherent to Python and not Qt per se.

As for the QListView, I think I understand what you're getting at when you say"rolling window". I did initially use QListView before ever starting with a buffer, but I saw that the performance was still bad. But now I think it was because like there is a setMaximumBlockCount() method for QPlainTextEdit, I never removed the items from the model. I was using QStringListModel and not a custom model. However, I am not familiar QStyledItemDelegate or custom models either. Could you direct me to some sources or examples that implement something like this? Thank you so much for the help.

Edit:
After some digging around, I found promising documentation for model view programming. I think this is what I was looking for. I want to thank everyone for sticking with my mess and helping me anyway. I'll mark this thread as closed. Thank you guys/gals once again.

daegontaven

Guys/Gals, I guess I gave up too quickly because someone on stack-overflow solved the problem. I'm just going to leave the solution here in case someone needs it in the future.

Here is the change that fixed the issue.

daegontaven

@SGaist @kshegunov I am afraid I have to change the status of this thread to unsolved because after successfully making a buffer that works with QPlainTextEdit (To remove delay in output). The problem started again because I set parents for every child of PythonInterpreter except for itself so I can move it to a thread when using the new buffer. Now the signals can't be emitted anymore again. It might not be a matter of them not emitting, but they are perhaps being queue in the EventLoop of the thread while InteractiveConsole.runcode is blocking.

When I say the signals can't be emitted, they can't be emitted when/if I run a blocking function like time.sleep() is run in the interpreter. So this subsequently makes InteractiveConsole.runcode blocking. Is there a way around this ?

Here is the new MCVE(Which is shorter now).

This is the signal not being emitted.

Any help would be amazing. Thank you !

SGaist

What would you want to happen while that blocking code runs ?

daegontaven

@SGaist I need the slot send_console_log to be called when the signal written in ConsoleStream is emitted. When running the program with a normal while/for loop in the interpreter, the slot will be called becuase at that moment its non blocking. But the moment I add time.sleep into a loop, slots will only be called at the end when all the blocking is over. Thank you for looking into this.