Execute Code on multiple Cores
-
Hello all!
I am in sort of a strange situation here. my code makes use of QThreads to do different single things on multiple threads. (windows 10 machine, with an i7, 4 cores 2 Threads each.)
Recently I installed Twincat 3 on the same machine, and isolated 2 cores, so they are not available to the OS, and windows has only 2 cores to work with.
my software consists of 2 parts, the first one is yolo, and the second one is my part of the code that saves images and does some more stuff..
What I saw, is that as expected, the Yolo part slowed down a lot with only two cores, but my part of the code that uses QThreads has sped up by over 30% with only two cores.
My assumption, is that the code using Qthreads runs on a single core and is not affected by the reduced amount of cores, and that the fact that there are only 2 cores causes yolo to increase the clock speed of the CPU, thus speeding up the execution of the QThreads. Could that be possible?
I'd Like to run these tasks that at the moment run on different threads on different cores. Can that be done? At the time each Thread is getting its data with signals, would signals work also if the code runs on different Cores? I am a bit confused from all this, so i hope that my question makes sense.
Any help is greatly appreciated.
Thank you
Best regards
Igor
-
Hello all!
I am in sort of a strange situation here. my code makes use of QThreads to do different single things on multiple threads. (windows 10 machine, with an i7, 4 cores 2 Threads each.)
Recently I installed Twincat 3 on the same machine, and isolated 2 cores, so they are not available to the OS, and windows has only 2 cores to work with.
my software consists of 2 parts, the first one is yolo, and the second one is my part of the code that saves images and does some more stuff..
What I saw, is that as expected, the Yolo part slowed down a lot with only two cores, but my part of the code that uses QThreads has sped up by over 30% with only two cores.
My assumption, is that the code using Qthreads runs on a single core and is not affected by the reduced amount of cores, and that the fact that there are only 2 cores causes yolo to increase the clock speed of the CPU, thus speeding up the execution of the QThreads. Could that be possible?
I'd Like to run these tasks that at the moment run on different threads on different cores. Can that be done? At the time each Thread is getting its data with signals, would signals work also if the code runs on different Cores? I am a bit confused from all this, so i hope that my question makes sense.
Any help is greatly appreciated.
Thank you
Best regards
Igor
@Igor86 said in Execute Code on multiple Cores:
and that the fact that there are only 2 cores causes yolo to increase the clock speed of the CPU, thus speeding up the execution of the QThreads. Could that be possible?
It's the OS and the CPU who manage the clock speed of the CPU. It is possible that if only two cores are active those are running at higher speed compared to the situation when all cores are used.
"I'd Like to run these tasks that at the moment run on different threads on different cores" - this should already be the case. Why do you think it is not the case. It is the job of the OS to schedule processes and threads across available cores. Keep in mind that if a thread is waiting (not activelly!) for something it is not active and does not cause any load on any core!
-
@Igor86 said in Execute Code on multiple Cores:
and that the fact that there are only 2 cores causes yolo to increase the clock speed of the CPU, thus speeding up the execution of the QThreads. Could that be possible?
It's the OS and the CPU who manage the clock speed of the CPU. It is possible that if only two cores are active those are running at higher speed compared to the situation when all cores are used.
"I'd Like to run these tasks that at the moment run on different threads on different cores" - this should already be the case. Why do you think it is not the case. It is the job of the OS to schedule processes and threads across available cores. Keep in mind that if a thread is waiting (not activelly!) for something it is not active and does not cause any load on any core!
@jsulm Hello, thanks for your answer..
I have been reading around these days, and it was my understanding (probably wrong...) that QT in Python does not support running on multiple cores natively. It runs on multiple threads, but on the same core tough..
-
@jsulm Hello, thanks for your answer..
I have been reading around these days, and it was my understanding (probably wrong...) that QT in Python does not support running on multiple cores natively. It runs on multiple threads, but on the same core tough..
-
@jsulm Hello, thanks for your answer..
I have been reading around these days, and it was my understanding (probably wrong...) that QT in Python does not support running on multiple cores natively. It runs on multiple threads, but on the same core tough..
@Igor86 said in Execute Code on multiple Cores:
that QT in Python does not support running on multiple cores natively. It runs on multiple threads, but on the same core tough..
For sure wrong.
But easy to check: write a small application which spawns as many threads as you have cores and each thread consumes 100% cpu time. Start the app and check the CPU utilisation. -
@Igor86 said in Execute Code on multiple Cores:
that QT in Python does not support running on multiple cores natively. It runs on multiple threads, but on the same core tough..
For sure wrong.
But easy to check: write a small application which spawns as many threads as you have cores and each thread consumes 100% cpu time. Start the app and check the CPU utilisation.For sure wrong
https://stackoverflow.com/a/7542983
Python threads cannot take advantage of many cores. This is due to an internal implementation detail called the GIL (global interpreter lock) in the C implementation of python (cPython) which is almost certainly what you use.
The workaround is the multiprocessing module http://www.python.org/dev/peps/pep-0371/ which was developed for this purpose.
Documentation: http://docs.python.org/library/multiprocessing.html
https://stackoverflow.com/questions/53722818/python-how-to-make-use-of-multiple-cpu-cores
In general, you're right: you'll use one CPU core with one python process. However, there are many ways which allow you to use more than one CPU core. Have a look at the official Python docs about multiprocessing.
Etc.?
-
For sure wrong
https://stackoverflow.com/a/7542983
Python threads cannot take advantage of many cores. This is due to an internal implementation detail called the GIL (global interpreter lock) in the C implementation of python (cPython) which is almost certainly what you use.
The workaround is the multiprocessing module http://www.python.org/dev/peps/pep-0371/ which was developed for this purpose.
Documentation: http://docs.python.org/library/multiprocessing.html
https://stackoverflow.com/questions/53722818/python-how-to-make-use-of-multiple-cpu-cores
In general, you're right: you'll use one CPU core with one python process. However, there are many ways which allow you to use more than one CPU core. Have a look at the official Python docs about multiprocessing.
Etc.?
@JonB Yeah, I was wrong. Didn't touch Python much for many years already.
-
@jsulm
It's this Python "GIL" stuff which makes it different. My understanding is that whenever you execute any Python statement it becomes single-core (single-thread?) because the interpreter cannot run multiple in parallel.@Igor86
I'm not sure how this affects your code. If one thread(?) of yours is running non-Python code ("yolo"??) that may not be affected by the Python GIL limitation. Otherwise I think you might want to read about this Python "multiprocessing" module. -
@jsulm
It's this Python "GIL" stuff which makes it different. My understanding is that whenever you execute any Python statement it becomes single-core (single-thread?) because the interpreter cannot run multiple in parallel.@Igor86
I'm not sure how this affects your code. If one thread(?) of yours is running non-Python code ("yolo"??) that may not be affected by the Python GIL limitation. Otherwise I think you might want to read about this Python "multiprocessing" module.@JonB thank you for your answer!
its all python code. yolo runs on python too. yolo is not a problem here, what I was trying to state is that by reducing the cores yolo gets slower, my other code that makes use of QThreads not. Thus, I assumed that yolo runs python on multiple cores, and my code that makes use of the QThreads runs on a single core and is not affected by the reduced core count.
The question (sorry, I realize that it is not really obvious here and I couldhave had a better wording) is if QT has an alternative to QThreads, that alows me to run on multiple cores instead of multiple threads. I'll have a look at the links you shared.
Thank you!
