Compile Qt natively on Raspberry Pi 2/3

[sierdzio]

@jars121 said in Compile Qt natively on Raspberry Pi 2/3:

I'm expecting for it to take at least overnight, if not a couple of days.

I think the last time I tried (I skipped a bit more modules, see below) it took ~4 hours to compile on Raspberry Pi 3.

../5.9.0-src/configure -prefix $PWD/../5.9.0-bin -opengl es2 -confirm-license -opensource -release -nomake examples -skip qtwebview -skip qt3d -skip qtquick1 -skip qtwayland -skip qtvirtualkeyboard -skip qtwebchannel -skip qtwebengine -skip qtscript -skip qtscxml -skip qtspeech -skip qtpurchasing -skip qtgamepad -skip qtlocation

In my experience the biggest "offender" is QtWebEngine, it takes forever to build - so if only you can, I recommend skipping it.

[jars121]

@sierdzio That shouldn't be a problem, I've certainly not used QtWebEngine before. 4 hours on a RPi3 is pretty good; I've only got a RPi2 at the moment, so hopefully I can leave it to compile overnight.

[jars121]

Bit of an update. I tried compiling on the Pi last night using @sierdzio 's configure list, but ran into issues with XCB. It looks like not all the dependencies were captured in the 'build-dep qt5-default' command. Is XCB required if I don't plan on using X11? If not, is there a -skip xcb flag I can use in the configure? Edit: It looks like there's a 'no-xcb' flag which I'm going to give a go shortly.

[sierdzio]

Oops, sorry about that! I've tried compiling in a dozen different ways and wrote the results down, so probably I installed additional dependencies in another try.

Anyway, yes, XCB plugin is not needed if you plan only to use EGLFS.

[jars121]

Good news! It would appear that I've managed to successfully compile and install Qt on my Raspberry Pi 2! Thanks @sierdzio for all your help! It ended up taking around 12 hours as I forgot to 'make' with multiple cores, oops!

The only I've got now is when trying to 'wget' SIP prior to building PyQt5, I'm being told that my device has run out of storage. I'm using an 8Gb card, and a 'df' shows considerable space on the /boot partition, so I guess I should just resize my /dev partition to accommodate?

[jsulm]

@jars121 I'm not sure 8GB will be enough to build whole Qt. You could limit your build to the modules you need and omit building examples.

[sierdzio]

Hey, I'm glad to hear that! Happy coding, and good luck with PyQt (here I can't help, sorry).

If you have any "final" set of commands to build it, please post it here (or maybe create a wiki page), maybe it will help somebody in the future.

Space: Raspbian automatically extends the partition so you should already have max that is available and sane. You can try to resize /boot, but I recommend doing a backup/ clone of the SD card just in case it fails.

Other solutions:

• you can remove 5.9.0-src and 5.9.0-build folders if you have not done it already. That's because by using -prefix all compiled Qt libraries and tools should be in 5.9.0-bin
• use bigger SD card ;)
• plug in some USB HDD or pendrive, create partition there and use it to build PyQt5

[jars121]

Excellent, thanks both for your suggestions.

I hadn't yet removed the src or build folders, so I suspect there'll be considerable space saved there. I'll give that a go tomorrow and report back. I'll certainly post my finalised/refined build instructions when verified :)

[jars121]

Just another update, in case anyone is playing along at home :)

Since my last comment, I've installed Python 3.6 from source, and also tried building PyQt5 from source, pointing to my Qt 5.9.0 installation (qmake) and SIP. The installation was somewhat successful, but components of Qt seemed to be missing (QtGui for example). I could import PyQt5 within Python successfully, as well as some modules (such as QtCore). I have a feeling that the 8Gb card was the cause, as it showed 0kB of space available at the end of building Qt, which seemed highly unlikely that it would fully build Qt to completion and have exactly 0kB of space left.

As such, I've now built Qt (using the exact same process) on a 16Gb SD card (Sandisk Ultra, 80MB/s), which showed several gigabytes of space remaining once completed. Perhaps some components of the build were truncated when it ran out of space on the 8Gb card? It certainly didn't raise any errors, but I'm happy to err on the side of caution with the larger card.

I'll build Python, SIP and PyQt5 again this evening on the larger card and see if I have any issues with QtGui. The next step will be to go through my application and convert it from PySide (Qt4) to PyQt5 (Qt5). For the modules I'm using (almost exclusively QtCore and QtGui), the transition shouldn't be too onerous. One notable exception I've picked up so far is the splitting of Qt4 QtGui to Qt5 QtGui and QtWidgets.

[jars121]

Another update, as I've made considerable progress this week.

I ended up re-building Python, SIP and PyQt5 again (as per previous post), but was having issues with the -platform eglfs flag when running my sample .py file. I ended up having to ./configure my Qt build again with the -device and -device-option flags, as I hadn't used them initially.

I've now got Qt 5.9.0, Python 3.6.2, SIP and PyQt5 installed and working on my RPi2.

Unfortunately, this is where the good news ends. I've spent the last day or so working through my PySide (Qt4-based) application, porting it to PyQt5 (Qt5-based). I've established a PyQt5 development environment on my Windows 7 machine for development, and have the application running under PyQt5 as it does under PySide, which is great. I am similarly able to run the application under PyQt5 (with eglfs writing directly to the frame buffer) on my RPi, BUT the refresh rate is still abysmal. I'd estimate that the frame rate is the same, if not slower than when I was running the application under PySide WITH the full X server/window management setup.

I'm pretty confused by this, as the research I've done heavily suggested that writing directly to the frame buffer should be significantly faster. Perhaps hardware acceleration isn't being used? How can I test/confirm hardware acceleration is being used?

My next steps are to try and compile my application using nuitka. I've given it a quick go and received '[module] can't be imported absolutely' (or words to that effect) for seemingly every single linked module within my application. I haven't delved into this further, but I may have to given my current level of performance. I also intend on profiling my application to understand where the major bottlenecks are.

If anyone has any other pointers I'd be more than happy to hear them :)

[jars121]

I've built a profiler into my application, which will measure the number of calls and the time required to execute 600 refreshes of the main QWidget (representing approximately 10 seconds of application runtime at ~60Hz). On my development machine (i7, quad core with hyperthreading, etc.), even the most taxing modules only take a fraction of a second, so finding bottlenecks isn't particularly easy. I'm hopeful that when I run the same profiler on my RPi this evening I'll be able to detect significant bottlenecks given the RPi's relative performance.

[sierdzio]

About checking the OpenGL renderer: see this post https://stackoverflow.com/questions/36331404/check-which-opengl-engine-is-used-by-qt-at-runtime-for-release-builds

However, now that you've mentioned QtWidgets (I rather thought we were talking about QtQuick) - widgets are painted using raster engine (software rendering, using your main CPU). If you want to make sure the rendering is done using OpenGL, you should use QOpenGLWidget as your topmost widget. I think that forces all subwidgets to use it, too. Not sure, though, I've never actually used QOpenGLWidget (neither the older QGLWidget).

[jars121]

Thanks for that @sierdzio, I wasn't aware that QtWidgets were raster-based, I had assumed the paint method was determined by the underlying system OpenGL capability. I've had a look through some of the QOpenGLWidget documentation, but am really struggling to make sense of it; my c++ isn't the best, so trying to convert the syntax into Python on the fly is proving difficult.

The main paint event in my application is a QPainter (.drawPie, .drawEllipse and .drawPixmap), and some QWidgets with an SVG image on top which changes periodically. Ideally I'd be able to simply swap the QPainter raster process with a similar QPainter OpenGL process; I've not been able to determine the means for doing so as of yet.

In other news, I've been playing around with my new profiler, which gives the following (high-level) results (each measured in cumulative time with cProfile):

• Total execution time: Development Computer: ~10s, RPi2: 49.9s

• Main paintEvent(): Development Computer: 1.25s, RPi2: 36.6s

I commented out the entire paintEvent() function, and the RPi2 executed the 600 function calls in a little over 11s, so even in the non-graphical Python logic I need some considerable optimisation. I'll continue reading on the QOpenGLWidget potential of my QPainter functions; I'm weary that if I can't get that to work I'll need to consider more drastic measures (i.e. re-writing the entire application in c++).

[jars121]

@sierdzio you are a champion. I've replaced all QWidgets (and even some QMainWindows) with QtWidgets.QOpenGLWidgets, and the main application appears to be running at the full 60Hz on the RPi2, with a responsive mouse and interaction! I've still got a few issues to address (when I open a page on top of the main/home page it seems to be using software rendering), but this is a huge step forward. Thanks again for your help and input on the QOpenGLWidget, it would have taken me a considerable amount of time to stumble across that in the c++ documentation.

[sierdzio]

Nice progress! Good luck :-)

@jars121 said in Compile Qt natively on Raspberry Pi 2/3:

some QWidgets with an SVG image on top which changes periodically

SVG parsing and painting (no matter if in QWidgets or QML) is much slower than "normal" JPG or PNG images. So if you aim for performance, I recommend switching to PNG.

[raven-worx]

@jars121
just to note it here: www.qtrpi.com

[jars121]

@raven-worx How did I not find this earlier!? haha

Thanks for the link :)