How to compile single Qt module and use it on app?

jsulm · _ _BPL_ 28 Jun 2019, 11:34

@_BPL_ said in How to compile single Qt module and use it on app?:

what are the steps to build qtbase?

In Qt source code tree go to qtbase and do

configure
make
make install

Or jom instead of make if you're using Microsoft compiler.

If you need an exact version then check out the proper tag (like for example https://github.com/qt/qtbase/tree/v5.12.3).

_BPL_ · wrote on 28 Jun 2019, 13:42

@jsulm I see, so your suggestion is basically using the default target, which means I'd have to compile a lot of stuff I won't need to find the underlying issue of the bug... that's exactly what I wanted to avoid in the first place, for instance, if you do:

git clone https://github.com/qt/qtbase
cd qtbase
git checkout -b 5.12.3 --track origin/5.12.3
configure.bat -prefix "d:/qt/5.12.3" -opensource -confirm-license -nomake examples -nomake tests
jom
jom install

That cold build would take me probably few hours.

Another thing I've tried was disabling all available-features (which I've got by using configure.bat -list-features), that would
end up giving errors though.

In any case, my question still remains. I just want to learn how to build a single module from sources without having to compile any unnecesary stuff, I've seen there is some config tool but that's only available in the commercial version.

SGaist · 28 Jun 2019, 21:25

Hi,

The qtbase module is some kind of an exception. It provides all the infrastructure needed for the other modules.

For all the others you can build them one by one.

JKSH · S SGaist 28 Jun 2019, 21:04

@_BPL_ said in How to compile single Qt module and use it on app?:

I just want to learn how to build a single module from sources without having to compile any unnecesary stuff

You can learn by practicing on a smaller, simpler module. Try building the Qt SVG module, for example.

find the underlying issue of the bug

Note that QTBUG-74492 talks about a bug in Qt Quick. Qt Quick does not use the Qt OpenGL module!

The Qt OpenGL module contains old widgets for rendering OpenGL, like QGLWidget. This module has been deprecated in favour of newer OpenGL classes and methods within the Qt GUI module

@SGaist said in How to compile single Qt module and use it on app?:

The qtbase module is some kind of an exception. It provides all the infrastructure needed for the other modules.

To be clear, qtbase.git is a repository that contains several modules.

See https://wiki.qt.io/Qt_5_Structure (it's outdated, but it shows the idea)

_BPL_ · 29 Jun 2019, 02:51

@SGaist @JKSH

Hi guys, today I've been all day long getting familiar about building qt from sources. Right now I've found a way to configure qtbase that's giving me a 20min cold build using jom in my old laptop .Then I just can rebuild individual modules in few seconds (warm builds) by cding in the module directories and run nmake or jom.

That said, I've also making a little bit of bug hanting... you can read my last comment here asking for help to Qt devs.

In any case, I'd like to know something which it's actually related to another question I've done here https://forum.qt.io/topic/104426/how-to-build-efficiently-different-qt-versions-from-sources.

Right now I know the bug 74492 isn't living in version 5.11.2 but it's living in version 5.11.3! So, let's say I'd like to use some binary search where I'll recompile and test intermediate commits in this range... obviously you wouldn't use something like git checkout -- . && git clean -fxd each time you checkout a new possible commit... otherwise you'd need to do cold builds all the time... but the question here is. Is it safe to configure after you've checked out a new qtbase commit and rebuilding?

Asking cos once you run configure.bat it warns about:

Prior to reconfiguration, make sure you remove any leftovers from
the previous build.

Hopefully you can advice, btw, are you guys Qt devs?

Thanks in advance :)

JKSH · 29 Jun 2019, 10:58

@_BPL_ said in How to compile single Qt module and use it on app?:

Right now I know the bug 74492 isn't living in version 5.11.2 but it's living in version 5.11.3!

Good work; this information should help identify the issue.

In any case, I'd like to know something which it's actually related to another question I've done here https://forum.qt.io/topic/104426/how-to-build-efficiently-different-qt-versions-from-sources.

...

So, let's say I'd like to use some binary search where I'll recompile and test intermediate commits in this range... obviously you wouldn't use something like git checkout -- . && git clean -fxd each time you checkout a new possible commit... otherwise you'd need to do cold builds all the time... but the question here is. Is it safe to configure after you've checked out a new qtbase commit and rebuilding?

Ok, so you want to do a git-bisect. Someone did that at https://bugreports.qt.io/browse/QTBUG-73715

I've never done this on Qt itself before, so I don't know what's the correct/safe way to do it.

are you guys Qt devs?

None of us who have replied here work for the Qt Company; we're just Qt "users". You can find Qt devs at the Development mailing list: https://lists.qt-project.org/listinfo/development (subscribe first, then you can post there)

_BPL_ · J JKSH 29 Jun 2019, 02:51

@JKSH said in How to compile single Qt module and use it on app?:

Ok, so you want to do a git-bisect. Someone did that at https://bugreports.qt.io/browse/QTBUG-73715
I've never done this on Qt itself before, so I don't know what's the correct/safe way to do it.

Probably... if this project was as easy changing the commit + press 1 button and getting a fresh build I'd try that straightaway. Problem is a cold build right now is ~20min and If I switch to another commit and I reconfigure it (without cleaning) I'll get errors so I need to be careful with the test strategy :/ . Right now I've built v5.11.2 using a prefix of "d:/qt/5.11.2" and I've figured out all candidate commits that have potentially introduced the bug:

08de243eaa (tag: v5.11.3, 5.11.3) Add changes file for Qt 5.11.3                                      
c05080102f tst_QNetworkReply: Blacklist getFromHttp:success-external                                  
c69f43594f Merge 5.11 into 5.11.3                                                                     
9137691e74 Windows QPA: Fix crash showing QSystemTrayIcon's context menu with PROCESS_DPI_UNAWARE     
dec7961709 QSyntaxHighlighter: Delay all highlights until first rehighlight                           
bdebc90c28 Bump version                                                                               
38b87cc4bb Doc: Clarify what samples() returns if not explicitly set                                  
9933511838 Fix typo in define. s/GL_FRAMEBUFFER_SRB/GL_FRAMEBUFFER_SRGB                               
825f988156 Modernize the "textcodec" feature                                                          
b5d249f953 Update the floppy disk icon (save) to be physically correct                                
9f2216667a Fix memory copy in QGIFFormat::disposePrevious()                                           
033cc3403a mkspecs: use cross compile tools with LTCG                                                 
67c66c4ea4 windows: Give up on SwitchableComposition                                                  
c9d18d4a9c eglfs_kms: initialize m_deviceListener                                                     
1b9af84c1b Don't create an offscreen surface when not on the GUI thread                               
3b8075de3b Fix deleting of QSharedPointer internals in case QPointer loses the race                   
d4e937a628 xcb: Don't get initial screen rotation                                                     
d2e0e416d4 Fix leaking QTabletEventPrivate instance                                                   
38afa46c47 macOS: Only detect changes to the SDK version within the same developer dir                
509d566ec0 Don't block mouse events if the window is a Tooltip type                                   
72bedd49bf [cocoa] Disable offline renderers for dual AMD FirePro GPU                                 
0cb44e2cfb Fix stylesheet example for QLineEdit:read-only code example                                
2708c6c11d OpenSSL: force the "1.0.0" soname when loading OpenSSL 1.0                                 
948f8ce2ec QWinEventNotifier: fix crash on application shutdown                                       
44eeeb8e81 Upgrade PCRE2 to 10.32                                                                     
6599c1f758 QPicture: fix crash for malformed picture                                                  
7f60940fbe Re-disable statx() on Android                                                              
2624676b57 qmake: Remove the extra space before -MT                                                   
1cd2955173 Fix enum passed to QFontDatabase::findFont                                                 
fc4b0769a5 Fix pdf printing in static builds                                                          
b7887f9b4f Linux: Remove our use of syscall() for statx(2) and renameat2(2)                           
3eebadc173 Modernize the "mimetype" feature                                                           
9c8ca26a48 Modernize the "codecs" feature                                                             
4e7b58629a Modernize the "big_codecs" feature                                                         
c593492d16 Modernize the "animation" feature                                                          
0509383cf2 Bump copyright year in executable metadata                                                 
0d7c049e44 Update bundled libpng to version 1.6.35                                                    
dc5f9d0c31 Only use a translucent background if there is support for alpha                            
091a386eaf Use native visual ID when creating GBM surfaces for KMS                                    
4dc251879c Ssl: Fix contrived crash when calling resume                                               
ba0ff45109 Update the DNS public suffix list from publicsuffix.org                                    
836a2fb887 [macOS] Fix position of sheets when using unifiedTitleAndToolBarOnMac                      
7146c9075c Fix DejaVu fonts URL                                                                       
92f42caff1 Fix ICE on QNX 6.6                                                                         
04aeffbe8f Doc: Describe behavior of QSslConfiguration::caCertificates() on iOS                       
18ec0a8b09 Windows QPA: Fix WM_NCHITTEST not being sent to QAbstractNativeEventFilter                 
d8817ddde6 Use update() instead of repaint() when displaying a new message                            
ced34cb3d5 QDateTimeParser: avoid using an invalid hour by default                                    
5a295a1009 Scale seconds by a thousand to get milliseconds                                            
c958fb8b48 zlib: Fix spelling of license                                                              
b2b32d3147 fix HTML subset documentation is not very readable on smaller screens                      
caa598c843 Fix QUrl::matches for when removing authority parts (other than host)                      
94884246d4 QCommandLineParser: Ensure that an option text ends with a newline                         
ef4ba0285f SSL: Don't write to closed socket or write to deallocated buffer                           
3ed8dc3788 Android: fix log output pattern                                                            
857a0d4c51 Fix the /J option for MSVC project generation                                              
555a6b5d5d Modernize the "filesystemwatcher" feature                                                  
5e64957ee4 Fix QCompleter popups preventing the application from exiting                              
f99e956d65 Add QT_REQUIRE_CONFIG(ssl) to pre-shared key authenticator                                 
02663718a9 QHeaderView: Don't unhide hidden sections on layoutChanged()                               
b26cd68bf6 Modernize the "datestring" feature                                                         
e226b0f94a Modernize the "textdate" feature                                                           
6948bf20a7 QSslContext: Use 0 instead of TLS_MAX_VERSION                                              
4b7ff8e98c Protect HSTS code for no-feature-settings build                                            
4fc4f7b0ce Export qt_open64 from QtCore                                                               
8aa9bb6d3f Clarify docs regarding the states of a QFutureWatcher with no future set                   
12c357bebb Document IAccessible2 version                                                              
cb5c24fa26 Fix integer overflow in very long sections in ELF objects                                  
1511bfef52 Disable RGB64 backend for ARGB32 when it will be very slow                                 
6af8b5e791 Merge remote-tracking branch 'origin/5.11.2' into 5.11                                     
49efea26a5 sqlite: Fix QSqlError handling when opening/closing database                               
45c1473847 Detect when we are at the sentence boundary                                                
b0dce506cc (HEAD -> 5.11.2, tag: v5.11.2) add buildsystem+qmake changelog

Problem is I'm not sure what's the safest way to test them out without spawning a cold build on each new test :/ . I'm using visual studio compiler and I haven't found any "standard" alternative to ccache, just a bunch of github projects that looked kind of experimental.

In any case, you can see there are 73 commits over there to check and in the worst case scenario would be 73*20min if
cleaning before building, that's nuts and I definitely won't go down that path :)

Christian Ehrlicher · wrote on 29 Jun 2019, 11:02

@_BPL_ said in How to compile single Qt module and use it on app?:

In any case, you can see there are 73 commits over there

No, not 73 - max. 7 - see git bisect

_BPL_ · 29 Jun 2019, 11:37

@Christian-Ehrlicher said in How to compile single Qt module and use it on app?:

No, not 73 - max. 7 - see git bisect

Mmmm, interesting command indeed, it uses binary search... guess that 7 comes from ceil(log2(73))? Worst case of 7*20 isn't that bad (I guess :P)... I'll get familiar with that command first using some dummy projects for not screwing up with the real thing.

In any case, it'd be interesting to know what's the workflow used by qt developers, which I'm pretty much sure it'll be quite optimized so they can check different versions relatively fast.

JKSH · 29 Jun 2019, 13:55

@_BPL_ said in How to compile single Qt module and use it on app?:

Mmmm, interesting command indeed, it uses binary search... guess that 7 comes from ceil(log2(73))? Worst case of 7*20 isn't that bad (I guess :P)... I'll get familiar with that command first using some dummy projects for not screwing up with the real thing.

In any case, it'd be interesting to know what's the workflow used by qt developers, which I'm pretty much sure it'll be quite optimized so they can check different versions relatively fast.

See my previous post. It contains a link to an example where someone used git-bisect to hunt down a Qt bug. It also contains a link to the Developer mailing list where you can talk to the Qt developers directly.

_BPL_ · J JKSH 29 Jun 2019, 11:37

@JKSH said in How to compile single Qt module and use it on app?:

See my previous post. It contains a link to an example where someone used git-bisect to hunt down a Qt bug. It also contains a link to the Developer mailing list where you can talk to the Qt developers directly.

Thanks, but... I've decided to ask to one of the devs... It'd be great if git-bisect could be used effectively on windows+visual_studio although my hopes are quite low.

The closes I've been to avoid doing git checkout -- . && git clean -fxd was to checkout a possible bad commit, cd qmake && nmake clean and configure.bat <options>... in this case configure.bat wouldn't give me errors... but when building again qtbase times would still be as similar as a cold build :(

Christian Ehrlicher · 29 Jun 2019, 16:13

Why not simply using git-bisect on the command line and compile without any make clean before? I don't see any need to call configure.bat again - nothing configure specific will change inbetween those 72 commits in a stable branch - and if you will notice it during compilation.

_BPL_ · wrote on 29 Jun 2019, 16:13

@Christian-Ehrlicher said in How to compile single Qt module and use it on app?:

Why not simply using git-bisect on the command line and compile without any make clean before? I don't see any need to call configure.bat again - nothing configure specific will change inbetween those 72 commits in a stable branch - and if you will notice it during compilation.

Maybe you're right and probably that's the faster startegy for bug hunting using git-bisect on qtbase. Only advantage of cleaning & reconfiguring is you can keep the good/bad commits living in separate folders (each time I was calling configure.bat I was using as a prefix the result of git describe --tags commit_hash) but consider the only goal is bug hunting probably these good/bad commit folders shouldn't be kept anyway (are they useful for other purposes?)

Anyway, I've found the commit that introduced the bug and it's 1 year old, that's crazy :P !!!! It'd be really interesting to know how long would it take to hunt it using your method when working on latest master (configured&built ready).

I've learned quite a lot thanks to this bug but it's still unclear to me when it's required to reconfigure again before building... it'd be interesting to learn that for future bug huntings :)

_BPL_ · wrote on 29 Jun 2019, 16:36

Btw, for those interested on this issue I've posted the results of my bug hunting on the original thread.

Possible fixes are welcome :D

_BPL_ · wrote on 29 Jun 2019, 16:46

@Christian-Ehrlicher said in How to compile single Qt module and use it on app?:

Why not simply using git-bisect on the command line and compile without any make clean before? I don't see any need to call configure.bat again - nothing configure specific will change inbetween those 72 commits in a stable branch - and if you will notice it during compilation.

Btw... I've tested this suggestion and I can tell it won't work, give it a shot and you'll see, you'll probably end up with compiler/linking errors mostly of the time (even reconfiguring), so... yeah, unfortunately the only reliable way I've found to build a new local branch is by doing:

git checkout -- . && git clean -fxd :)
configure <options>
make

Not cool at all :/

Christian Ehrlicher · 29 Jun 2019, 17:44

You can be sure that I compile qtbase more than once a day and calling configure is only needed when switching branches.

_BPL_ · wrote on 29 Jun 2019, 17:44

@Christian-Ehrlicher Sure thing, just to be clear I don't doubt your suggestion works fin... my comment was based on the fact each time I've tried it out here I ended up having errors. Anyway, I'd really like to confirm it so I'll really know when I can apply it. Let's say I've checked out origin/5.12 [5d7e221bbf] and it's built succesfully. Now let's say I'd like to start bisecting without having to reconfigure, what'd be the oldest commit where I can be sure I don't need to reconfigure again? Could you please post the git hash of such a commit? I'd like to check it out.

Thanks in advance!

SGaist · 29 Jun 2019, 20:50

One small suggestion, use out of source builds. This will avoid the problem of cleaning stuff up in the source tree. If your build is broken for some reason, just nuke the folder and start fresh.

_BPL_ · S SGaist 29 Jun 2019, 20:38

@SGaist Actually... That's a really good advice, I didn't know it was possible to achieve with this particular qtbase project, I've been doing it wrong all the time :-P . Could you please explain how to do that?

SGaist · 29 Jun 2019, 22:19

Here is a generic *nix version but it's easy to translate to Windows style.

mkdir /some_path/builddir/qt5/qtbase
cd /some_path/builddir/qt5/qtbase
/path/to/Qt_sources/qtbase/configure

/some_path/builddir is a path you use for building stuff.

I usually keep the qt5/module_names structure so if I build an other module it follows the sources folders structure. It's by no mean mandatory just a habit of mine.