Qt Creator: Linker errors in shadow build only

[richardjdare]

Hi folks,
I'm having a problem with a qmake project that won't build using shadow builds, but will build with shadow builds turned off.

I'm using Qt Creator 13.0.2 and Desktop Qt 6.7.2 MSVC2019 64bit on Windows 10

When I try a shadow build, it fails to link with the following errors:

parser.obj : error LNK2005: ts_parser_new already defined in lib.obj
parser.obj : error LNK2005: ts_parser_delete already defined in lib.obj
parser.obj : error LNK2005: ts_parser_language already defined in lib.obj

.. (and a bunch more for other functions in parser.obj)

It works fine when I turn off shadow builds.

I made a minimal test program that demonstrates the problem. Here is its .pro file:

TREE_SITTER_SRC_ROOT = F:/projects/buildtest/src/3rdparty/tree-sitter/lib/src
TREE_SITTER_PARSER_SRC_ROOT = F:/projects/buildtest/src/3rdparty/tree-sitter-commonlisp/src

TEMPLATE = app
TARGET = SLIDE
QT += core gui widgets network svg

CONFIG += c++14
CONFIG += console
CONFIG -= qml_debug

SOURCES += \
    main.cpp \
    testmainwindow.cpp \
    $${TREE_SITTER_SRC_ROOT}/lib.c \
    $${TREE_SITTER_PARSER_SRC_ROOT}/parser.c

RESOURCES += \
    slide.qrc

HEADERS += \
    testmainwindow.h \

#tree-sitter
INCLUDEPATH += F:/projects/buildtest/src/3rdparty/tree-sitter/lib/src
INCLUDEPATH += F:/projects/buildtest/src/3rdparty/tree-sitter/lib/include

The program itself does nothing but open a QMainWindow.

Tree-sitter is a C library, included as per the documentation. (https://tree-sitter.github.io/tree-sitter/using-parsers) I've been using it happily for a long time with this exact setup.

My app worked fine until last week when my Qt install stopped being able to build anything, including empty new-project-wizard projects. I reinstalled Qt and Qt Creator and since then my app won't build unless I turn off shadow builds.

I have a Linux machine running Qt Creator 13.0.0, and Qt 6.4.0 and the app (and the minimal test app) builds fine on that.

I'd be grateful for any insights.

[Christian Ehrlicher]

Lib.c already includes parser.c so the symbols are there twice. Remove parser.c

[richardjdare]

Thanks for your reply,

I should probably explain a little more.
lib.c is the tree-sitter 'library'.

parser.c is separate. It's a specific parser generated by other software in the tree-sitter system for use with lib.c.
In this case, it is a Common Lisp parser (from https://github.com/tree-sitter-grammars/tree-sitter-commonlisp)
There are many other parsers available for parsing other languages.

I implemented a basic example inside my test application:
(These are now the contents of testmainwindow.cpp referenced in the .pro file in my original post)

#include "testmainwindow.h"
#include <tree_sitter/api.h>

extern "C" { TSLanguage *tree_sitter_commonlisp(); }

TestMainWindow::TestMainWindow(QWidget *parent)
    : QMainWindow{parent}
{
    QString testStr = "(defun foo (x)\r\n  (dotimes (i x)\r\n    (format t \"hello, world ~a~%\")))";
    auto utf8Str = testStr.toUtf8().toStdString();

    TSParser *parser = ts_parser_new();
    ts_parser_set_language(parser, tree_sitter_commonlisp());

    TSTree *tree = ts_parser_parse_string(
        parser,
        nullptr,
        utf8Str.c_str(),
        static_cast<uint32_t>(utf8Str.length())
    );

    TSNode root_node = ts_tree_root_node(tree);
    QString s =  QString(ts_node_string(root_node));
    qDebug() << s;
}

The 'extern "C" TSLanguage *tree_sitter_commonlisp()' comes from parser.c, the other ts_* functions are from lib.c

If I exclude parser.c and build "testmainwindow.cpp" I get "unresolved external symbol tree_sitter_commonlisp"
If I exclude lib.c I get unresolved externals for the ts_* functions
if I keep them both, I get "parser.obj : error LNK2005: ts_parser_new already defined in lib.obj" etc.

If I keep them both, and turn off shadow build, the app works. It used to work for me in the past with shadow build. It still works on my Linux machine running an older Qt and Qt Creator, that's what I don't understand.

[hskoglund]

Perhaps verify that your Qt installation is kosher, you could try building a project from the examples directory included with Qt 6.7.2,

[richardjdare]

Thanks for your suggestion. I tried the Mandelbrot example. I ran CMake then tried to build and got the following error:
ninja: error: build.ninja:184: multiple outputs aren't (yet?) supported by depslog; bring this up on the mailing list if it affects you
(I admit I don't have much experience with CMake yet)

Then I used the New Project wizard to create a qmake Qt Widgets Application. When I tried to build it I got the error:
:-1: error: dependent '............\Qt\6.7.2\msvc2019_64\include\QtWidgets\QMainWindow' does not exist.

I turned off shadow builds, and it built ok.

So something is definitely wrong. I actually had similar errors last week, trying to create a small test app, which prompted me to reinstall Qt. Since there is still a problem it may be my system has some underlying issues. I will investigate my MSVC install too.

Thank you both for taking time to help.

[Christian Ehrlicher]

Look into lib.c and you will see that parser.c is included. The linker tells you the same... But feel free to ignore me and the linker.

[richardjdare]

@Christian-Ehrlicher If you read my earlier reply, you'll see that I did in fact try to build it without parser.c as you suggested. (and conversely, without lib.c, as an experiment)

The 'extern "C" TSLanguage *tree_sitter_commonlisp()' comes from parser.c, the other ts_* functions are from lib.c

If I exclude parser.c and build testmainwindow.cpp I get "unresolved external symbol tree_sitter_commonlisp"
If I exclude lib.c I get unresolved externals for the ts_* functions
if I keep them both, I get "parser.obj : error LNK2005: ts_parser_new already defined in lib.obj" etc.

If I keep them both, and turn off shadow build, the app works.

Can you tell me why the linker doesn't complain if I turn off shadow builds, or why I can build and run the project without error on my Linux machine? I've been using these libraries in this very way for many months without complaint from the linker.

I believe I am using the library correctly, as described by these instructions (https://tree-sitter.github.io/tree-sitter/using-parsers#an-example-program)

The evidence seems to suggest that my system is busted, since I can no longer build a even a basic app created by the project wizard.

[Christian Ehrlicher]

@richardjdare said in Qt Creator: Linker errors in shadow build only:

$${TREE_SITTER_PARSER_SRC_ROOT}/parser.c

The env var points to the wrong directory and therefore the wrong parser.c is used. Using the same file name twice in a project is bad design.

[richardjdare]

I'm sorry mate I appreciate your help, but I don't understand what you are suggesting. Lets look at the two variables:

TREE_SITTER_SRC_ROOT = F:/projects/buildtest/src/3rdparty/tree-sitter/lib/src
TREE_SITTER_PARSER_SRC_ROOT = F:/projects/buildtest/src/3rdparty/tree-sitter-commonlisp/src

they are used like so:

SOURCES += /
  $${TREE_SITTER_SRC_ROOT}/lib.c \
  $${TREE_SITTER_PARSER_SRC_ROOT}/parser.c

Lets print these out from the .pro file using message():

message($${TREE_SITTER_SRC_ROOT}/lib.c)
message($${TREE_SITTER_PARSER_SRC_ROOT}/parser.c)

which gives us:

Project MESSAGE: F:/projects/buildtest/src/3rdparty/tree-sitter/lib/src/lib.c
Project MESSAGE: F:/projects/buildtest/src/3rdparty/tree-sitter-commonlisp/src/parser.c

These are correct. I want lib.c from tree-sitter. and parser.c from tree-sitter-commonlisp.

If this is indeed the problem, then why does the app build ok when I turn off shadow builds? Shouldn't I get the same compiler and linker errors whether I use shadow build or not? Why does it work fine on my Linux machine?

[richardjdare]

I've been investigating my system. I just want to add that my test program (and my larger app) works fine with shadow build if I change the kit to Desktop Qt 6.7.2 MinGW 64bit.

I also reimplemented my test app in Visual Studio 2022. not using Qt, but linking to the tree-sitter code in as similar a way as possible. (to see how msvc handled it in that environment) It worked fine.

[neit95]

Hi, I have a similar problem:

1. Create project with QWindow:

QT       += core gui

greaterThan(QT_MAJOR_VERSION, 4): QT += widgets

CONFIG += c++17

# You can make your code fail to compile if it uses deprecated APIs.
# In order to do so, uncomment the following line.
#DEFINES += QT_DISABLE_DEPRECATED_BEFORE=0x060000    # disables all the APIs deprecated before Qt 6.0.0

SOURCES += \
    main.cpp \
    MainWindow.cpp

HEADERS += \
    MainWindow.hpp

FORMS += \
    MainWindow.ui

# Default rules for deployment.
qnx: target.path = /tmp/$${TARGET}/bin
else: unix:!android: target.path = /opt/$${TARGET}/bin
!isEmpty(target.path): INSTALLS += target

Build dirrectory: C:\my\<project_dir>\build\debug
2. Build this.

I get the following error:

C:\Qt\Tools\QtCreator\bin\jom\jom.exe -f Makefile.Debug
Error: dependent '..\..\..\..\Qt\6.7.2\msvc2019_64\include\QtWidgets\QMainWindow' does not exist.
jom: C:\my\Repeater_2\build\Makefile [debug] Error 2

What I found out:
Makefile ( C:\my\<project_dir>\build\debug\Makefile.Debug) has such lines:

debug\moc_MainWindow.cpp: ..\..\MainWindow.hpp \
		..\..\..\..\..\..\Qt\6.7.2\msvc2019_64\include\QtWidgets\QMainWindow \
		..\..\..\..\..\..\Qt\6.7.2\msvc2019_64\include\QtWidgets\qmainwindow.h \
		..\..\..\..\..\..\Qt\6.7.2\msvc2019_64\include\QtWidgets\qtwidgetsglobal.h \
		..\..\..\..\..\..\Qt\6.7.2\msvc2019_64\include\QtGui\qtguiglobal.h \
...

But build catalog is C:\my\<project_dir>\build\debug\Makefile.Debug. It turns out that the path to Qt includes is generated incorrectly.
If build dirrectory change to C:\my\<project_dir> build will be correct.

[jsulm]

@neit95 How did you install Qt?

[neit95]

@jsulm I used Qt Installer. It is installed in C:\Qt