QtCreator. CUDA. Make final linking with NVCC
-
Hello. I found many examples of CUDA + Qt in Internet, but they all were compiled with -c option of nvcc compiler. I need also -dc -rdc=true options too. When I enable them, I got error: undefined reference to cudaRegisterLinkedBinary. I found this topic
cuda shared library linking: undefined reference to cudaRegisterLinkedBinary on StackOverflow. How to implement it in QtCreator? As I understood it needs to make final linking with nvcc, not g++. -
Hi,
What are you using currently to build your CUDA kernels ?
-
This .pro file, that I downloaded from github and corrected for myself with my experience:
#------------------------------------------------- # # Project created by QtCreator 2019-12-10T19:31:27 # #------------------------------------------------- QT += core gui greaterThan(QT_MAJOR_VERSION, 4): QT += widgets TARGET = 13-word-correction-fnn TEMPLATE = app # The following define makes your compiler emit warnings if you use # any feature of Qt which has been marked as deprecated (the exact warnings # depend on your compiler). Please consult the documentation of the # deprecated API in order to know how to port your code away from it. DEFINES += QT_DEPRECATED_WARNINGS # You can also make your code fail to compile if you use deprecated APIs. # In order to do so, uncomment the following line. # You can also select to disable deprecated APIs only up to a certain version of Qt. #DEFINES += QT_DISABLE_DEPRECATED_BEFORE=0x060000 # disables all the APIs deprecated before Qt 6.0.0 SOURCES += \ main.cpp \ mainwindow.cpp HEADERS += \ mainwindow.h \ word-correction-fnn-cuda.h \ actfunc.h FORMS += \ mainwindow.ui QMAKE_CXXFLAGS += "-fno-sized-deallocation" # Cuda sources CUDA_SOURCES += word-correction-fnn-thrust.cu \ word-correction-fnn-cuda.cu \ actfunc.cu # Project dir and outputs PROJECT_DIR = $$system(pwd) OBJECTS_DIR = $$PROJECT_DIR/Obj DESTDIR = ../bin # Path to cuda toolkit install CUDA_DIR = /usr/local/cuda # GPU architecture #https://docs.nvidia.com/cuda/cuda-compiler-driver-nvcc/index.html#virtual-architecture-feature-list CUDA_VARCH = compute_35 #https://docs.nvidia.com/cuda/cuda-compiler-driver-nvcc/index.html#gpu-feature-list-dc -rdc=true CUDA_GPU_ARCH = sm_35 # nvcc flags (ptxas option verbose is always useful) NVCCFLAGS = --compiler-options -use-fast-math --Wno-deprecated-gpu-targets # include paths INCLUDEPATH += $$CUDA_DIR/include #INCLUDEPATH += /opt/cub/cub #Поддержка заголовочных файлов библиотеки CUB (Более производительная и низкоуровневая замена Thrust) # include paths #INCLUDEPATH += /usr/local/cuda/include # lib dirs QMAKE_LIBDIR += $$CUDA_DIR/lib64 # libs - note than i'm using a x_86_64 machine LIBS += -lcuda -lcudart -lcurand # join the includes in a line CUDA_INC = $$join(INCLUDEPATH,' -I','-I',' ') # Prepare the extra compiler configuration (taken from the nvidia forum - i'm not an expert in this part) cuda.input = CUDA_SOURCES cuda.output = ${OBJECTS_DIR}${QMAKE_FILE_BASE}_cuda.o #Улучшенный вывод сообещний об ошибках. Более понятный для QtCreator, чтобы можно было по двойному щелчку переходить к строке в файле #Параметр -dc __device__ функцию используемую __kernel__ функцией в одном файле, можно объявить/определить в другом файле #-dc == --device-c -> Compile each .c, .cc, .cpp, .cxx, and .cu input file into an object file that contains relocatable device code. # -rdc=true|false == relocatable device code #-dc is equivalent to --relocatable-device-code=true #--compiler-options '-fPIC' -dc −rdc=true #Если добавить -x, то nvcc будет воспринимать .cpp файлы, как .cu (содержащие cuda-код). #Должен будет более адекватно работать QtCreator, эти .cpp файлы всё равно надо будет происывать в CUDA_SOURCES. #Но для наглядности лучше оставить их .cu и эту опцию не использовать. Иначе можно запутаться. CONFIG(debug, debug|release) { # NEED: cuda.commands = $$CUDA_DIR/bin/nvcc -m64 -g -G -gencode arch=$$CUDA_VARCH,code=$$CUDA_GPU_ARCH -c -dc -rdc=true $$NVCCFLAGS \ !!!!!!!!!!!!!!!!!!!!! cuda.commands = $$CUDA_DIR/bin/nvcc -m64 -g -G -gencode arch=$$CUDA_VARCH,code=$$CUDA_GPU_ARCH -c $$NVCCFLAGS \ $$CUDA_INC $$LIBS ${QMAKE_FILE_NAME} -o ${QMAKE_FILE_OUT} \ 2>&1 | sed -r \"s/\\(([0-9]+)\\)/:\\1/g\" 1>&2 } else { cuda.commands = $$CUDA_DIR/bin/nvcc -m64 -O3 -gencode arch=$$CUDA_VARCH,code=$$CUDA_GPU_ARCH -c $$NVCCFLAGS \ $$CUDA_INC $$LIBS ${QMAKE_FILE_NAME} -o ${QMAKE_FILE_OUT} \ 2>&1 | sed -r \"s/\\(([0-9]+)\\)/:\\1/g\" 1>&2 } #Оригинал с GitHub #cuda.commands = $$CUDA_DIR/bin/nvcc -m64 -g -G -gencode arch=$$CUDA_VARCH,code=$$CUDA_GPU_ARCH -c $$NVCCFLAGS $$CUDA_INC $$LIBS ${QMAKE_FILE_NAME} -o ${QMAKE_FILE_OUT} cuda.dependency_type = TYPE_C # there was a typo here. Thanks workmate! CONFIG(debug, debug|release) { cuda.depend_command = $$CUDA_DIR/bin/nvcc -g -G -M $$CUDA_INC $$NVCCFLAGS ${QMAKE_FILE_NAME} | sed \"s/^.*: //\" } else { cuda.depend_command = $$CUDA_DIR/bin/nvcc -O3 -M $$CUDA_INC $$NVCCFLAGS ${QMAKE_FILE_NAME} | sed \"s/^.*: //\" } # Tell Qt that we want add more stuff to the Makefile QMAKE_EXTRA_UNIX_COMPILERS += cuda -
Hi,
you don't need to do the final link with nvcc, but relocatable cuda-code needs a link pass.
I had the same problem some times ago, found a solution in the net and changed it to my needs.
If you want to use this, you have to to the following steps:- save the code below as "cuda.pri"
- remove all cuda related stuff from your pro-file
- put the cuda sources in a node named "CUSOURCES"
- create an empty file file named "cu_devlink.cu" (or any other name) and put it in a node named "CUDEVLINK"
- include the cuda.pri file into your pro-file
- add an evironment var named "CUDA_PATH" to your environmnet (or change the pri-file, whatever you like)
- check all settings in the cuda.pri and make the changes you need before starting qmake!
With this all cu-files will be compiled without linking, and as a prelink step all cuda files will be linked together in one obj file that is finally given, together with your other obj-files, to the standard linker.
Gerd
CUDA_BASE=$$quote($$clean_path($$(CUDA_PATH))) isEmpty( CUDA_BASE ) { error("environment-variable CUDA_PATH is not defined!") } CUDA_BIN_PATH=$$CUDA_BASE/bin CUDA_INC_PATH=$$CUDA_BASE/include CUDA_LIB_PATH=$$CUDA_BASE/lib !contains(QMAKE_HOST.arch, x86_64) { CUDA_LIB_PATH=$$CUDA_LIB_PATH/win32 } else { CUDA_LIB_PATH=$$CUDA_LIB_PATH/x64 } # GPU architecture CUDA_ARCH = sm_52 # Add the necessary libraries # CUDA < 9.0 #CUDA_LIBS=cudart_static nppi nppc # CUDA >= 10.0 CUDA_LIBS=cuda cudart_static nppc curand cudadevrt # setting the CUdaCompiler QMAKE_CUC = $$CUDA_BIN_PATH/nvcc.exe win32 { !exists($$QMAKE_CUC) { warning("can't find cuda compiler($$QMAKE_CUC)") } !exists($$CUDA_INC_PATH/cuda.h) { warning("can't find cuda include ($$CUDA_INC_PATH/cuda.h)") } !exists($$CUDA_LIB_PATH/cuda.lib) { warning("can't find cuda lib ($$CUDA_LIB_PATH/cuda.lib)") } } # Cuda extra-compiler for handling files specified in the CUSOURCES variable { cu.name = Cuda Sourcefiles cu.input = CUSOURCES cu.dependency_type = TYPE_C cu.CONFIG += no_link cu.variable_out = OBJECTS isEmpty(QMAKE_CUC) { win32:QMAKE_CUC = $$CUDA_BIN_PATH/nvcc.exe else:QMAKE_CUC = nvcc } isEmpty(CU_DIR):CU_DIR = . isEmpty(QMAKE_CPP_MOD_CU):QMAKE_CPP_MOD_CU = cu_ isEmpty(QMAKE_EXT_CPP_CU):QMAKE_EXT_CPP_CU = .cu INCLUDEPATH += $$CUDA_INC_PATH CONFIG(debug, debug|release) { QMAKE_CUFLAGS += $$QMAKE_CXXFLAGS $$QMAKE_CXXFLAGS_DEBUG $$QMAKE_CXXFLAGS_RTTI_ON $$QMAKE_CXXFLAGS_WARN_ON $$QMAKE_CXXFLAGS_STL_ON QMAKE_NVVFLAGS += -G } CONFIG(release, debug|release) { QMAKE_CUFLAGS += $$QMAKE_CXXFLAGS $$QMAKE_CXXFLAGS_RELEASE $$QMAKE_CXXFLAGS_RTTI_ON $$QMAKE_CXXFLAGS_WARN_ON $$QMAKE_CXXFLAGS_STL_ON } #since qt5.9 they use /Zc:rvalueCast- /Zc:inline- in the msvc-mkspecs # we have to switch that off!! # else linking __device__ __managed__ vars wont work correctly (in release mode only) QMAKE_CUFLAGS = $$replace(QMAKE_CUFLAGS, -Zc:inline, ) QMAKE_NVVFLAGS += -arch=$$CUDA_ARCH QMAKE_NVVFLAGS += -rdc=true # -keep for holding intermediat files QMAKE_CUEXTRAFLAGS += -Xcompiler $$join(QMAKE_CUFLAGS, ",") !contains(QMAKE_HOST.arch, x86_64) { ## Windows x86 (32bit) specific build here QMAKE_CUEXTRAFLAGS += --machine 32 --debug } else { ## Windows x64 (64bit) specific build here QMAKE_CUEXTRAFLAGS += --machine 64 } QMAKE_CUEXTRAFLAGS += $(DEFINES) $(INCPATH) $$join(QMAKE_COMPILER_DEFINES, " -D", -D) -Xcompiler /Zc:__cplusplus QMAKE_CUEXTRAFLAGS += -Xcudafe "--diag_suppress=field_without_dll_interface" QMAKE_CUEXTRAFLAGS += -Xcudafe "--diag_suppress=code_is_unreachable" #-Xcudafe "--diag_suppress=boolean_controlling_expr_is_constant" CONFIG(debug, debug|release) { CUDA_OBJ_DIR = cuda/debug } else { CUDA_OBJ_DIR = cuda/release } cu.dependency_type = TYPE_C cu.commands = \"$$QMAKE_CUC\" $$QMAKE_NVVFLAGS $$QMAKE_CUEXTRAFLAGS -c -o $${CUDA_OBJ_DIR}/$${QMAKE_CPP_MOD_CU}${QMAKE_FILE_BASE}$${QMAKE_EXT_OBJ} ${QMAKE_FILE_NAME}$$escape_expand(\\n\\t) cu.output = $${CUDA_OBJ_DIR}/$${QMAKE_CPP_MOD_CU}${QMAKE_FILE_BASE}$${QMAKE_EXT_OBJ} #silent:cu.commands = @echo nvcc ${QMAKE_FILE_IN} && $$cu.commands cu.commands = @echo nvcc ${QMAKE_FILE_IN} && $$cu.commands cu.commands = $$replace(cu.commands,-D__cplusplus=199711L,) QMAKE_EXTRA_COMPILERS += cu build_pass|isEmpty(BUILDS):cuclean.depends = compiler_cu_clean else:cuclean.CONFIG += recursive QMAKE_EXTRA_TARGETS += cuclean # another compiler-entry for linking the device-code device_link_target.target = $${CUDA_OBJ_DIR}/$${TARGET}_cu_device_link$${QMAKE_EXT_OBJ} #device_link_target.CONFIG += no_check_exist executable for(var, CUSOURCES) { var = $$basename(var) var = $$replace(var,"\\.cu",$${QMAKE_EXT_OBJ}) CUDEP += $${CUDA_OBJ_DIR}/$${QMAKE_CPP_MOD_CU}$$var } var = $$basename(CUDEVLINK) var = $$replace(var,"\\.cu",$${QMAKE_EXT_OBJ}) CUDEVLINK_OBJ = $${CUDA_OBJ_DIR}/$${QMAKE_CPP_MOD_CU}$$var cu_devlink.output = $${CUDA_OBJ_DIR}/$${TARGET}_cu_device_link$${QMAKE_EXT_OBJ} cu_devlink.input = CUDEVLINK cu_devlink.depends = $$CUDEP cu_devlink.dependency_type = TYPE_C cu_devlink.commands = @echo "link cuda device-code" && \"$$QMAKE_CUC\" -dlink -w -arch=$$CUDA_ARCH $$QMAKE_CUEXTRAFLAGS -o $$device_link_target.target $$CUDEP cu_devlink.name = cuda_devlink cu_devlink.variable_out = OBJECTS cu_devlink.CONFIG = silent QMAKE_EXTRA_COMPILERS += cu_devlink QMAKE_PRE_LINK += $${cu_devlink.commands} } # add the cuda-libraries to the project LIBS += -L$$CUDA_LIB_PATH for(lnam, CUDA_LIBS) { LIBS+=$$join(lnam, " -l", -l) } -
Thank you, Gerd I used your instructions and get this errors (I even try gcc & g++ 7.5.0. Before I used 8.3 with Cuda 10.2. Why it thinks that I use gcc version later than 8? How it was redefined? Ubuntu 18.04 LTS):
<command-line>:0:0: warning: "__GNUC__" redefined <built-in>: note: this is the location of the previous definition <command-line>:0:0: warning: "__GNUC_MINOR__" redefined <built-in>: note: this is the location of the previous definition <command-line>:0:0: warning: "__GNUC_PATCHLEVEL__" redefined <built-in>: note: this is the location of the previous definition In file included from /usr/local/cuda/include/cuda_runtime.h:83:0, from <command-line>:0: /usr/local/cuda/include/crt/host_config.h:138:2: error: #error -- unsupported GNU version! gcc versions later than 8 are not supported! #error -- unsupported GNU version! gcc versions later than 8 are not supported! ^~~~~ <command-line>:0:0: warning: "__GNUC__" redefined <built-in>: note: this is the location of the previous definition <command-line>:0:0: warning: "__GNUC_MINOR__" redefined <built-in>: note: this is the location of the previous definition <command-line>:0:0: warning: "__GNUC_PATCHLEVEL__" redefined <built-in>: note: this is the location of the previous definition In file included from /usr/local/cuda/include/cuda_runtime.h:83:0, from <command-line>:0: /usr/local/cuda/include/crt/host_config.h:138:2: error: #error -- unsupported GNU version! gcc versions later than 8 are not supported! #error -- unsupported GNU version! gcc versions later than 8 are not supported! ^~~~~ g++ -c -pipe -fno-sized-deallocation -g -Wall -Wextra -D_REENTRANT -fPIC -DQT_DEPRECATED_WARNINGS -DQT_QML_DEBUG -DQT_WIDGETS_LIB -DQT_GUI_LIB -DQT_CORE_LIB -I../src -I. -I/usr/local/cuda/include -I../../../Qt/5.14.0/gcc_64/include -I../../../Qt/5.14.0/gcc_64/include/QtWidgets -I../../../Qt/5.14.0/gcc_64/include/QtGui -I../../../Qt/5.14.0/gcc_64/include/QtCore -I. -isystem /usr/include/libdrm -I. -I../../../Qt/5.14.0/gcc_64/mkspecs/linux-g++ -o ../src/Obj/main.o ../src/main.cpp g++ -c -pipe -fno-sized-deallocation -g -Wall -Wextra -D_REENTRANT -fPIC -DQT_DEPRECATED_WARNINGS -DQT_QML_DEBUG -DQT_WIDGETS_LIB -DQT_GUI_LIB -DQT_CORE_LIB -I../src -I. -I/usr/local/cuda/include -I../../../Qt/5.14.0/gcc_64/include -I../../../Qt/5.14.0/gcc_64/include/QtWidgets -I../../../Qt/5.14.0/gcc_64/include/QtGui -I../../../Qt/5.14.0/gcc_64/include/QtCore -I. -isystem /usr/include/libdrm -I. -I../../../Qt/5.14.0/gcc_64/mkspecs/linux-g++ -o ../src/Obj/mainwindow.o ../src/mainwindow.cpp g++ -pipe -fno-sized-deallocation -g -Wall -Wextra -dM -E -o moc_predefs.h ../../../Qt/5.14.0/gcc_64/mkspecs/features/data/dummy.cpp /opt/Qt/5.14.0/gcc_64/bin/moc -DQT_DEPRECATED_WARNINGS -DQT_QML_DEBUG -DQT_WIDGETS_LIB -DQT_GUI_LIB -DQT_CORE_LIB --include /opt/cuda-qt/13-word-correction-fnn/bin/moc_predefs.h -I/opt/Qt/5.14.0/gcc_64/mkspecs/linux-g++ -I/opt/cuda-qt/13-word-correction-fnn/src -I/usr/local/cuda/include -I/opt/Qt/5.14.0/gcc_64/include -I/opt/Qt/5.14.0/gcc_64/include/QtWidgets -I/opt/Qt/5.14.0/gcc_64/include/QtGui -I/opt/Qt/5.14.0/gcc_64/include/QtCore -I. -I/usr/include/c++/9 -I/usr/include/x86_64-linux-gnu/c++/9 -I/usr/include/c++/9/backward -I/usr/lib/gcc/x86_64-linux-gnu/9/include -I/usr/local/include -I/usr/lib/gcc/x86_64-linux-gnu/9/include-fixed -I/usr/include/x86_64-linux-gnu -I/usr/include ../src/mainwindow.h -o moc_mainwindow.cpp <command-line>:0:0: warning: "__GNUC__" redefined <built-in>: note: this is the location of the previous definition <command-line>:0:0: warning: "__GNUC_MINOR__" redefined <built-in>: note: this is the location of the previous definition <command-line>:0:0: warning: "__GNUC_PATCHLEVEL__" redefined <built-in>: note: this is the location of the previous definition In file included from /usr/local/cuda/include/cuda_runtime.h:83:0, from <command-line>:0: /usr/local/cuda/include/crt/host_config.h:138:2: error: #error -- unsupported GNU version! gcc versions later than 8 are not supported! #error -- unsupported GNU version! gcc versions later than 8 are not supported! ^~~~~ Makefile:697: recipe for target 'cuda/debug/cu_word-correction-fnn-thrust.o' failed make: *** [cuda/debug/cu_word-correction-fnn-thrust.o] Error 1 make: *** Waiting for unfinished jobs.... Makefile:755: recipe for target 'cuda/debug/cu_word-correction-fnn-cuda.o' failed make: *** [cuda/debug/cu_word-correction-fnn-cuda.o] Error 1 Makefile:812: recipe for target 'cuda/debug/cu_actfunc.o' failed make: *** [cuda/debug/cu_actfunc.o] Error 1 In file included from ../src/word-correction-fnn-cuda.h:22:0, from ../src/mainwindow.h:24, from ../src/main.cpp:1: /usr/local/cuda/samples/common/inc/helper_cuda.h:223:20: warning: ‘const char* _cudaGetErrorEnum(curandStatus_t)’ defined but not used [-Wunused-function] static const char *_cudaGetErrorEnum(curandStatus_t error) { ^~~~~~~~~~~~~~~~~ In file included from ../src/word-correction-fnn-cuda.h:22:0, from ../src/mainwindow.h:24, from ../src/mainwindow.cpp:1: /usr/local/cuda/samples/common/inc/helper_cuda.h:223:20: warning: ‘const char* _cudaGetErrorEnum(curandStatus_t)’ defined but not used [-Wunused-function] static const char *_cudaGetErrorEnum(curandStatus_t error) { ^~~~~~~~~~~~~~~~~ -
Hi,
those defines find their way into this over the
QMAKE_COMPILER_DEFINES
macro i use in that file. Here the GNUC version should be detected by the toolchain you use, so you are shure you use GCC 8?
Just addmessage("QMAKE_COMPILER_DEFINES = $$QMAKE_COMPILER_DEFINES")in the .pri file to see what this macro contains
I think you can drop this from the file by changing
QMAKE_CUEXTRAFLAGS += $(DEFINES) $(INCPATH) $$join(QMAKE_COMPILER_DEFINES, " -D", -D) -Xcompiler /Zc:__cplusplusto
QMAKE_CUEXTRAFLAGS += $(DEFINES) $(INCPATH) -Xcompiler /Zc:__cplusplusand then try again
-
@Gerd said in QtCreator. CUDA. Make final linking with NVCC:
QMAKE_COMPILER_DEFINES
Now I got this (warnings for all .cu files):
nvcc ../src/word-correction-fnn-thrust.cu nvcc ../src/word-correction-fnn-cuda.cu nvcc ../src/actfunc.cu g++: warning: /tmp/tmpxft_000011d4_00000000-12_cu_word-correction-fnn-thrust_dlink.o: linker input file unused because linking not done g++: warning: /tmp/tmpxft_000011d4_00000000-10_word-correction-fnn-thrust.o: linker input file unused because linking not done g++: warning: /tmp/tmpxft_000011da_00000000-12_cu_actfunc_dlink.o: linker input file unused because linking not done g++: warning: /tmp/tmpxft_000011da_00000000-10_actfunc.o: linker input file unused because linking not done g++: warning: /tmp/tmpxft_000011d9_00000000-12_cu_word-correction-fnn-cuda_dlink.o: linker input file unused because linking not done g++: warning: /tmp/tmpxft_000011d9_00000000-10_word-correction-fnn-cuda.o: linker input file unused because linking not done link cuda device-code Makefile:265: recipe for target '13-word-correction-fnn' failed nvlink fatal : Could not open input file 'cuda/debug/13-word-correction-fnn_cu_device_link.o' make: *** [13-word-correction-fnn] Error 1 -
@Gerd
.pro file#------------------------------------------------- # # Project created by QtCreator 2019-12-10T19:31:27 # #------------------------------------------------- QT += core gui greaterThan(QT_MAJOR_VERSION, 4): QT += widgets TARGET = 13-word-correction-fnn TEMPLATE = app # The following define makes your compiler emit warnings if you use # any feature of Qt which has been marked as deprecated (the exact warnings # depend on your compiler). Please consult the documentation of the # deprecated API in order to know how to port your code away from it. DEFINES += QT_DEPRECATED_WARNINGS # You can also make your code fail to compile if you use deprecated APIs. # In order to do so, uncomment the following line. # You can also select to disable deprecated APIs only up to a certain version of Qt. #DEFINES += QT_DISABLE_DEPRECATED_BEFORE=0x060000 # disables all the APIs deprecated before Qt 6.0.0 CONFIG += c++11 SOURCES += \ main.cpp \ mainwindow.cpp HEADERS += \ mainwindow.h \ word-correction-fnn-cuda.h \ actfunc.h FORMS += \ mainwindow.ui QMAKE_CXXFLAGS += "-fno-sized-deallocation" # Cuda sources CUDA_SOURCES += word-correction-fnn-thrust.cu \ word-correction-fnn-cuda.cu \ actfunc.cu # Project dir and outputs PROJECT_DIR = $$system(pwd) OBJECTS_DIR = $$PROJECT_DIR/Obj DESTDIR = ../bin include($$PWD/cuda.pri)cuda.pri file
CUDA_BASE=$$quote($$clean_path($$(CUDA_PATH))) isEmpty( CUDA_BASE ) { error("environment-variable CUDA_PATH is not defined!") } CUDA_BIN_PATH=$$CUDA_BASE/bin CUDA_INC_PATH=$$CUDA_BASE/include #CUDA_LIB_PATH=$$CUDA_BASE/lib CUDA_LIB_PATH=$$CUDA_BASE/lib64 #!contains(QMAKE_HOST.arch, x86_64) { # CUDA_LIB_PATH=$$CUDA_LIB_PATH/win32 #} else { # CUDA_LIB_PATH=$$CUDA_LIB_PATH/x64 #} # GPU architecture CUDA_ARCH = sm_35 CUDA_VARCH = compute_35 # Add the necessary libraries # CUDA < 9.0 #CUDA_LIBS=cudart_static nppi nppc # CUDA >= 10.0 CUDA_LIBS=cuda cudart_static nppc curand cudadevrt # setting the CUdaCompiler #QMAKE_CUC = $$CUDA_BIN_PATH/nvcc.exe QMAKE_CUC = $$CUDA_BIN_PATH/nvcc win32 { !exists($$QMAKE_CUC) { warning("can't find cuda compiler($$QMAKE_CUC)") } !exists($$CUDA_INC_PATH/cuda.h) { warning("can't find cuda include ($$CUDA_INC_PATH/cuda.h)") } !exists($$CUDA_LIB_PATH/cuda.lib) { warning("can't find cuda lib ($$CUDA_LIB_PATH/cuda.lib)") } } # Cuda extra-compiler for handling files specified in the CUDA_SOURCES variable { cu.name = Cuda Sourcefiles cu.input = CUDA_SOURCES cu.dependency_type = TYPE_C cu.CONFIG += no_link cu.variable_out = OBJECTS isEmpty(QMAKE_CUC) { win32:QMAKE_CUC = $$CUDA_BIN_PATH/nvcc.exe else:QMAKE_CUC = nvcc } isEmpty(CU_DIR):CU_DIR = . isEmpty(QMAKE_CPP_MOD_CU):QMAKE_CPP_MOD_CU = cu_ isEmpty(QMAKE_EXT_CPP_CU):QMAKE_EXT_CPP_CU = .cu INCLUDEPATH += $$CUDA_INC_PATH CONFIG(debug, debug|release) { QMAKE_CUFLAGS += $$QMAKE_CXXFLAGS $$QMAKE_CXXFLAGS_DEBUG $$QMAKE_CXXFLAGS_RTTI_ON $$QMAKE_CXXFLAGS_WARN_ON $$QMAKE_CXXFLAGS_STL_ON QMAKE_NVVFLAGS += -G } CONFIG(release, debug|release) { QMAKE_CUFLAGS += $$QMAKE_CXXFLAGS $$QMAKE_CXXFLAGS_RELEASE $$QMAKE_CXXFLAGS_RTTI_ON $$QMAKE_CXXFLAGS_WARN_ON $$QMAKE_CXXFLAGS_STL_ON } #since qt5.9 they use /Zc:rvalueCast- /Zc:inline- in the msvc-mkspecs # we have to switch that off!! # else linking __device__ __managed__ vars wont work correctly (in release mode only) QMAKE_CUFLAGS = $$replace(QMAKE_CUFLAGS, -Zc:inline, ) QMAKE_NVVFLAGS += -gencode arch=$$CUDA_VARCH,code=$$CUDA_ARCH QMAKE_NVVFLAGS += -rdc=true # -keep for holding intermediat files QMAKE_CUEXTRAFLAGS += -Xcompiler $$join(QMAKE_CUFLAGS, ",") !contains(QMAKE_HOST.arch, x86_64) { ## Windows x86 (32bit) specific build here QMAKE_CUEXTRAFLAGS += --machine 32 --debug } else { ## Windows x64 (64bit) specific build here QMAKE_CUEXTRAFLAGS += --machine 64 } #QMAKE_CUEXTRAFLAGS += $(DEFINES) $(INCPATH) $$join(QMAKE_COMPILER_DEFINES, " -D", -D) -Xcompiler #/Zc:__cplusplus this is for Windows QMAKE_CUEXTRAFLAGS += $(DEFINES) $(INCPATH) -Xcompiler #/Zc:__cplusplus this is for Windows #QMAKE_CUEXTRAFLAGS += -Xcudafe "--diag_suppress=field_without_dll_interface" this is for Windows #QMAKE_CUEXTRAFLAGS += -Xcudafe "--diag_suppress=code_is_unreachable" this is for Windows #-Xcudafe "--diag_suppress=boolean_controlling_expr_is_constant" CONFIG(debug, debug|release) { CUDA_OBJ_DIR = cuda/debug } else { CUDA_OBJ_DIR = cuda/release } cu.dependency_type = TYPE_C cu.commands = \"$$QMAKE_CUC\" $$QMAKE_NVVFLAGS $$QMAKE_CUEXTRAFLAGS -c -o $${CUDA_OBJ_DIR}/$${QMAKE_CPP_MOD_CU}${QMAKE_FILE_BASE}$${QMAKE_EXT_OBJ} ${QMAKE_FILE_NAME}$$escape_expand(\\n\\t) cu.output = $${CUDA_OBJ_DIR}/$${QMAKE_CPP_MOD_CU}${QMAKE_FILE_BASE}$${QMAKE_EXT_OBJ} #silent:cu.commands = @echo nvcc ${QMAKE_FILE_IN} && $$cu.commands cu.commands = @echo nvcc ${QMAKE_FILE_IN} && $$cu.commands cu.commands = $$replace(cu.commands,-D__cplusplus=199711L,) QMAKE_EXTRA_COMPILERS += cu build_pass|isEmpty(BUILDS):cuclean.depends = compiler_cu_clean else:cuclean.CONFIG += recursive QMAKE_EXTRA_TARGETS += cuclean # another compiler-entry for linking the device-code device_link_target.target = $${CUDA_OBJ_DIR}/$${TARGET}_cu_device_link$${QMAKE_EXT_OBJ} #device_link_target.CONFIG += no_check_exist executable for(var, CUDA_SOURCES) { var = $$basename(var) var = $$replace(var,"\\.cu",$${QMAKE_EXT_OBJ}) CUDEP += $${CUDA_OBJ_DIR}/$${QMAKE_CPP_MOD_CU}$$var } var = $$basename(CUDEVLINK) var = $$replace(var,"\\.cu",$${QMAKE_EXT_OBJ}) CUDEVLINK_OBJ = $${CUDA_OBJ_DIR}/$${QMAKE_CPP_MOD_CU}$$var cu_devlink.output = $${CUDA_OBJ_DIR}/$${TARGET}_cu_device_link$${QMAKE_EXT_OBJ} cu_devlink.input = CUDEVLINK cu_devlink.depends = $$CUDEP cu_devlink.dependency_type = TYPE_C cu_devlink.commands = @echo "link cuda device-code" && \"$$QMAKE_CUC\" -dlink -w -gencode arch=$$CUDA_VARCH,code=$$CUDA_ARCH $$QMAKE_CUEXTRAFLAGS -o $$device_link_target.target $$CUDEP cu_devlink.name = cuda_devlink cu_devlink.variable_out = OBJECTS cu_devlink.CONFIG = silent QMAKE_EXTRA_COMPILERS += cu_devlink QMAKE_PRE_LINK += $${cu_devlink.commands} } # add the cuda-libraries to the project LIBS += -L$$CUDA_LIB_PATH for(lnam, CUDA_LIBS) { LIBS+=$$join(lnam, " -l", -l) } -
create an empty file file named "cu_devlink.cu" (or any other name) and put it in a node named "CUDEVLINK"
can't see this in your pro-file...
-
Shouldn't you use CUDA_SOURCES ?
-
change this line
QMAKE_CUEXTRAFLAGS += $(DEFINES) $(INCPATH) -Xcompiler #/Zc:__cplusplus this is for Windowsto
QMAKE_CUEXTRAFLAGS += $(DEFINES) $(INCPATH) -
Hi,
you don't need to do the final link with nvcc, but relocatable cuda-code needs a link pass.
I had the same problem some times ago, found a solution in the net and changed it to my needs.
If you want to use this, you have to to the following steps:- save the code below as "cuda.pri"
- remove all cuda related stuff from your pro-file
- put the cuda sources in a node named "CUSOURCES"
- create an empty file file named "cu_devlink.cu" (or any other name) and put it in a node named "CUDEVLINK"
- include the cuda.pri file into your pro-file
- add an evironment var named "CUDA_PATH" to your environmnet (or change the pri-file, whatever you like)
- check all settings in the cuda.pri and make the changes you need before starting qmake!
With this all cu-files will be compiled without linking, and as a prelink step all cuda files will be linked together in one obj file that is finally given, together with your other obj-files, to the standard linker.
Gerd
CUDA_BASE=$$quote($$clean_path($$(CUDA_PATH))) isEmpty( CUDA_BASE ) { error("environment-variable CUDA_PATH is not defined!") } CUDA_BIN_PATH=$$CUDA_BASE/bin CUDA_INC_PATH=$$CUDA_BASE/include CUDA_LIB_PATH=$$CUDA_BASE/lib !contains(QMAKE_HOST.arch, x86_64) { CUDA_LIB_PATH=$$CUDA_LIB_PATH/win32 } else { CUDA_LIB_PATH=$$CUDA_LIB_PATH/x64 } # GPU architecture CUDA_ARCH = sm_52 # Add the necessary libraries # CUDA < 9.0 #CUDA_LIBS=cudart_static nppi nppc # CUDA >= 10.0 CUDA_LIBS=cuda cudart_static nppc curand cudadevrt # setting the CUdaCompiler QMAKE_CUC = $$CUDA_BIN_PATH/nvcc.exe win32 { !exists($$QMAKE_CUC) { warning("can't find cuda compiler($$QMAKE_CUC)") } !exists($$CUDA_INC_PATH/cuda.h) { warning("can't find cuda include ($$CUDA_INC_PATH/cuda.h)") } !exists($$CUDA_LIB_PATH/cuda.lib) { warning("can't find cuda lib ($$CUDA_LIB_PATH/cuda.lib)") } } # Cuda extra-compiler for handling files specified in the CUSOURCES variable { cu.name = Cuda Sourcefiles cu.input = CUSOURCES cu.dependency_type = TYPE_C cu.CONFIG += no_link cu.variable_out = OBJECTS isEmpty(QMAKE_CUC) { win32:QMAKE_CUC = $$CUDA_BIN_PATH/nvcc.exe else:QMAKE_CUC = nvcc } isEmpty(CU_DIR):CU_DIR = . isEmpty(QMAKE_CPP_MOD_CU):QMAKE_CPP_MOD_CU = cu_ isEmpty(QMAKE_EXT_CPP_CU):QMAKE_EXT_CPP_CU = .cu INCLUDEPATH += $$CUDA_INC_PATH CONFIG(debug, debug|release) { QMAKE_CUFLAGS += $$QMAKE_CXXFLAGS $$QMAKE_CXXFLAGS_DEBUG $$QMAKE_CXXFLAGS_RTTI_ON $$QMAKE_CXXFLAGS_WARN_ON $$QMAKE_CXXFLAGS_STL_ON QMAKE_NVVFLAGS += -G } CONFIG(release, debug|release) { QMAKE_CUFLAGS += $$QMAKE_CXXFLAGS $$QMAKE_CXXFLAGS_RELEASE $$QMAKE_CXXFLAGS_RTTI_ON $$QMAKE_CXXFLAGS_WARN_ON $$QMAKE_CXXFLAGS_STL_ON } #since qt5.9 they use /Zc:rvalueCast- /Zc:inline- in the msvc-mkspecs # we have to switch that off!! # else linking __device__ __managed__ vars wont work correctly (in release mode only) QMAKE_CUFLAGS = $$replace(QMAKE_CUFLAGS, -Zc:inline, ) QMAKE_NVVFLAGS += -arch=$$CUDA_ARCH QMAKE_NVVFLAGS += -rdc=true # -keep for holding intermediat files QMAKE_CUEXTRAFLAGS += -Xcompiler $$join(QMAKE_CUFLAGS, ",") !contains(QMAKE_HOST.arch, x86_64) { ## Windows x86 (32bit) specific build here QMAKE_CUEXTRAFLAGS += --machine 32 --debug } else { ## Windows x64 (64bit) specific build here QMAKE_CUEXTRAFLAGS += --machine 64 } QMAKE_CUEXTRAFLAGS += $(DEFINES) $(INCPATH) $$join(QMAKE_COMPILER_DEFINES, " -D", -D) -Xcompiler /Zc:__cplusplus QMAKE_CUEXTRAFLAGS += -Xcudafe "--diag_suppress=field_without_dll_interface" QMAKE_CUEXTRAFLAGS += -Xcudafe "--diag_suppress=code_is_unreachable" #-Xcudafe "--diag_suppress=boolean_controlling_expr_is_constant" CONFIG(debug, debug|release) { CUDA_OBJ_DIR = cuda/debug } else { CUDA_OBJ_DIR = cuda/release } cu.dependency_type = TYPE_C cu.commands = \"$$QMAKE_CUC\" $$QMAKE_NVVFLAGS $$QMAKE_CUEXTRAFLAGS -c -o $${CUDA_OBJ_DIR}/$${QMAKE_CPP_MOD_CU}${QMAKE_FILE_BASE}$${QMAKE_EXT_OBJ} ${QMAKE_FILE_NAME}$$escape_expand(\\n\\t) cu.output = $${CUDA_OBJ_DIR}/$${QMAKE_CPP_MOD_CU}${QMAKE_FILE_BASE}$${QMAKE_EXT_OBJ} #silent:cu.commands = @echo nvcc ${QMAKE_FILE_IN} && $$cu.commands cu.commands = @echo nvcc ${QMAKE_FILE_IN} && $$cu.commands cu.commands = $$replace(cu.commands,-D__cplusplus=199711L,) QMAKE_EXTRA_COMPILERS += cu build_pass|isEmpty(BUILDS):cuclean.depends = compiler_cu_clean else:cuclean.CONFIG += recursive QMAKE_EXTRA_TARGETS += cuclean # another compiler-entry for linking the device-code device_link_target.target = $${CUDA_OBJ_DIR}/$${TARGET}_cu_device_link$${QMAKE_EXT_OBJ} #device_link_target.CONFIG += no_check_exist executable for(var, CUSOURCES) { var = $$basename(var) var = $$replace(var,"\\.cu",$${QMAKE_EXT_OBJ}) CUDEP += $${CUDA_OBJ_DIR}/$${QMAKE_CPP_MOD_CU}$$var } var = $$basename(CUDEVLINK) var = $$replace(var,"\\.cu",$${QMAKE_EXT_OBJ}) CUDEVLINK_OBJ = $${CUDA_OBJ_DIR}/$${QMAKE_CPP_MOD_CU}$$var cu_devlink.output = $${CUDA_OBJ_DIR}/$${TARGET}_cu_device_link$${QMAKE_EXT_OBJ} cu_devlink.input = CUDEVLINK cu_devlink.depends = $$CUDEP cu_devlink.dependency_type = TYPE_C cu_devlink.commands = @echo "link cuda device-code" && \"$$QMAKE_CUC\" -dlink -w -arch=$$CUDA_ARCH $$QMAKE_CUEXTRAFLAGS -o $$device_link_target.target $$CUDEP cu_devlink.name = cuda_devlink cu_devlink.variable_out = OBJECTS cu_devlink.CONFIG = silent QMAKE_EXTRA_COMPILERS += cu_devlink QMAKE_PRE_LINK += $${cu_devlink.commands} } # add the cuda-libraries to the project LIBS += -L$$CUDA_LIB_PATH for(lnam, CUDA_LIBS) { LIBS+=$$join(lnam, " -l", -l) }@Gerd Thank you for these build instructions! Though I only found this post AFTER figuring out the problem. My CUDA code stopped working after upgrading Qt and some other libraries. I was getting invalid symbol and invalid texture errors at runtime, not linking errors. So it took days to track this down. How did you find this?
#since qt5.9 they use /Zc:rvalueCast- /Zc:inline- in the msvc-mkspecs # we have to switch that off!! # else linking __device__ __managed__ vars wont work correctly (in release mode only) QMAKE_CUFLAGS = $$replace(QMAKE_CUFLAGS, -Zc:inline, )Do you think it makes sense to report this change as a bug in qmake? It's easy to work around and I doubt they'd revert it, but it's a frustrating invisible change that breaks things.
Here is the change that broke it:
https://code.qt.io/cgit/qt/qtbase.git/commit/mkspecs/common/msvc-version.conf?h=5.9&id=255d291efd5ed3e193a6340055c35887f687f0caTo fix the problem and make the build consistent between Qt 5.8 and newer, I added this to my general configuration:
equals(QT_MAJOR_VERSION, 5):lessThan(QT_MINOR_VERSION, 9):*msvc* { QMAKE_CXXFLAGS += -Zc:rvalueCast QMAKE_CXXFLAGS += -Zc:referenceBinding # Need to unset this for CUDA code QMAKE_CXXFLAGS += -Zc:inline }and this to just the library that uses CUDA:
*msvc* { # Fix symbols and textures removed from library # https://forum.qt.io/post/584959 QMAKE_CXXFLAGS -= -Zc:inline QMAKE_CXXFLAGS += -Zc:inline- } -
P Pl45m4 referenced this topic on
