Building Qt Apps to Wasm

DragonOsman · 14 Oct 2019, 23:42

What do I do after installing Mingw and sed and adding them to my system PATH variable? I did that and tried running this command again:

em++ -std=c++17 -Wall -pedantic -I "C:/Qt/5.13.1/wasm_32/include" -I "C:/Qt/5.13.1/wasm_32/include/QtCore" -I "C:/Qt/5.13.1/wasm_32/include/QtPrintSupport" -I "C:/Qt/5.13.1/wasm_32/include/QtWidgets" -L "C:/Qt/5.13.1/wasm_32/lib" main.cpp notepad.cpp -s WASM=1 -s EXPORT_ALL=1 -o notepad.js

but it still fails with the same message.

Edit: I ran qmake and nmake, but got this error:

shared:ERROR: BINARYEN_TRAP_MODE is not supported by the LLVM wasm backend
NMAKE : fatal error U1077: 'C:\emsdk\upstream\emscripten\em++.BAT' : return code '0x1'
Stop.

Do you guys know how to make it work with the Wasm backend, or should I try asking at the Emscripten GitHub page? The LLVM Wasm backend is the latest, tip-of-the-tree build of vanilla LLVM. It can be installed with the latest-upstream package (emsdk install latest-upstream).

lorn.potter · 15 Oct 2019, 10:20

@DragonOsman said in Building Qt Apps to Wasm:

What do I do after installing Mingw and sed and adding them to my system PATH variable? I did that and tried running this command again:
em++ -std=c++17 -Wall -pedantic -I "C:/Qt/5.13.1/wasm_32/include" -I "C:/Qt/5.13.1/wasm_32/include/QtCore" -I "C:/Qt/5.13.1/wasm_32/include/QtPrintSupport" -I "C:/Qt/5.13.1/wasm_32/include/QtWidgets" -L "C:/Qt/5.13.1/wasm_32/lib" main.cpp notepad.cpp -s WASM=1 -s EXPORT_ALL=1 -o notepad.js
but it still fails with the same message.

Edit: I ran qmake and nmake, but got this error:

It will not work using nmake either, for various reasons. You must use mingw32-make

shared:ERROR: BINARYEN_TRAP_MODE is not supported by the LLVM wasm backend
NMAKE : fatal error U1077: 'C:\emsdk\upstream\emscripten\em++.BAT' : return code '0x1'
Stop.

What version of Qt are you trying to build?

https://wiki.qt.io/Qt_for_WebAssembly
Qt 5.12: 1.38.16
Qt 5.13: 1.38.27 (multithreading: 1.38.30)

Do you guys know how to make it work with the Wasm backend, or should I try asking at the Emscripten GitHub page?  The LLVM Wasm backend is the latest, tip-of-the-tree build of vanilla LLVM.  It can be installed with the `latest-upstream` package (`emsdk install latest-upstream`).

DragonOsman · 15 Oct 2019, 20:17

@lorn-potter said in Building Qt Apps to Wasm:

@DragonOsman said in Building Qt Apps to Wasm:
What do I do after installing Mingw and sed and adding them to my system PATH variable? I did that and tried running this command again:
em++ -std=c++17 -Wall -pedantic -I "C:/Qt/5.13.1/wasm_32/include" -I "C:/Qt/5.13.1/wasm_32/include/QtCore" -I "C:/Qt/5.13.1/wasm_32/include/QtPrintSupport" -I "C:/Qt/5.13.1/wasm_32/include/QtWidgets" -L "C:/Qt/5.13.1/wasm_32/lib" main.cpp notepad.cpp -s WASM=1 -s EXPORT_ALL=1 -o notepad.js
but it still fails with the same message.

Edit: I ran qmake and nmake, but got this error:
It will not work using nmake either, for various reasons. You must use mingw32-make
shared:ERROR: BINARYEN_TRAP_MODE is not supported by the LLVM wasm backend
NMAKE : fatal error U1077: 'C:\emsdk\upstream\emscripten\em++.BAT' : return code '0x1'
Stop.
What version of Qt are you trying to build?

https://wiki.qt.io/Qt_for_WebAssembly
Qt 5.12: 1.38.16
Qt 5.13: 1.38.27 (multithreading: 1.38.30)
Do you guys know how to make it work with the Wasm backend, or should I try asking at the Emscripten GitHub page?  The LLVM Wasm backend is the latest, tip-of-the-tree build of vanilla LLVM.  It can be installed with the `latest-upstream` package (`emsdk install latest-upstream`).

I'm not building Qt. I installed Qt 5.13.1 binaries. I'm trying to build the Notepad App to Wasm (from the Getting Started Guide (I already built it natively in Qt Creator)).

The problem I'm having right now is that the Makefile that the qmake program generated for the Notepad app has BINARYEN_TRAP_MODE set which for the Emscripten Wasm backend is bad. But even after removing that I still get errors.

lorn.potter · wrote on 15 Oct 2019, 20:17

@DragonOsman said in Building Qt Apps to Wasm:

@lorn-potter said in Building Qt Apps to Wasm:
@DragonOsman said in Building Qt Apps to Wasm:
What do I do after installing Mingw and sed and adding them to my system PATH variable? I did that and tried running this command again:
em++ -std=c++17 -Wall -pedantic -I "C:/Qt/5.13.1/wasm_32/include" -I "C:/Qt/5.13.1/wasm_32/include/QtCore" -I "C:/Qt/5.13.1/wasm_32/include/QtPrintSupport" -I "C:/Qt/5.13.1/wasm_32/include/QtWidgets" -L "C:/Qt/5.13.1/wasm_32/lib" main.cpp notepad.cpp -s WASM=1 -s EXPORT_ALL=1 -o notepad.js
but it still fails with the same message.

Edit: I ran qmake and nmake, but got this error:
It will not work using nmake either, for various reasons. You must use mingw32-make
shared:ERROR: BINARYEN_TRAP_MODE is not supported by the LLVM wasm backend
NMAKE : fatal error U1077: 'C:\emsdk\upstream\emscripten\em++.BAT' : return code '0x1'
Stop.
What version of Qt are you trying to build?

https://wiki.qt.io/Qt_for_WebAssembly
Qt 5.12: 1.38.16
Qt 5.13: 1.38.27 (multithreading: 1.38.30)
Do you guys know how to make it work with the Wasm backend, or should I try asking at the Emscripten GitHub page?  The LLVM Wasm backend is the latest, tip-of-the-tree build of vanilla LLVM.  It can be installed with the `latest-upstream` package (`emsdk install latest-upstream`).
I'm not building Qt. I installed Qt 5.13.1 binaries. I'm trying to build the Notepad App to Wasm (from the Getting Started Guide (I already built it natively in Qt Creator)).

The problem I'm having right now is that the Makefile that the qmake program generated for the Notepad app has BINARYEN_TRAP_MODE set which for the Emscripten Wasm backend is bad. But even after removing that I still get errors.

That is because 5.13 needs emscripten 1.38.27 or 1.38.30.
Thats said, if you apply this patch https://codereview.qt-project.org/c/qt/qtbase/+/266814/2/mkspecs/wasm-emscripten/qmake.conf

To the mkspec/wasm-emscripten/qmake.conf file for the Qt you are using, it should compile.
Or you can just remove that line too.

DragonOsman · 15 Oct 2019, 22:42

I installed the latest version of Emscripten. I did need to execute git pull just now though. But it could be that the version I have is too new for Qt (could that be the case?).

By the way. Do you understand what I mean when I say "Emscripten Wasm backend"? That's the backend I'm using. It's the one using LLVM without the Emscripten changes to LLVM. Basically "vanilla" LLVM which you need the latest tip-of-the-tree version of. Later than the current stable version. The regular backend is called the Fastcomp backend. I heard they're going to make the Wasm backend the default backend. And the BINARYEN_TRAP_MODE setting being used is bad for the Wasm backend. (I don't mean to look down on you or anything. I'm really just checking to make sure we're on the same page here.)

I don't understand what the condition is in the Makefile that has to be false for the EMCC_COMMON_LFLAGS += -s \"BINARYEN_TRAP_MODE=\'clamp\'\" line to be added in its else clause. What kind of object files is it looking for?

lorn.potter · wrote on 15 Oct 2019, 22:42

@DragonOsman said in Building Qt Apps to Wasm:

I installed the latest version of Emscripten. I did need to execute git pull just now though. But it could be that the version I have is too new for Qt (could that be the case?).

By the way. Do you understand what I mean when I say "Emscripten Wasm backend"? That's the backend I'm using. It's the one using LLVM without the Emscripten changes to LLVM. Basically "vanilla" LLVM which you need the latest tip-of-the-tree version of. Later than the current stable version. The regular backend is called the Fastcomp backend. I heard they're going to make the Wasm backend the default backend. And the BINARYEN_TRAP_MODE setting being used is bad for the Wasm backend. (I don't mean to look down on you or anything. I'm really just checking to make sure we're on the same page here.)

I don't understand what the condition is in the Makefile that has to be false for the EMCC_COMMON_LFLAGS += -s \"BINARYEN_TRAP_MODE=\'clamp\'\" line to be added in its else clause. What kind of object files is it looking for?

Yes, I am well aware of emscripten/wasm issues.

To use wasm object files, you need to have a Qt built using -device-option=WASM_OBJECT_FILES=1, which will use emscripten upstream to bypass creating the intermediate javascript file when transpiling to wasm. This speeds up linking time.

The Qt released binaries were not built using the new upstream backend.

Apparently we need another check to detect when using the upstream emscripten and not apply the clamp mode.

You can either:

install and use emscripten 1.38.27 without the benefit of wasm object files, and will work correctly with Qt webassembly binaries.
edit the wasm-emscripten/qmake.conf file and simply remove the clamp mode line, and hope things work correctly due to mis-matched emscripten versions

DragonOsman · 16 Oct 2019, 23:57

If I apply the patch, will I not get the JavaScript glue code? Because I do want that along with the .wasm file.

What part of the Makefile should the patch go in? Is there a way to directly install the patch so that whenever I generate a Makefile the patch will be in it?

lorn.potter · wrote on 16 Oct 2019, 23:57

@DragonOsman
You will still get glue code. The clamp mode only tells emscripten what to do in case of a javascript overflow, which the wasm binaries does not have an issue with.

Remove the clamp line from ~/Qt/<version>/wasm_32/mkspecs/wasm-emscripten/qmake.conf
or whereever you installed Qt

DragonOsman · wrote on 17 Oct 2019, 09:53

Is there something I need to add in place of the clamp mode line? Because I do get a whole bunch of errors and exceptions from Python when I run nmake after removing the clamp mode line from the Makefile. Is it okay to assume that it'll work fine if I change the .conf file instead of the Makefile?

Edit: This is my qmake.conf file now:

# qmake configuration for building with emscripten
MAKEFILE_GENERATOR      = UNIX
QMAKE_PLATFORM          = wasm unix

include(../common/gcc-base.conf)
include(../common/clang.conf)

load(device_config)

# Support setting WASM_OBJECT_FILES with -device-option WASM_OBJECT_FILES=1
!isEmpty(WASM_OBJECT_FILES): {
    !equals(WASM_OBJECT_FILES, 1):!equals(WASM_OBJECT_FILES, 0): \
        message(Error: The value for WASM_OBJECT_FILES must be 0 or 1)
    QMAKE_CFLAGS += -s WASM_OBJECT_FILES=$$WASM_OBJECT_FILES
    QMAKE_CXXFLAGS += -s WASM_OBJECT_FILES=$$WASM_OBJECT_FILES
    QMAKE_LFLAGS += -s WASM_OBJECT_FILES=$$WASM_OBJECT_FILES
} else {
    EMCC_COMMON_LFLAGS +=  -s \"BINARYEN_TRAP_MODE=\'clamp\'\"
}

EMTERP_FLAGS = \
    -s EMTERPRETIFY=1  \
    -s EMTERPRETIFY_ASYNC=1 \
    -s \"EMTERPRETIFY_FILE=\'data.binary\'\" \
    -s ASSERTIONS=1 \
    --profiling-funcs

EMCC_COMMON_LFLAGS = \
    -s WASM=1 \
    -s FULL_ES2=1 \
    -s USE_WEBGL2=1 \
    -s NO_EXIT_RUNTIME=0 \
    -s ERROR_ON_UNDEFINED_SYMBOLS=1 \
    --bind

# The -s arguments can also be used with release builds,
# but are here in debug for clarity.
EMCC_COMMON_LFLAGS_DEBUG = \
    $$EMCC_COMMON_LFLAGS \
    -s ASSERTIONS=2 \
    -s DEMANGLE_SUPPORT=1 \
    # -s LIBRARY_DEBUG=1 \ #print out library calls, verbose
    # -s SYSCALL_DEBUG=1 \ #print out sys calls, verbose
    # -s FS_LOG=1 \ #print out filesystem ops, verbose
    # -s SOCKET_DEBUG \ #print out socket,network data transfer
    -s GL_DEBUG=1

# Set up debug/optimization flags
QMAKE_CXXFLAGS_RELEASE -= -O2
QMAKE_CFLAGS_RELEASE   -= -O2
equals(WASM_OBJECT_FILES, 1) {
    QMAKE_LFLAGS_DEBUG     += -g

    QMAKE_CXXFLAGS_RELEASE += -O3
    QMAKE_CFLAGS_RELEASE   += -O3
    QMAKE_LFLAGS_RELEASE   += -O3
    QMAKE_CFLAGS_OPTIMIZE  += -O3
    QMAKE_CFLAGS_OPTIMIZE_FULL += -Oz
} else {
    # Practical debugging setup:
    #    "-g4" preserves function names for stack traces
    #    "-Os" produces reasonably sized binaries
    QMAKE_CFLAGS_DEBUG     -= -g
    QMAKE_CXXFLAGS_DEBUG   -= -g
    QMAKE_CFLAGS_DEBUG     += -Os -g4
    QMAKE_CXXFLAGS_DEBUG   += -Os -g4
    QMAKE_LFLAGS_DEBUG     += -Os -g4

    QMAKE_CXXFLAGS_RELEASE += -O3
    QMAKE_CFLAGS_RELEASE   += -O3
    QMAKE_LFLAGS_RELEASE   += -O3
    QMAKE_CFLAGS_OPTIMIZE  += -O3
    QMAKE_CFLAGS_OPTIMIZE_FULL += -Oz
}


QMAKE_COMPILER         += emscripten

QMAKE_CC                = emcc
QMAKE_CXX               = em++

QMAKE_LINK              = $$QMAKE_CXX
QMAKE_LINK_SHLIB        = $$QMAKE_CXX
QMAKE_LINK_C            = $$QMAKE_CC
QMAKE_LINK_C_SHLIB      = $$QMAKE_CC

QMAKE_LIBS_THREAD       = $$QMAKE_CFLAGS_THREAD

QMAKE_LFLAGS           += $$EMCC_COMMON_LFLAGS
QMAKE_LFLAGS_DEBUG     += $$EMCC_COMMON_LFLAGS_DEBUG

QMAKE_PREFIX_SHLIB      = lib
QMAKE_EXTENSION_SHLIB   = so       # llvm bitcode, linked to js in post_link
QMAKE_PREFIX_STATICLIB  = lib
QMAKE_EXTENSION_STATICLIB = a      # llvm bitcode

QMAKE_AR                = emar cqs
equals(QMAKE_HOST.os, Windows) {
    QMAKE_AR_CMD = \
        "$(file >$(OBJECTS_DIR)/$(TARGET).rsp, $(subst \\,/,$(OBJECTS)))$$escape_expand(\\n\\t)" \
        "$(AR) $(DESTDIR)$(TARGET) @$(OBJECTS_DIR)/$(TARGET).rsp"
}

QMAKE_DISTCLEAN            += *.html *.js *.wasm

load(qt_config)

lorn.potter · wrote on 17 Oct 2019, 10:09

Just place a # in front of that line EMCC_COMMON_LFLAGS += -s "BINARYEN_TRAP_MODE='clamp'" and run qmake again - that will recreate the Makefile

DragonOsman · wrote on 17 Oct 2019, 11:07

I took out that line completely, added the patch, and then ran qmake and nmake again but have the same error as before. A whole bunch of red lines with "unknown warning option" and other stuff in it. And I don't know why the -Werror option is being passed to the compiler backend.

I also have to forcibly close the window using the red button because just giving it the keyboard interrupt isn't enough.

Note: This is one part of the stream of errors:

error: unknown warning option '-Wno-int-in-bool-context'; did you mean '-Wno-gnu-include-next'?
      [-Werror,-Wunknown-warning-option]
error: unknown warning option '-Wno-int-in-bool-context'; did you mean '-Wno-gnu-include-next'?
      [-Werror,-Wunknown-warning-option]
shared:ERROR: 'C:/emsdk/upstream/bin\clang.exe -target wasm32-unknown-emscripten -D__EMSCRIPTEN_major__=1 -D__EMSCRIPTEN_minor__=38 -D__EMSCRIPTEN_tiny__=46 -D_LIBCPP_ABI_VERSION=2 -Dunix -D__unix -D__unix__ -Werror=implicit-function-declaration -Xclang -nostdsysteminc -Xclang -isystemC:\emsdk\upstream\emscripten\system\include\libcxx -Xclang -isystemC:\emsdk\upstream\emscripten\system\lib\libcxxabi\include -Xclang -isystemC:\emsdk\upstream\emscripten\system\include\compat -Xclang -isystemC:\emsdk\upstream\emscripten\system\include -Xclang -isystemC:\emsdk\upstream\emscripten\system\include\libc -Xclang -isystemC:\emsdk\upstream\emscripten\system\lib\libc\musl\arch\emscripten -Xclang -isystemC:\emsdk\upstream\emscripten\system\local\include -Wno-int-in-bool-context -c -Werror -Os -fno-builtin -Wno-return-type -Wno-parentheses -Wno-ignored-attributes -Wno-shift-count-overflow -Wno-shift-negative-value -Wno-dangling-else -Wno-unknown-pragmas -Wno-shift-op-parentheses -Wno-string-plus-int -Wno-logical-op-parentheses -Wno-bitwise-op-parentheses -Wno-visibility -Wno-pointer-sign -Wno-absolute-value -Wno-empty-body -Wno-int-in-bool-context -IC:\emsdk\upstream\emscripten\system\lib\libc\musl\src\internal -IC:\emsdk\upstream\emscripten\system\lib\libc\musl\arch\js -DEMSCRIPTEN C:\emsdk\upstream\emscripten\system\lib\libc\musl\src\stdlib\qsort.c -fno-inline-functions -Xclang -isystemC:\emsdk\upstream\emscripten\system\include\SDL -c -o C:\Users\Osman\AppData\Local\Temp\emscripten_temp_bduibeac\qsort.c.o -mllvm -combiner-global-alias-analysis=false -mllvm -enable-emscripten-sjlj -mllvm -disable-lsr' failed (1)

Actually it did end on its own this time. I'll be able to give you a log file after running nmake 2> build.log now. Here's a link.

lorn.potter · wrote on 18 Oct 2019, 05:21

You need to use mingw32-make and not nmake.

DragonOsman · wrote on 18 Oct 2019, 10:37

The reason for the unknown warning error was a mismatch between the Clang version and the Emscripten version. But after fixing that I have an error from wasm-ld.exe saying that the machine type of library files needs to be wasm32. I don't know what I need to do here so some help would be good. I mentioned this in the Emscripten GitHub as well.

And whether I run nmake or mingw32-make, the result is still pretty much the same: https://gist.github.com/DragonOsman/4b5a04e0546d3ccc21c67cc978510e70

DragonOsman · wrote on 22 Oct 2019, 10:49

There are errors from wasm-ld.exe saying that there are object files that don't have the machine-type wasm32. The object files generated by running nmake definitely hve wasm code in them, even though wasm-ld is flagging them as well. Which is really weird. I see that the object files that come from the Qt library itself are also getting flagged though. Could you guys verify using wasm-dis if those are also wasm object files? Although there's still a problem there because wasm object files shouldn't even have a machine type--only bitcode files should which these aren't. This is really confusing.

lorn.potter · 22 Oct 2019, 21:25

Did you install Emscripten 1.38.27 ? That would be needed for Qt 5.13

DragonOsman · 23 Oct 2019, 05:47

@lorn-potter Do I need to do a complete downgrade? I have version 1.39.0 right now. Could that be what's causing the problem with the wasm-ld error saying that my object files aren't of machine-type wasm32?

lorn.potter · 24 Oct 2019, 16:42

@DragonOsman said in Building Qt Apps to Wasm:

@lorn-potter Do I need to do a complete downgrade? I have version 1.39.0 right now. Could that be what's causing the problem with the wasm-ld error saying that my object files aren't of machine-type wasm32?

Yes, because 1.39.0 switched to wasm object files by default, where as the older version used fastcomp to build.

You can install a different version alongside 1.39.0:
./emsdk install sdk-1.38.27-64bit

and then to switch to the newly installed version:
./emsdk activate --embedded sdk-1.38.27-64bit

When you want ot use a different version just use the activate line with the correct installed version.

DragonOsman · wrote on 23 Oct 2019, 10:06

Is the default backend being switched to the Wasm one the only reason? I don't think that would be enough to give those machine-type errors.

I hope Qt 5 will eventually support version 1.38.48 and newer of Emscripten at some point as well.

DragonOsman · wrote on 24 Oct 2019, 13:00

@lorn-potter Would it be a good idea to get version 1.38.30 instead?

DragonOsman · wrote on 24 Oct 2019, 16:42

@lorn-potter What's the use of the patch if Qt 5 still can't handle the Wasm backend? The reason for the machine type error from wasm-ld is something else: in later versions of Emscripten, it doesn't generate bitcode for Wasm object files. But for some weird reason wasm-ld still gives an error saying that the machine type of the object files isn't wasm32. That's very strange. Please read my conversation with this LLVM developer on this Emscripten GitHub issue I've opened. It would be good if these problems are fixed so that the Wasm backend on the latest version of Emscripten works well with Qt 5.

And right now I can't install the 1.38.27 SDK itself directly using the Emsdk; I have to install Emscripten, Clang and Binaryen separately through Emsdk and since I want to use my own Node, Python and Java anyway, I don't also get those. I had to use CMake to build and install Binaryen on my own though because what Emsdk did failed for me. When I try to install the SDK through Emsdk directly, I get a System Error Access denied when it unzips the Python zip file and gets to trying to add DLLs to the Emsdk Python installation directory.

Edit: I deleted the old files and tried to configure and build again, but now I have this error: shared:ERROR: fastcomp is not compatible with wasm object files:main.obj. I have Emscripten 1.38.27 now with the asm.js backend, so what's going on now? Why is it generating a Wasm object file? Do I still need to add something to the .conf file so that it won't generate Wasm object files when the asm.js backend is being used?

Also, it probably shouldn't generate Wasm object files with the Wasm backend anyway. Wasm object files aren't standard Wasm files.

Edit: Actually, I should be able to make it work if I build Qt for WebAssembly myself. I just need to know what modules are usually built for Qt 5 for Windows. I'll build those same modules. Except if threading support isn't included in there, then I want to include that as well in addition to the rest of it. What module do I need for threading support?