WebAssembly and sound effects

KroMignon

Hi all,

is it possible to play a sound (e.g. MP3 file) with Qt for WebAssembly?

Could someone provide an example or a link which describes how to do this?

lorn.potter

Not using Qt Multimedia at this time.
You could use javascript
https://emscripten.ru/docs/porting/Audio.html

Also this explains how to use it in c++

https://emscripten.org/docs/porting/connecting_cpp_and_javascript/embind.html?highlight=audiocontext#using-val-to-transliterate-javascript-to-c

KroMignon

@lorn-potter Many thanks for your reply, I will try this out, but after reading the documentation I cannot figure out how to play a MP3 file. The link only describe how to play a ton.
Seems to be complicate to do... especially if the MP3 file is stored in resources!

It looks like Qt for WebAssembly not really mature/usable :(
Perhaps I have to look to another way to create WebAssembly application.

terma.aba

@KroMignon said in WebAssembly and sound effects:

It looks like Qt for WebAssembly not really mature/usable :(

The word "not usable" seems a bit unfair, it is clearly documented that multimedia isn't supported yet. WASM in the browser is a platform where you might expect interfacing to HW devices to be very much dependent on browser/emscripten, and difficult to present as a platform independent API just like a desktop. Seeing QT Multimedia not supported didn't surprise me at all.

For a QML UI with QWebsocket behind it, I'm finding WASM more usable and much faster than comparable JavaScript. Not perfect but still very promising compared with alternatives.

KroMignon

@terma-aba said in WebAssembly and sound effects:

The word "not usable" seems a bit unfair,

Yes maybe unfair, but it is my feeling.
I have stopped to try to create software with Qt for WebAssembly, too many bottlenecks in my eyes:

configuring QtCreator to be able to use Qt for WebAssembly take me days of search on internet and many tries until it work (about 3 days!)
compiling a very simple example takes hours!
not possible to play any sound
the generated result is huge!

I love to develop applications with Qt, it is a fantastic framework, but WebAssembly is far to be ready, as far as Qt 6!
Again this is a very personal point of view.

After many hours/days of search and tests, I've decide to go with Blazor to create my WebAssembly modules to replace old Flash modules.

C# and Microsoft are not my first choice, but it simply works!

I have spent as many time with Blazor as I have done with Qt for WebAssembly, but it took me only 3 hours to setup Blazor and create my first application.
With Qt for WebAssembly, I took my 2 days to be able to build my first app. And building the app was done in more than 15 minutes, in comparison building Blazor app takes seconds, with the same PC.

terma.aba

@KroMignon these are fair points.

For me, a big plus is inline with the Qt "concept" which is that mostly my QML source for desktop works directly in WASM and the UI looks the same & behaves the same. That could be luck about the nature of the application.

There is some pain about including qrc images, etc. This took several very boring days. But in comparison our competitors have spent many years of effort to make JavaScript parallel web implementation of their desktops. I certainly feel I am winning with Qt for WASM!

I gave up on Creator and just use make/qmake for WASM. I do the main day to day development and first tests on desktop, because it will always be faster. Then look at WASM platform after a whole batch of changes are working on desktop.

I have to tweak a little to avoid bugs and limitations, performance is less on WASM, but actually I sometimes had to do this kind of tweaking between different desktops anyway (fonts, dpi, )

I looked at Blazer.. Certainly interesting. I don't have the possibility to switch our desktop UI and system behind it over to Blazer, and it would massively increase our team costs if we have to maintain parallel source for same functionality.

terma.aba

What worries me is that Qt for WASM does need attention (you are right), and I'm afraid that the focus will turn 100% to Qt 6 and WASM might be a forgotten platform for a long time.

KroMignon

@terma-aba said in WebAssembly and sound effects:

I looked at Blazer.. Certainly interesting. I don't have the possibility to switch our desktop UI and system behind it over to Blazer, and it would massively increase our team costs if we have to maintain parallel source for same functionality.

Yes Blazor is totally new, also for me!
I have a little experience with C# but this is over 10 years ago!

The advantage I see with Blazor, for my use case, is that debugging works out of the box. And doing Blazor for WebAssembly (and not ASP.Net), I can use it on any server (even with static pages), and it works with almost any device which is able to run a browser with WebAssembly support like FireFox, Chorme, Edge, Vivaldi and so on. Even the OS is not really relevant.

I will continue to develop app with Qt (for Windows/Linux/Android), but for WebAssembly my choice will be Blazor... Until Qt for WASM is going to be usable!

Trigve

@KroMignon
Depending on your application type you could also use Wt (https://www.webtoolkit.eu/wt). Also bear in mind that WASM is "new" technology and isn't mature as others.

I'm too trying to port my application to Qt WASM and I've stumbled on some shortcoming ins QWebsocket. But in my case I already have web client in Wt so I could play with WASM.

As you mentioned Blazor... I was looking into it by I don't like the way it is implemented. From what I've seen it is modifying the DOM and you couldn't write the whole application using C# only. You must use HTML somewhere and incorporate the C# code there. Am I right or I've been mislead by examples?

KroMignon

@Trigve said in WebAssembly and sound effects:

Am I right or I've been mislead by examples?

You right, Blazor is a mix of multiple technologies: HTML5+CSS+JavaScript+C# and optionally ASP.Net.

It is not my favorite combination, of course I would prefer doing it with C++/QML. But Qt WASM is far away to be usable, in my eyes and Blazor is just working.
I still have many to learn about Blazor, but in just 3 days I have built my first Blazor App which perfectly works on my PC (and Android or iOS smartphones) browser.
And that is pretty cool.

lorn.potter

@KroMignon said in WebAssembly and sound effects:

configuring QtCreator to be able to use Qt for WebAssembly take me days of search on internet and many tries until it work (about 3 days!)

Emscripten changed the configure files right after QtCreator came out with experimental WebAssembly support, and it takes QtCreator time to catch up. I would say stick to commandline building. You can stil use QtCreator for editing.

compiling a very simple example takes hours!

This has not much to do with Qt, this is all emscripten.
Not if you use a later version of Qt (5.15) and a more recent emscripten that does not compile to javascript first before then compiling to webassembly.

not possible to play any sound

It is! I posted eample who - you just need to use javascript, which emscripten has several ways to implement - using macros or straight c++.

the generated result is huge!

Ya, well. it is statically linked. Suggested to use compression on the server.

lorn.potter

@terma-aba said in WebAssembly and sound effects:

What worries me is that Qt for WASM does need attention (you are right), and I'm afraid that the focus will turn 100% to Qt 6 and WASM might be a forgotten platform for a long time.

Over the history of Qt for WebAssembly there has been only one or two people working on it- and Qt and it's modules are huge. Other platforms have a nice team of developers to maintain and develop new features for. People seem to think all the bits and pieces magically work once you get a build working. It has been quite a struggle and not like any other platform. Heck the platform itself is experimental and constantly moving, add the different browsers and each has their own implementation and bugs to go with it.

There are quite of bit of differences between normal platforms and WebAssembly, such as the javascript sandbox, threads not really available.

Qt6 Wasm will happen, the biggest block is having to move to building it with cmake.

KroMignon

@lorn-potter said in WebAssembly and sound effects:

It is! I posted eample who - you just need to use javascript, which emscripten has several ways to implement - using macros or straight c++.

I don't want to hurt you, there no raison for me to do it.
I have followed the Qt mailing lists, and I know that Qt embrasse many domains.
The Qt company had to made choices to ensure they will win enough money to pay their developers, which is fine.

But I need a solution now to replace some Flash plugins I have developed for years.
I tried Qt WASM because I know Qt and QML and I know how to work with.
I tried it hard to made it works, and really takes days to be able to build successfully a basic example.
The build itself takes close to 15 minutes.
And I wasn't able to start debugger.

Doing the same with Blazor was very easy, in one day I have installed all needed software, build a demo application (in less than 20 seconds on same machine) and be able to do debugging.

Those are fact, and I don't have so much time to create a new plugin.

My prefered choice would be Qt WASM but I have to be "raisonnable" and go with Blazor.

ericfont

I noticed that QTBUG-69444
webassembly: support QtMultimedia (audio) is marked "Status:CLOSED, Resolution:Done, Fix Version/s: 6.2" and the merge seems to have happened on August 17. I am installing Qt 6.2.0-beta4 now and am going to test it out. I can see the platform target code in https://code.qt.io/cgit/qt/qtmultimedia.git/tree/src/multimedia/platform/wasm?h=6.2.0 which contains the files

qwasmmediadevices.cpp
qwasmmediadevices_p.h
qwasmmediaintegration.cpp
qwasmmediaintegration_p.h

and a subdirectory "audio" containing:

qwasmaudiodevice.cpp
qwasmaudiodevice_p.h
qwasmaudiosink.cpp
qwasmaudiosink_p.h
qwasmaudiosource.cpp
qwasmaudiosource_p.h

Apologize for bumping this thread.

https://wiki.qt.io/Qt_for_WebAssembly still has qtmultimedia listed under "unsupported", though so I suspect there will be issues still.

JoeCFD

Use QtGstreamer + gstreamer pipeline.
examples are here
https://gstreamer.freedesktop.org/data/doc/gstreamer/head/qt-gstreamer/html/examples.html

lorn.potter

@JoeCFD said in WebAssembly and sound effects:

Use QtGstreamer + gstreamer pipeline.

One could, but would have to build gstreamer and all it's friends with emscripten and then statically link that into the wasm binary.

Or, could just use QtMutimedia 6.2.

PeterT

My apologies, what is the actual current status of Qt Multimedia for WebAssembly?

For some reason I couldn't make it work for me. I realize this thread is already quite old and may not get noticed soon, so I created a new topic about my issue here:

https://forum.qt.io/topic/136931/qsoundeffect-for-webassembly

Thanks,
Peter