WebRTC with H264 on Windows fails to "setRemoteDescription"

Makitsune

I'm using Janus Gateway and Gstreamer to stream H264 and Opus to web browsers using WebRTC. However, I'm having issues when I'm using QtWebEngine on Windows.

My current streaming setup works on:

Windows with Chrome 80 and Chromium 80
Linux with Chrome 80 and Chromium 80
Linux with QtWebEngine 5.15 LTS (which uses Chromium 80)
Latest version of Android

But it doesn't work on Windows with QtWebEngine 5.15 LTS.

I built Qt myself and made sure -webengine-proprietary-codecs was added. Playing H264 content works perfectly fine on all platforms.

But when doing H264 with WebRTC I get this error:

DOMException: Failed to execute 'setRemoteDescription' on 'RTCPeerConnection': Failed to set remote offer sdp: Failed to set remote video description send parameters.

Here'ss the sdp it's trying to read:

v=0
o=- 1597689735918520 1 IN IP4 83.50.54.84
s=Mountpoint 1
t=0 0
a=group:BUNDLE audio video
a=msid-semantic: WMS janus
m=audio 9 UDP/TLS/RTP/SAVPF 111
c=IN IP4 83.50.54.84
a=sendonly
a=mid:audio
a=rtcp-mux
a=ice-ufrag:2Rp4
a=ice-pwd:PWHzE7YAFE8wQyj3nmcSO9
a=ice-options:trickle
a=fingerprint:sha-256 2E:AA:54:E7:0B:D4:85:F7:1B:85:D7:FD:59:E4:8B:BB:AE:1E:69:3E:60:C6:EC:A8:17:20:59:8F:90:3C:13:BA
a=setup:actpass
a=rtpmap:111 opus/48000/2
a=extmap:1 urn:ietf:params:rtp-hdrext:sdes:mid
a=msid:janus janusa0
a=ssrc:1408573669 cname:janus
a=ssrc:1408573669 msid:janus janusa0
a=ssrc:1408573669 mslabel:janus
a=ssrc:1408573669 label:janusa0
a=candidate:1 1 udp 2015363327 192.168.1.110 10008 typ host
a=candidate:2 1 udp 1679819007 83.50.54.84 10008 typ srflx raddr 192.168.1.110 rport 10008
a=end-of-candidates
m=video 9 UDP/TLS/RTP/SAVPF 96 97
c=IN IP4 83.50.54.84
a=sendonly
a=mid:video
a=rtcp-mux
a=ice-ufrag:2Rp4
a=ice-pwd:PWHzE7YAFE8wQyj3nmcSO9
a=ice-options:trickle
a=fingerprint:sha-256 2E:AA:54:E7:0B:D4:85:F7:1B:85:D7:FD:59:E4:8B:BB:AE:1E:69:3E:60:C6:EC:A8:17:20:59:8F:90:3C:13:BA
a=setup:actpass
a=rtpmap:96 H264/90000
a=fmtp:96 profile-level-id=42e01f;packetization-mode=1
a=rtcp-fb:96 nack
a=rtcp-fb:96 nack pli
a=rtcp-fb:96 goog-remb
a=extmap:1 urn:ietf:params:rtp-hdrext:sdes:mid
a=rtpmap:97 rtx/90000
a=fmtp:97 apt=96
a=ssrc-group:FID 4187158848 1278254890
a=msid:janus janusv0
a=ssrc:4187158848 cname:janus
a=ssrc:4187158848 msid:janus janusv0
a=ssrc:4187158848 mslabel:janus
a=ssrc:4187158848 label:janusv0
a=ssrc:1278254890 cname:janus
a=ssrc:1278254890 msid:janus janusv0
a=ssrc:1278254890 mslabel:janus
a=ssrc:1278254890 label:janusv0
a=candidate:1 1 udp 2015363327 192.168.1.110 10008 typ host
a=candidate:2 1 udp 1679819007 83.50.54.84 10008 typ srflx raddr 192.168.1.110 rport 10008
a=end-of-candidates

Which contains:

a=rtpmap:96 H264/90000
a=fmtp:96 profile-level-id=42e01f;packetization-mode=1

If I use the latest version of qutebrowser to try to access my site, it works perfectly fine on Linux but not on Windows, with the same error.

However it works on Windows with Chrome 80 and Chromium 80.

I've tried running QtWebEngine with flags --enable-logging --log-level=0 --v=1 but it doesn't log anything other than the error above.

I've setup a server online if it helps:
https://webrtc-h264-test.tivolicloud.com
It may take a couple of seconds before the video shows.

Here's a screenshot of it working on latest qutebrowser on Arch Linux:

And here's a screenshot of it not working on latest qutebrowser on Windows 10:

My Janus Gateway janus.plugin.streaming.jcfg config looks like this:

tivoli-shared-desktop: {
  type = "rtp"
  id = 1
  description = "Tivoli Shared Desktop"
  
  audio = true
  audioport = 5002
  audiopt = 111
  audiortpmap = "opus/48000/2"
  
  video = true
  videoport = 5004
  videopt = 96
  videortpmap = "H264/90000"
  videofmtp = "profile-level-id=42e01f;packetization-mode=1"
}

And my Gstreamer script looks like this:

gst-launch-1.0 -e \
  audiotestsrc volume=0.01 ! \
    opusenc bitrate=48000 audio-type=generic bandwidth=fullband ! \
      rtpopuspay pt=111 ! udpsink host=127.0.0.1 port=5002 \
  videotestsrc ! timeoverlay ! \
    videoscale method=3 ! videorate ! \
    video/x-raw,width=1280,height=720,framerate=30/1 ! \
    videoconvert ! queue ! \
    openh264enc multi-thread=4 complexity=low bitrate=4000000 max-bitrate=4500000 ! \
      rtph264pay config-interval=1 pt=96 ! \
      capssetter caps="application/x-rtp,profile-level-id=(string)42e01f" ! \
      queue ! udpsink host=127.0.0.1 port=5004

Thanks for all the help ❤️
Maki

Makitsune

I just did some tests on macOS 10.15 and my streaming system works with my self-compiled Qt build! It seems to be just an issue with Windows 10.

However, it doesn't work on qutebrowser on macOS. https://html5test.com reports that H264 isn't available. chrome://webrtc-internals reports that "setRemoteDescription" fails.

I checked https://html5test.com on qutebrowser on Windows 10 and it also doesn't support H264. That might explain why "setRemoteDescription" fails on Windows and Mac. On Linux, H264 is available for me and it works.

So now I'm trying to figure out how to debug the issue on my Qt build on Windows 10 since that's the only place it fails. https://html5test.com reports H264 is available on all platforms.

Makitsune

I compiled nanobrowser with my self-compiled Qt build on Windows 10 and it too doesn't work (html5test.com says H264 is supported). I'm trying to look into the QtWebEngine source code now. I'm so confused why this works on Linux and macOS but not Windows.

I've been asking lots of other developers to try https://webrtc-h264-test.tivolicloud.com in their browsers and as long as there is H264 support, it always works.

Makitsune

I found this page https://mozilla.github.io/webrtc-landing/pc_test_no_h264.html
and so I tested something new.

I created a html page:

<script>
    new RTCPeerConnection()
        .createOffer({ offerToReceiveVideo: true })
        .then(offer => {
            const pre = document.createElement("pre");
            pre.innerText = offer.sdp;
            document.body.appendChild(pre);
            console.log(offer.sdp);
        });
</script>

Which returns:

v=0
o=- 116961586097677018 2 IN IP4 127.0.0.1
s=-
t=0 0
a=group:BUNDLE 0
a=msid-semantic: WMS
m=video 9 UDP/TLS/RTP/SAVPF 96 97 98 99 100 101 102 125 127
c=IN IP4 0.0.0.0
a=rtcp:9 IN IP4 0.0.0.0
a=ice-ufrag:XC62
a=ice-pwd:etDmHp8l2kXLqwNqwQ4yCTqU
a=ice-options:trickle
a=fingerprint:sha-256 5B:3D:69:06:82:83:10:E6:0F:E1:C5:26:92:1F:57:49:24:14:61:3F:12:E4:9D:6F:D0:B0:A3:2F:CA:5F:E3:F1
a=setup:actpass
a=mid:0
a=extmap:14 urn:ietf:params:rtp-hdrext:toffset
a=extmap:2 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time
a=extmap:13 urn:3gpp:video-orientation
a=extmap:3 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01
a=extmap:12 http://www.webrtc.org/experiments/rtp-hdrext/playout-delay
a=extmap:11 http://www.webrtc.org/experiments/rtp-hdrext/video-content-type
a=extmap:7 http://www.webrtc.org/experiments/rtp-hdrext/video-timing
a=extmap:8 http://tools.ietf.org/html/draft-ietf-avtext-framemarking-07
a=extmap:9 http://www.webrtc.org/experiments/rtp-hdrext/color-space
a=extmap:4 urn:ietf:params:rtp-hdrext:sdes:mid
a=extmap:5 urn:ietf:params:rtp-hdrext:sdes:rtp-stream-id
a=extmap:6 urn:ietf:params:rtp-hdrext:sdes:repaired-rtp-stream-id
a=recvonly
a=rtcp-mux
a=rtcp-rsize
a=rtpmap:96 VP8/90000
a=rtcp-fb:96 goog-remb
a=rtcp-fb:96 transport-cc
a=rtcp-fb:96 ccm fir
a=rtcp-fb:96 nack
a=rtcp-fb:96 nack pli
a=rtpmap:97 rtx/90000
a=fmtp:97 apt=96
a=rtpmap:98 VP9/90000
a=rtcp-fb:98 goog-remb
a=rtcp-fb:98 transport-cc
a=rtcp-fb:98 ccm fir
a=rtcp-fb:98 nack
a=rtcp-fb:98 nack pli
a=fmtp:98 profile-id=0
a=rtpmap:99 rtx/90000
a=fmtp:99 apt=98
a=rtpmap:100 VP9/90000
a=rtcp-fb:100 goog-remb
a=rtcp-fb:100 transport-cc
a=rtcp-fb:100 ccm fir
a=rtcp-fb:100 nack
a=rtcp-fb:100 nack pli
a=fmtp:100 profile-id=2
a=rtpmap:101 rtx/90000
a=fmtp:101 apt=100
a=rtpmap:102 red/90000
a=rtpmap:125 rtx/90000
a=fmtp:125 apt=102
a=rtpmap:127 ulpfec/90000

And there's no H264 :( On my Linux build of Qt, it does have H264.

I compiled Qt with -webengine-proprietary-codecs on all platforms and H264 is supported on the Windows build, just not with WebRTC.

Makitsune

I think I've found the issue!

In qtwebengine/src/3rdparty/chromium/third_party/webrtc/webrtc.gni

  # Enabling H264 when building with MSVC is currently not supported, see
  # bugs.webrtc.org/9213#c13 for more info.
  rtc_use_h264 =
      proprietary_codecs && !is_android && !is_ios && !(is_win && !is_clang)

FFmpeg doesn't compile well with MSVC and so they dropped support for H264 for RTC on Windows with MSVC.

I'm looking into compiling QtWebEngine with Clang now but I'm not sure how far I'll get. I really hope I can get some help with this.

I found this bug https://bugreports.qt.io/browse/QTBUG-66664 where in the comments, someone explained how H264 with WebRTC on windows stopped working on Qt 5.13 (yay, someone with the same issue as me!)

Makitsune

I was able to solve the issue by applying this patch when building all of Qt on Windows.

Luckily this still works with 5.15 LTS so I'm good to go for a long while.

diff --git a/qtwebengine/src/3rdparty/chromium/third_party/webrtc/modules/video_coding/codecs/h264/h264_color_space.h b/qtwebengine/src/3rdparty/chromium/third_party/webrtc/modules/video_coding/codecs/h264/h264_color_space.h
index 392ccaf5639..aec76efbac3 100644
--- a/qtwebengine/src/3rdparty/chromium/third_party/webrtc/modules/video_coding/codecs/h264/h264_color_space.h
+++ b/qtwebengine/src/3rdparty/chromium/third_party/webrtc/modules/video_coding/codecs/h264/h264_color_space.h
@@ -16,10 +16,6 @@
 // #ifdef unless needed and tested.
 #ifdef WEBRTC_USE_H264
 
-#if defined(WEBRTC_WIN) && !defined(__clang__)
-#error "See: bugs.webrtc.org/9213#c13."
-#endif
-
 #include "api/video/color_space.h"
 
 extern "C" {
diff --git a/qtwebengine/src/3rdparty/chromium/third_party/webrtc/modules/video_coding/codecs/h264/h264_decoder_impl.h b/qtwebengine/src/3rdparty/chromium/third_party/webrtc/modules/video_coding/codecs/h264/h264_decoder_impl.h
index 3c038e6425c..80892de63bf 100644
--- a/qtwebengine/src/3rdparty/chromium/third_party/webrtc/modules/video_coding/codecs/h264/h264_decoder_impl.h
+++ b/qtwebengine/src/3rdparty/chromium/third_party/webrtc/modules/video_coding/codecs/h264/h264_decoder_impl.h
@@ -17,10 +17,6 @@
 // #ifdef unless needed and tested.
 #ifdef WEBRTC_USE_H264
 
-#if defined(WEBRTC_WIN) && !defined(__clang__)
-#error "See: bugs.webrtc.org/9213#c13."
-#endif
-
 #include <memory>
 
 #include "modules/video_coding/codecs/h264/include/h264.h"
diff --git a/qtwebengine/src/3rdparty/chromium/third_party/webrtc/modules/video_coding/codecs/h264/h264_encoder_impl.h b/qtwebengine/src/3rdparty/chromium/third_party/webrtc/modules/video_coding/codecs/h264/h264_encoder_impl.h
index ba996366a35..7fb87eae9d0 100644
--- a/qtwebengine/src/3rdparty/chromium/third_party/webrtc/modules/video_coding/codecs/h264/h264_encoder_impl.h
+++ b/qtwebengine/src/3rdparty/chromium/third_party/webrtc/modules/video_coding/codecs/h264/h264_encoder_impl.h
@@ -17,10 +17,6 @@
 // #ifdef unless needed and tested.
 #ifdef WEBRTC_USE_H264
 
-#if defined(WEBRTC_WIN) && !defined(__clang__)
-#error "See: bugs.webrtc.org/9213#c13."
-#endif
-
 #include <memory>
 #include <vector>
 
diff --git a/qtwebengine/src/3rdparty/chromium/third_party/webrtc/webrtc.gni b/qtwebengine/src/3rdparty/chromium/third_party/webrtc/webrtc.gni
index 56a1b0deca0..c07b63983f8 100644
--- a/qtwebengine/src/3rdparty/chromium/third_party/webrtc/webrtc.gni
+++ b/qtwebengine/src/3rdparty/chromium/third_party/webrtc/webrtc.gni
@@ -150,7 +150,7 @@ declare_args() {
   # Enabling H264 when building with MSVC is currently not supported, see
   # bugs.webrtc.org/9213#c13 for more info.
   rtc_use_h264 =
-      proprietary_codecs && !is_android && !is_ios && !(is_win && !is_clang)
+      proprietary_codecs && !is_android && !is_ios
 
   # By default, use normal platform audio support or dummy audio, but don't
   # use file-based audio playout and record.

Here's a screenshot of it working! :)

TimKieu

@Makitsune THANKS MUCH for your sharing.

I am using Java JxBrowser (Chromium core) with none of issues.
But trying to use Qt WebEngine 5.15.2 on MacOS M1 (Arm64) and get the same Issue!
I will try to compile WebEngine from source.

Do you try to compile Qt WebEngine for MacOS M1 (Arm64) and Windows 10 Arm64?

papinhio

Hello QtWebKit has RTCPeerConnection functionallity?
I am getting this error:

https://ajax.googleapis.com/ajax/libs/jquery/3.6.0/jquery.min.js line 2: jQuery.Deferred exception: Can't find variable: RTCPeerConnection
https://ajax.googleapis.com/ajax/libs/jquery/3.6.0/jquery.min.js line 2: ReferenceError: Can't find variable: RTCPeerConnection