QSslSocket Test Failures (Do I have an Invalid Qt/OpenSSL Configuration?)

crash_test

Hello,

I have encountered 41 QSslSocket test failures after building Qt 5.12.4 and Qt 5.13 from Git with OpenSSL linked.

I believe the possible source of error could be from an invalid network-settings.h, operating system environment or incompatible configuration options. I have provided my test environment details, my build script and the 41 QSslSocket test failures below.

Could anyone help point out a possible configuration or build flaw that would cause these QSslSocket failures?

Thank you for your time and consideration.

TEST ENVIRONMENT:
OS: "Ubuntu 18.04.2 LTS
I kept the network-settings.h file the same, but changed my /etc/hosts file to include the following:

127.1.1.1 qt-test-server.qt-test-net qt-test-server

BUILD SCRIPT:

#!/bin/bash
###############################################################################
# Environment
###############################################################################
HOME=$PWD
THREADS=4
export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:$HOME/openssl-install/lib"
###############################################################################
# OpenSSL Setup
###############################################################################
OPENSSL_QT_CONFIG="-openssl-linked -developer-build -opensource -confirm-license"
OPENSSL_CONFIG="--prefix=$HOME/openssl-install --openssldir=$HOME/openssl-install"
OPENSSL_INCLUDE="-I$HOME/openssl-install/include"
export OPENSSL_LIBS="-L$HOME/openssl-install/lib -lcrypto -lssl"
SSL_VERSION="OpenSSL"
###############################################################################
# User Setup
###############################################################################
echo "Choose Qt Version to build."
options=("5.12.4" "5.13" "Quit")
select opt in "${options[@]}"
do
    case $opt in
        "5.12.4")
            echo "Building Qt 5.12.4"
            QT_VERSION="5.12.4"
            break
            ;;
        "5.13")
            echo "Building Qt 5.13"
            QT_VERSION="5.13"
            break
            ;;
        "Quit")
            exit 0
            ;;
        *) echo "Unknown Option: $REPLY";;
    esac
done
echo "Building Qt $QT_VERSION with $SSL_VERSION"
###############################################################################
# Build SSL Library (OpenSSL)
###############################################################################
if [ ! -d "openssl" ]; then
    git clone git://git.openssl.org/openssl.git
    cd $HOME/openssl
    ./config ${OPENSSL_CONFIG}
fi
make install
###############################################################################
# Configure and build Qt
###############################################################################
cd $HOME
if [ ! -d "qt5" ]; then
    git clone git://code.qt.io/qt/qt5.git
    cd $HOME/qt5
    git checkout $QT_VERSION
    perl init-repository
fi

cd $HOME/qt5
./configure ${OPENSSL_QT_CONFIG} ${OPENSSL_INCLUDE}
if [ $? -ne 0 ]; then
    "FAILED: Building Qt"
fi
make -j$THREADS

TEST FAILURES:

  1 FAIL!  : tst_QSslSocket::simpleConnect(WithoutProxy) Compared values are not the same
  2 FAIL!  : tst_QSslSocket::simpleConnectWithIgnore(WithoutProxy) Compared values are not the same
  3 FAIL!  : tst_QSslSocket::connectToHostEncrypted(WithoutProxy) 'socket->waitForDisconnected()' returned FALSE. ()
  4 FAIL!  : tst_QSslSocket::connectToHostEncryptedWithVerificationPeerName(WithoutProxy) 'socket->waitForDisconnected()' returned FALSE. ()
  5 FAIL!  : tst_QSslSocket::sessionCipher(WithoutProxy) 'socket->waitForConnected(10000)' returned FALSE. (Connection refused)
  6 FAIL!  : tst_QSslSocket::sessionCipher(WithSocks5Proxy) 'socket->waitForConnected(10000)' returned FALSE. (Connection to proxy refused)
  7 FAIL!  : tst_QSslSocket::sessionCipher(WithSocks5ProxyAuth) 'socket->waitForConnected(10000)' returned FALSE. (Connection to proxy refused)
  8 FAIL!  : tst_QSslSocket::sessionCipher(WithHttpProxy) 'socket->waitForConnected(10000)' returned FALSE. (Proxy connection refused)
  9 FAIL!  : tst_QSslSocket::sessionCipher(WithHttpProxyBasicAuth) 'socket->waitForConnected(10000)' returned FALSE. (Proxy connection refused)
 10 FAIL!  : tst_QSslSocket::protocol(WithoutProxy) 'socket->waitForConnected()' returned FALSE. (Connection refused)
 11 FAIL!  : tst_QSslSocket::setSslConfiguration(WithoutProxy:set-root-cert) 'socket->waitForDisconnected()' returned FALSE. (Connection refused)
 12 FAIL!  : tst_QSslSocket::setSslConfiguration(WithoutProxy:secure) 'socket->waitForDisconnected()' returned FALSE. (Connection refused)
 13 FAIL!  : tst_QSslSocket::waitForConnectedEncryptedReadyRead(WithoutProxy) 'socket->waitForConnected(10000)' returned FALSE. (Connection refused)
 14 FAIL!  : tst_QSslSocket::waitForConnectedEncryptedReadyRead(WithSocks5Proxy) 'socket->waitForConnected(10000)' returned FALSE. (Connection to proxy refused)
 15 FAIL!  : tst_QSslSocket::waitForConnectedEncryptedReadyRead(WithSocks5ProxyAuth) 'socket->waitForConnected(10000)' returned FALSE. (Connection to proxy refused)
 16 FAIL!  : tst_QSslSocket::waitForConnectedEncryptedReadyRead(WithHttpProxy) 'socket->waitForConnected(10000)' returned FALSE. (Proxy connection refused)
 17 FAIL!  : tst_QSslSocket::waitForConnectedEncryptedReadyRead(WithHttpProxyBasicAuth) 'socket->waitForConnected(10000)' returned FALSE. (Proxy connection refused)
 18 FAIL!  : tst_QSslSocket::verifyMode(WithoutProxy) Compared lists have different sizes.
 19 FAIL!  : tst_QSslSocket::resetProxy(WithoutProxy) 'socket.waitForConnected(10000)' returned FALSE. (Connection refused)
 20 FAIL!  : tst_QSslSocket::readFromClosedSocket(WithoutProxy) 'socket->bytesAvailable()' returned FALSE. ()
 21 FAIL!  : tst_QSslSocket::resume(WithoutProxy:DoNotIgnoreErrors) Compared values are not the same
 22 FAIL!  : tst_QSslSocket::resume(WithoutProxy:ignoreAllErrors) Compared values are not the same
 23 FAIL!  : tst_QSslSocket::resume(WithoutProxy:ignoreSpecificErrors-Wrong) Compared values are not the same
 24 FAIL!  : tst_QSslSocket::resume(WithoutProxy:ignoreSpecificErrors-Right) Compared values are not the same
 25 FAIL!  : tst_QSslSocket::resume(WithSocks5Proxy:DoNotIgnoreErrors) Compared values are not the same
 26 FAIL!  : tst_QSslSocket::resume(WithSocks5Proxy:ignoreAllErrors) Compared values are not the same
 27 FAIL!  : tst_QSslSocket::resume(WithSocks5Proxy:ignoreSpecificErrors-Wrong) Compared values are not the same
 28 FAIL!  : tst_QSslSocket::resume(WithSocks5Proxy:ignoreSpecificErrors-Right) Compared values are not the same
 29 FAIL!  : tst_QSslSocket::resume(WithSocks5ProxyAuth:DoNotIgnoreErrors) Compared values are not the same
 30 FAIL!  : tst_QSslSocket::resume(WithSocks5ProxyAuth:ignoreAllErrors) Compared values are not the same
 31 FAIL!  : tst_QSslSocket::resume(WithSocks5ProxyAuth:ignoreSpecificErrors-Wrong) Compared values are not the same
 32 FAIL!  : tst_QSslSocket::resume(WithSocks5ProxyAuth:ignoreSpecificErrors-Right) Compared values are not the same
 33 FAIL!  : tst_QSslSocket::resume(WithHttpProxy:DoNotIgnoreErrors) Compared values are not the same
 34 FAIL!  : tst_QSslSocket::resume(WithHttpProxy:ignoreAllErrors) Compared values are not the same
 35 FAIL!  : tst_QSslSocket::resume(WithHttpProxy:ignoreSpecificErrors-Wrong) Compared values are not the same
 36 FAIL!  : tst_QSslSocket::resume(WithHttpProxy:ignoreSpecificErrors-Right) Compared values are not the same
 37 FAIL!  : tst_QSslSocket::resume(WithHttpProxyBasicAuth:DoNotIgnoreErrors) Compared values are not the same
 38 FAIL!  : tst_QSslSocket::resume(WithHttpProxyBasicAuth:ignoreAllErrors) Compared values are not the same
 39 FAIL!  : tst_QSslSocket::resume(WithHttpProxyBasicAuth:ignoreSpecificErrors-Wrong) Compared values are not the same
 40 FAIL!  : tst_QSslSocket::resume(WithHttpProxyBasicAuth:ignoreSpecificErrors-Right) Compared values are not the same
 41 FAIL!  : tst_QSslSocket::forwardReadChannelFinished(WithoutProxy) 'readChannelFinishedSpy.count()' returned FALSE.

SGaist

Hi and welcome to devnet,

Don't these tests use a Docker container as test server ?
Are you using it ?

crash_test

I wasn't sure if docker was necessary for the tests and haven't set anything up for it. It looks like there is a, "qtbase/tests/testserver", directory with a docker-compose.yml file and directories for various servers (apache2, common, danted, etc..).

The "network-settings.h" file has references to those test servers (apache2, common, etc.) if QT_TEST_SERVER is defined.

For Example:

#ifdef QT_TEST_SERVER
        return QString("squid.") % serverDomainName();
#else
        return serverName();
#endif

Maybe the next step would be trying to get QT_TEST_SERVER defined and have the docker containers deployed?

SGaist

Since you have the Docker-compose file, you can directly build the test server. Maybe just launch it before starting the test suit.