QSslSocket Test Failures (Do I have an Invalid Qt/OpenSSL Configuration?)



  • Hello,

    I have encountered 41 QSslSocket test failures after building Qt 5.12.4 and Qt 5.13 from Git with OpenSSL linked.

    I believe the possible source of error could be from an invalid network-settings.h, operating system environment or incompatible configuration options. I have provided my test environment details, my build script and the 41 QSslSocket test failures below.

    Could anyone help point out a possible configuration or build flaw that would cause these QSslSocket failures?

    Thank you for your time and consideration.

    TEST ENVIRONMENT:
    OS: "Ubuntu 18.04.2 LTS
    I kept the network-settings.h file the same, but changed my /etc/hosts file to include the following:

    127.1.1.1 qt-test-server.qt-test-net qt-test-server
    

    BUILD SCRIPT:

    #!/bin/bash
    ###############################################################################
    # Environment
    ###############################################################################
    HOME=$PWD
    THREADS=4
    export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:$HOME/openssl-install/lib"
    ###############################################################################
    # OpenSSL Setup
    ###############################################################################
    OPENSSL_QT_CONFIG="-openssl-linked -developer-build -opensource -confirm-license"
    OPENSSL_CONFIG="--prefix=$HOME/openssl-install --openssldir=$HOME/openssl-install"
    OPENSSL_INCLUDE="-I$HOME/openssl-install/include"
    export OPENSSL_LIBS="-L$HOME/openssl-install/lib -lcrypto -lssl"
    SSL_VERSION="OpenSSL"
    ###############################################################################
    # User Setup
    ###############################################################################
    echo "Choose Qt Version to build."
    options=("5.12.4" "5.13" "Quit")
    select opt in "${options[@]}"
    do
        case $opt in
            "5.12.4")
                echo "Building Qt 5.12.4"
                QT_VERSION="5.12.4"
                break
                ;;
            "5.13")
                echo "Building Qt 5.13"
                QT_VERSION="5.13"
                break
                ;;
            "Quit")
                exit 0
                ;;
            *) echo "Unknown Option: $REPLY";;
        esac
    done
    echo "Building Qt $QT_VERSION with $SSL_VERSION"
    ###############################################################################
    # Build SSL Library (OpenSSL)
    ###############################################################################
    if [ ! -d "openssl" ]; then
        git clone git://git.openssl.org/openssl.git
        cd $HOME/openssl
        ./config ${OPENSSL_CONFIG}
    fi
    make install
    ###############################################################################
    # Configure and build Qt
    ###############################################################################
    cd $HOME
    if [ ! -d "qt5" ]; then
        git clone git://code.qt.io/qt/qt5.git
        cd $HOME/qt5
        git checkout $QT_VERSION
        perl init-repository
    fi
    
    cd $HOME/qt5
    ./configure ${OPENSSL_QT_CONFIG} ${OPENSSL_INCLUDE}
    if [ $? -ne 0 ]; then
        "FAILED: Building Qt"
    fi
    make -j$THREADS
    
    

    TEST FAILURES:

      1 FAIL!  : tst_QSslSocket::simpleConnect(WithoutProxy) Compared values are not the same
      2 FAIL!  : tst_QSslSocket::simpleConnectWithIgnore(WithoutProxy) Compared values are not the same
      3 FAIL!  : tst_QSslSocket::connectToHostEncrypted(WithoutProxy) 'socket->waitForDisconnected()' returned FALSE. ()
      4 FAIL!  : tst_QSslSocket::connectToHostEncryptedWithVerificationPeerName(WithoutProxy) 'socket->waitForDisconnected()' returned FALSE. ()
      5 FAIL!  : tst_QSslSocket::sessionCipher(WithoutProxy) 'socket->waitForConnected(10000)' returned FALSE. (Connection refused)
      6 FAIL!  : tst_QSslSocket::sessionCipher(WithSocks5Proxy) 'socket->waitForConnected(10000)' returned FALSE. (Connection to proxy refused)
      7 FAIL!  : tst_QSslSocket::sessionCipher(WithSocks5ProxyAuth) 'socket->waitForConnected(10000)' returned FALSE. (Connection to proxy refused)
      8 FAIL!  : tst_QSslSocket::sessionCipher(WithHttpProxy) 'socket->waitForConnected(10000)' returned FALSE. (Proxy connection refused)
      9 FAIL!  : tst_QSslSocket::sessionCipher(WithHttpProxyBasicAuth) 'socket->waitForConnected(10000)' returned FALSE. (Proxy connection refused)
     10 FAIL!  : tst_QSslSocket::protocol(WithoutProxy) 'socket->waitForConnected()' returned FALSE. (Connection refused)
     11 FAIL!  : tst_QSslSocket::setSslConfiguration(WithoutProxy:set-root-cert) 'socket->waitForDisconnected()' returned FALSE. (Connection refused)
     12 FAIL!  : tst_QSslSocket::setSslConfiguration(WithoutProxy:secure) 'socket->waitForDisconnected()' returned FALSE. (Connection refused)
     13 FAIL!  : tst_QSslSocket::waitForConnectedEncryptedReadyRead(WithoutProxy) 'socket->waitForConnected(10000)' returned FALSE. (Connection refused)
     14 FAIL!  : tst_QSslSocket::waitForConnectedEncryptedReadyRead(WithSocks5Proxy) 'socket->waitForConnected(10000)' returned FALSE. (Connection to proxy refused)
     15 FAIL!  : tst_QSslSocket::waitForConnectedEncryptedReadyRead(WithSocks5ProxyAuth) 'socket->waitForConnected(10000)' returned FALSE. (Connection to proxy refused)
     16 FAIL!  : tst_QSslSocket::waitForConnectedEncryptedReadyRead(WithHttpProxy) 'socket->waitForConnected(10000)' returned FALSE. (Proxy connection refused)
     17 FAIL!  : tst_QSslSocket::waitForConnectedEncryptedReadyRead(WithHttpProxyBasicAuth) 'socket->waitForConnected(10000)' returned FALSE. (Proxy connection refused)
     18 FAIL!  : tst_QSslSocket::verifyMode(WithoutProxy) Compared lists have different sizes.
     19 FAIL!  : tst_QSslSocket::resetProxy(WithoutProxy) 'socket.waitForConnected(10000)' returned FALSE. (Connection refused)
     20 FAIL!  : tst_QSslSocket::readFromClosedSocket(WithoutProxy) 'socket->bytesAvailable()' returned FALSE. ()
     21 FAIL!  : tst_QSslSocket::resume(WithoutProxy:DoNotIgnoreErrors) Compared values are not the same
     22 FAIL!  : tst_QSslSocket::resume(WithoutProxy:ignoreAllErrors) Compared values are not the same
     23 FAIL!  : tst_QSslSocket::resume(WithoutProxy:ignoreSpecificErrors-Wrong) Compared values are not the same
     24 FAIL!  : tst_QSslSocket::resume(WithoutProxy:ignoreSpecificErrors-Right) Compared values are not the same
     25 FAIL!  : tst_QSslSocket::resume(WithSocks5Proxy:DoNotIgnoreErrors) Compared values are not the same
     26 FAIL!  : tst_QSslSocket::resume(WithSocks5Proxy:ignoreAllErrors) Compared values are not the same
     27 FAIL!  : tst_QSslSocket::resume(WithSocks5Proxy:ignoreSpecificErrors-Wrong) Compared values are not the same
     28 FAIL!  : tst_QSslSocket::resume(WithSocks5Proxy:ignoreSpecificErrors-Right) Compared values are not the same
     29 FAIL!  : tst_QSslSocket::resume(WithSocks5ProxyAuth:DoNotIgnoreErrors) Compared values are not the same
     30 FAIL!  : tst_QSslSocket::resume(WithSocks5ProxyAuth:ignoreAllErrors) Compared values are not the same
     31 FAIL!  : tst_QSslSocket::resume(WithSocks5ProxyAuth:ignoreSpecificErrors-Wrong) Compared values are not the same
     32 FAIL!  : tst_QSslSocket::resume(WithSocks5ProxyAuth:ignoreSpecificErrors-Right) Compared values are not the same
     33 FAIL!  : tst_QSslSocket::resume(WithHttpProxy:DoNotIgnoreErrors) Compared values are not the same
     34 FAIL!  : tst_QSslSocket::resume(WithHttpProxy:ignoreAllErrors) Compared values are not the same
     35 FAIL!  : tst_QSslSocket::resume(WithHttpProxy:ignoreSpecificErrors-Wrong) Compared values are not the same
     36 FAIL!  : tst_QSslSocket::resume(WithHttpProxy:ignoreSpecificErrors-Right) Compared values are not the same
     37 FAIL!  : tst_QSslSocket::resume(WithHttpProxyBasicAuth:DoNotIgnoreErrors) Compared values are not the same
     38 FAIL!  : tst_QSslSocket::resume(WithHttpProxyBasicAuth:ignoreAllErrors) Compared values are not the same
     39 FAIL!  : tst_QSslSocket::resume(WithHttpProxyBasicAuth:ignoreSpecificErrors-Wrong) Compared values are not the same
     40 FAIL!  : tst_QSslSocket::resume(WithHttpProxyBasicAuth:ignoreSpecificErrors-Right) Compared values are not the same
     41 FAIL!  : tst_QSslSocket::forwardReadChannelFinished(WithoutProxy) 'readChannelFinishedSpy.count()' returned FALSE. 
    

  • Lifetime Qt Champion

    Hi and welcome to devnet,

    Don't these tests use a Docker container as test server ?
    Are you using it ?



  • I wasn't sure if docker was necessary for the tests and haven't set anything up for it. It looks like there is a, "qtbase/tests/testserver", directory with a docker-compose.yml file and directories for various servers (apache2, common, danted, etc..).

    The "network-settings.h" file has references to those test servers (apache2, common, etc.) if QT_TEST_SERVER is defined.

    For Example:

    #ifdef QT_TEST_SERVER
            return QString("squid.") % serverDomainName();
    #else
            return serverName();
    #endif
    
    

    Maybe the next step would be trying to get QT_TEST_SERVER defined and have the docker containers deployed?


  • Lifetime Qt Champion

    Since you have the Docker-compose file, you can directly build the test server. Maybe just launch it before starting the test suit.


Log in to reply