QSslSocket Test Failures (Do I have an Invalid Qt/OpenSSL Configuration?)



  • Hello,

    I have encountered 41 QSslSocket test failures after building Qt 5.12.4 and Qt 5.13 from Git with OpenSSL linked.

    I believe the possible source of error could be from an invalid network-settings.h, operating system environment or incompatible configuration options. I have provided my test environment details, my build script and the 41 QSslSocket test failures below.

    Could anyone help point out a possible configuration or build flaw that would cause these QSslSocket failures?

    Thank you for your time and consideration.

    TEST ENVIRONMENT:
    OS: "Ubuntu 18.04.2 LTS
    I kept the network-settings.h file the same, but changed my /etc/hosts file to include the following:

    127.1.1.1 qt-test-server.qt-test-net qt-test-server
    

    BUILD SCRIPT:

    #!/bin/bash
    ###############################################################################
    # Environment
    ###############################################################################
    HOME=$PWD
    THREADS=4
    export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:$HOME/openssl-install/lib"
    ###############################################################################
    # OpenSSL Setup
    ###############################################################################
    OPENSSL_QT_CONFIG="-openssl-linked -developer-build -opensource -confirm-license"
    OPENSSL_CONFIG="--prefix=$HOME/openssl-install --openssldir=$HOME/openssl-install"
    OPENSSL_INCLUDE="-I$HOME/openssl-install/include"
    export OPENSSL_LIBS="-L$HOME/openssl-install/lib -lcrypto -lssl"
    SSL_VERSION="OpenSSL"
    ###############################################################################
    # User Setup
    ###############################################################################
    echo "Choose Qt Version to build."
    options=("5.12.4" "5.13" "Quit")
    select opt in "${options[@]}"
    do
        case $opt in
            "5.12.4")
                echo "Building Qt 5.12.4"
                QT_VERSION="5.12.4"
                break
                ;;
            "5.13")
                echo "Building Qt 5.13"
                QT_VERSION="5.13"
                break
                ;;
            "Quit")
                exit 0
                ;;
            *) echo "Unknown Option: $REPLY";;
        esac
    done
    echo "Building Qt $QT_VERSION with $SSL_VERSION"
    ###############################################################################
    # Build SSL Library (OpenSSL)
    ###############################################################################
    if [ ! -d "openssl" ]; then
        git clone git://git.openssl.org/openssl.git
        cd $HOME/openssl
        ./config ${OPENSSL_CONFIG}
    fi
    make install
    ###############################################################################
    # Configure and build Qt
    ###############################################################################
    cd $HOME
    if [ ! -d "qt5" ]; then
        git clone git://code.qt.io/qt/qt5.git
        cd $HOME/qt5
        git checkout $QT_VERSION
        perl init-repository
    fi
    
    cd $HOME/qt5
    ./configure ${OPENSSL_QT_CONFIG} ${OPENSSL_INCLUDE}
    if [ $? -ne 0 ]; then
        "FAILED: Building Qt"
    fi
    make -j$THREADS
    
    

    TEST FAILURES:

      1 FAIL!  : tst_QSslSocket::simpleConnect(WithoutProxy) Compared values are not the same
      2 FAIL!  : tst_QSslSocket::simpleConnectWithIgnore(WithoutProxy) Compared values are not the same
      3 FAIL!  : tst_QSslSocket::connectToHostEncrypted(WithoutProxy) 'socket->waitForDisconnected()' returned FALSE. ()
      4 FAIL!  : tst_QSslSocket::connectToHostEncryptedWithVerificationPeerName(WithoutProxy) 'socket->waitForDisconnected()' returned FALSE. ()
      5 FAIL!  : tst_QSslSocket::sessionCipher(WithoutProxy) 'socket->waitForConnected(10000)' returned FALSE. (Connection refused)
      6 FAIL!  : tst_QSslSocket::sessionCipher(WithSocks5Proxy) 'socket->waitForConnected(10000)' returned FALSE. (Connection to proxy refused)
      7 FAIL!  : tst_QSslSocket::sessionCipher(WithSocks5ProxyAuth) 'socket->waitForConnected(10000)' returned FALSE. (Connection to proxy refused)
      8 FAIL!  : tst_QSslSocket::sessionCipher(WithHttpProxy) 'socket->waitForConnected(10000)' returned FALSE. (Proxy connection refused)
      9 FAIL!  : tst_QSslSocket::sessionCipher(WithHttpProxyBasicAuth) 'socket->waitForConnected(10000)' returned FALSE. (Proxy connection refused)
     10 FAIL!  : tst_QSslSocket::protocol(WithoutProxy) 'socket->waitForConnected()' returned FALSE. (Connection refused)
     11 FAIL!  : tst_QSslSocket::setSslConfiguration(WithoutProxy:set-root-cert) 'socket->waitForDisconnected()' returned FALSE. (Connection refused)
     12 FAIL!  : tst_QSslSocket::setSslConfiguration(WithoutProxy:secure) 'socket->waitForDisconnected()' returned FALSE. (Connection refused)
     13 FAIL!  : tst_QSslSocket::waitForConnectedEncryptedReadyRead(WithoutProxy) 'socket->waitForConnected(10000)' returned FALSE. (Connection refused)
     14 FAIL!  : tst_QSslSocket::waitForConnectedEncryptedReadyRead(WithSocks5Proxy) 'socket->waitForConnected(10000)' returned FALSE. (Connection to proxy refused)
     15 FAIL!  : tst_QSslSocket::waitForConnectedEncryptedReadyRead(WithSocks5ProxyAuth) 'socket->waitForConnected(10000)' returned FALSE. (Connection to proxy refused)
     16 FAIL!  : tst_QSslSocket::waitForConnectedEncryptedReadyRead(WithHttpProxy) 'socket->waitForConnected(10000)' returned FALSE. (Proxy connection refused)
     17 FAIL!  : tst_QSslSocket::waitForConnectedEncryptedReadyRead(WithHttpProxyBasicAuth) 'socket->waitForConnected(10000)' returned FALSE. (Proxy connection refused)
     18 FAIL!  : tst_QSslSocket::verifyMode(WithoutProxy) Compared lists have different sizes.
     19 FAIL!  : tst_QSslSocket::resetProxy(WithoutProxy) 'socket.waitForConnected(10000)' returned FALSE. (Connection refused)
     20 FAIL!  : tst_QSslSocket::readFromClosedSocket(WithoutProxy) 'socket->bytesAvailable()' returned FALSE. ()
     21 FAIL!  : tst_QSslSocket::resume(WithoutProxy:DoNotIgnoreErrors) Compared values are not the same
     22 FAIL!  : tst_QSslSocket::resume(WithoutProxy:ignoreAllErrors) Compared values are not the same
     23 FAIL!  : tst_QSslSocket::resume(WithoutProxy:ignoreSpecificErrors-Wrong) Compared values are not the same
     24 FAIL!  : tst_QSslSocket::resume(WithoutProxy:ignoreSpecificErrors-Right) Compared values are not the same
     25 FAIL!  : tst_QSslSocket::resume(WithSocks5Proxy:DoNotIgnoreErrors) Compared values are not the same
     26 FAIL!  : tst_QSslSocket::resume(WithSocks5Proxy:ignoreAllErrors) Compared values are not the same
     27 FAIL!  : tst_QSslSocket::resume(WithSocks5Proxy:ignoreSpecificErrors-Wrong) Compared values are not the same
     28 FAIL!  : tst_QSslSocket::resume(WithSocks5Proxy:ignoreSpecificErrors-Right) Compared values are not the same
     29 FAIL!  : tst_QSslSocket::resume(WithSocks5ProxyAuth:DoNotIgnoreErrors) Compared values are not the same
     30 FAIL!  : tst_QSslSocket::resume(WithSocks5ProxyAuth:ignoreAllErrors) Compared values are not the same
     31 FAIL!  : tst_QSslSocket::resume(WithSocks5ProxyAuth:ignoreSpecificErrors-Wrong) Compared values are not the same
     32 FAIL!  : tst_QSslSocket::resume(WithSocks5ProxyAuth:ignoreSpecificErrors-Right) Compared values are not the same
     33 FAIL!  : tst_QSslSocket::resume(WithHttpProxy:DoNotIgnoreErrors) Compared values are not the same
     34 FAIL!  : tst_QSslSocket::resume(WithHttpProxy:ignoreAllErrors) Compared values are not the same
     35 FAIL!  : tst_QSslSocket::resume(WithHttpProxy:ignoreSpecificErrors-Wrong) Compared values are not the same
     36 FAIL!  : tst_QSslSocket::resume(WithHttpProxy:ignoreSpecificErrors-Right) Compared values are not the same
     37 FAIL!  : tst_QSslSocket::resume(WithHttpProxyBasicAuth:DoNotIgnoreErrors) Compared values are not the same
     38 FAIL!  : tst_QSslSocket::resume(WithHttpProxyBasicAuth:ignoreAllErrors) Compared values are not the same
     39 FAIL!  : tst_QSslSocket::resume(WithHttpProxyBasicAuth:ignoreSpecificErrors-Wrong) Compared values are not the same
     40 FAIL!  : tst_QSslSocket::resume(WithHttpProxyBasicAuth:ignoreSpecificErrors-Right) Compared values are not the same
     41 FAIL!  : tst_QSslSocket::forwardReadChannelFinished(WithoutProxy) 'readChannelFinishedSpy.count()' returned FALSE. 
    

  • Lifetime Qt Champion

    Hi and welcome to devnet,

    Don't these tests use a Docker container as test server ?
    Are you using it ?



  • I wasn't sure if docker was necessary for the tests and haven't set anything up for it. It looks like there is a, "qtbase/tests/testserver", directory with a docker-compose.yml file and directories for various servers (apache2, common, danted, etc..).

    The "network-settings.h" file has references to those test servers (apache2, common, etc.) if QT_TEST_SERVER is defined.

    For Example:

    #ifdef QT_TEST_SERVER
            return QString("squid.") % serverDomainName();
    #else
            return serverName();
    #endif
    
    

    Maybe the next step would be trying to get QT_TEST_SERVER defined and have the docker containers deployed?


  • Lifetime Qt Champion

    Since you have the Docker-compose file, you can directly build the test server. Maybe just launch it before starting the test suit.


 

Looks like your connection to Qt Forum was lost, please wait while we try to reconnect.