How to build and include stxxl in a Qt project on Windows?



  • Hi everybody

    I'm developing a qt desktop application, in which I need the stxxl library for big containers (http://stxxl.sourceforge.net/). Now unfortunately I'm a complete noob when it comes to programming with libraries that are not part of an installation. And Everything I found so far on the topic was either Linux related or not specific enough. I would be very thankful if somebody could explain to me step by step what I need to do to include the library in my qt desktop application on Windows (I use the Desktop Qt 5.5.1 MinGW 32bit Kit)

    Thank you in advance


  • Lifetime Qt Champion

    Hi and welcome to devnet,

    Do you mean add it as a dependency of your project or build a release package to distribute that includes that dependency ?



  • Hi

    I'd rather include it in the build since I would like to ship my application without the user needing to install any 3rd party software.


  • Lifetime Qt Champion

    Again, that's two different things:

    • Building against the stxxl library
    • Ship it as part of your application package

    Those are two different steps taken at different time of your application development cycle.

    If you are using Qt Creator, here there's an explanation on how to add an external library with it.

    As for deployment, here you have the guide.



  • Ok, yes I understand! Thank you!
    I guess I just really have to find out what steps I need to perform so the stxxl library is recognized as such.
    The QT creator asks for a .lib file which stxxl does not provide me (Or I just can't find it at least). Does anybody know how I can get that?


  • Lifetime Qt Champion

    Did you already build stxxl ?

    If not, there's the INSTALL file that describes the steps



  • Yes I did build it with CMake (The windows version of course) once via command line and once via the gui just to make sure there are no differences. However the installation instructions say that after that I need to run make (or in my case I guess mingw32-make) but neither the command line nor the gui process created a makefile (at least none that's called that way). And like I already said, in both cases I couldn't find a .lib file (If that's even necessary which I think because the QT wizard demands one).Is the make step only necessary on Linux platforms?



  • Ah wait! I finally get it: I have to use visual studio to build the LIBRARY and not the projects! Ok, so I got the .lib file now. But I'm struggling with which path I should choose as the include path.



  • So I think I figured it out: I have to select the include folder in the sources folder of stxxl but now I get a bunch of errors of the following kind:

    • D:\Coding\c++\stxxl-1.4.1\include\stxxl\bits\config.h:28: Error: invalid preprocessing directive #cmakedefine
      #cmakedefine STXXL_VERSION_GIT_REFSPEC "${STXXL_VERSION_GIT_REFSPEC}"
      ^
    • D:\Coding\c++\stxxl-1.4.1\include\stxxl\bits\common\mutex.h:31: Error: #error "Thread implementation not detected."
      #error "Thread implementation not detected."
      ^

  • Lifetime Qt Champion

    Did you call make install and point to that folder/folder set ?



  • No, that would be the case if I were on linux. Since I'm on windows, I had to open the visualstudio project that was created by cmake and run the build process from there. That way I got the .lib file I needed and I went through the steps of the QT creator wizard. I chose external library, then selected the .lib file and the include folder with the necessary header files. When I then copy pasted the sample test-program provided by stxxl, I was able to confirm that the includes worked as expected. But when I try to build it (The QT application) I get the errors mentioned above. Inside the config.h there are a lot of those #cmakedefine macros which don't get recognized by QT; therefore throw an error. And the other error is in connection with all header files that have something to do with multi threading (at least as far as I can tell).

    this is the config.h file:

    // -*- mode: c++ -*-
    /***************************************************************************
     *  include/stxxl/bits/config.h.in
     *
     *  Template file processed by cmake to set all define switches for this build
     *  according to the cmake build options.
     *
     *  Part of the STXXL. See http://stxxl.sourceforge.net
     *
     *  Copyright (C) 2012-2013 Timo Bingmann <tb@panthema.net>
     *
     *  Distributed under the Boost Software License, Version 1.0.
     *  (See accompanying file LICENSE_1_0.txt or copy at
     *  http://www.boost.org/LICENSE_1_0.txt)
     **************************************************************************/
    
    #ifndef STXXL_CONFIG_HEADER
    #define STXXL_CONFIG_HEADER
    
    // the STXXL library version variables
    #define STXXL_VERSION_MAJOR ${STXXL_VERSION_MAJOR}
    #define STXXL_VERSION_MINOR ${STXXL_VERSION_MINOR}
    #define STXXL_VERSION_PATCH ${STXXL_VERSION_PATCH}
    #define STXXL_VERSION_STRING "${STXXL_VERSION_STRING}"
    #define STXXL_VERSION_PHASE "${STXXL_VERSION_PHASE}"
    
    // if this is a git repository, add the refspec and commit sha
    #cmakedefine STXXL_VERSION_GIT_REFSPEC "${STXXL_VERSION_GIT_REFSPEC}"
    #cmakedefine STXXL_VERSION_GIT_SHA1 "${STXXL_VERSION_GIT_SHA1}"
    
    #cmakedefine STXXL_DIRECT_IO_OFF ${STXXL_DIRECT_IO_OFF}
    // default: 0/1 (platform dependent)
    // cmake:   detection of platform and flag
    // effect:  disables use of O_DIRECT flag on unsupported platforms
    
    #cmakedefine STXXL_HAVE_MMAP_FILE ${STXXL_HAVE_MMAP_FILE}
    // default: 0/1 (platform dependent)
    // used in: io/mmap_file.h/cpp
    // effect:  enables/disables memory mapped file implementation
    
    #cmakedefine STXXL_HAVE_LINUXAIO_FILE ${STXXL_HAVE_LINUXAIO_FILE}
    // default: 0/1 (platform dependent)
    // used in: io/linuxaio_file.h/cpp
    // effect:  enables/disables Linux AIO file implementation
    
    #cmakedefine STXXL_POSIX_THREADS ${STXXL_POSIX_THREADS}
    // default: off
    // cmake:   detection of pthreads by cmake
    // effect:  uses POSIX thread and mutexes on linux
    
    #cmakedefine STXXL_STD_THREADS ${STXXL_STD_THREADS}
    // default: off
    // cmake:   detection of <thread> and <mutex>
    // effect:  uses std thread and mutex on windows or (forced on linux)
    
    #cmakedefine STXXL_WINDOWS ${STXXL_WINDOWS}
    // default: off
    // cmake:   detection of ms windows platform (32- or 64-bit)
    // effect:  enables windows-specific api calls (mingw or msvc)
    
    #cmakedefine STXXL_MSVC ${STXXL_MSVC}
    // default: off
    // cmake:   detection of ms visual c++ via CMake (contains version number)
    // effect:  enables msvc-specific headers and macros
    
    #cmakedefine STXXL_HAVE_CXX11_RANGE_FOR_LOOP ${STXXL_HAVE_CXX11_RANGE_FOR_LOOP}
    // default: off
    // run-time: detection C++11 support for "for (auto i : obj) { }"
    // effect:  enables some C++11 construct (currently only allowed in examples)
    
    #cmakedefine STXXL_HAVE_SYNC_ADD_AND_FETCH ${STXXL_HAVE_SYNC_ADD_AND_FETCH}
    // default: off
    // cmake:   detection of __sync_add_and_fetch() intrinsic
    // effect:  enables use of atomics in counting_ptr
    
    #cmakedefine STXXL_PARALLEL_MODE_EXPLICIT ${STXXL_PARALLEL_MODE_EXPLICIT}
    // default: off
    // cmake:   -DUSE_GNU_PARALLEL=ON
    // effect:  explicitly enables use of __gnu_parallel algorithms
    
    #cmakedefine STXXL_BOOST_CONFIG ${STXXL_BOOST_CONFIG}
    #cmakedefine STXXL_BOOST_FILESYSTEM ${STXXL_BOOST_FILESYSTEM}
    #cmakedefine STXXL_BOOST_RANDOM ${STXXL_BOOST_RANDOM}
    #cmakedefine STXXL_BOOST_THREADS ${STXXL_BOOST_THREADS}
    #cmakedefine STXXL_BOOST_TIMESTAMP ${STXXL_BOOST_TIMESTAMP}
    // default: off
    // cmake:   -DUSE_BOOST=ON
    // effect:  enables use of boost libraries in different parts of STXXL.
    
    #if STXXL_BOOST_CONFIG
      #include <boost/config.hpp>
    #endif
    
    #cmakedefine STXXL_STD_RANDOM ${STXXL_STD_RANDOM}
    // default: off
    // cmake:   detection of <random>
    // effect:  uses std random generator on windows or (forced on linux)
    
    #cmakedefine STXXL_HAVE_MALLINFO_PROTO ${STXXL_HAVE_MALLINFO_PROTO}
    // default: off
    // cmake:   detection of mallinfo() function in <malloc.h>
    // effect:  used by stxxl_tool/mallinfo for malloc stats
    
    #cmakedefine STXXL_HAVE_MLOCK_PROTO ${STXXL_HAVE_MLOCK_PROTO}
    // default: off
    // cmake:   detection of mlock() function in <sys/mman.h>
    // effect:  used by stxxl_tool/mlock for locking physical pages
    
    #cmakedefine STXXL_WITH_VALGRIND ${STXXL_WITH_VALGRIND}
    // default: off
    // cmake:   option USE_VALGRIND=ON
    // effect:  run all tests with valgrind and pre-initialize some memory buffers
    
    #endif // !STXXL_CONFIG_HEADER
    
    

    It was originally called config.h.in but when it was like that, I got the error that it could not find a file called config.h so I changed the name (maybe that was a mistake?)

    This is one of the files that throws the "Thread implementation not detected error":

    /***************************************************************************
     *  include/stxxl/bits/common/mutex.h
     *
     *  Part of the STXXL. See http://stxxl.sourceforge.net
     *
     *  Copyright (C) 2002 Roman Dementiev <dementiev@mpi-sb.mpg.de>
     *  Copyright (C) 2008 Andreas Beckmann <beckmann@cs.uni-frankfurt.de>
     *  Copyright (C) 2013 Timo Bingmann <tb@panthema.net>
     *
     *  Distributed under the Boost Software License, Version 1.0.
     *  (See accompanying file LICENSE_1_0.txt or copy at
     *  http://www.boost.org/LICENSE_1_0.txt)
     **************************************************************************/
    
    #ifndef STXXL_COMMON_MUTEX_HEADER
    #define STXXL_COMMON_MUTEX_HEADER
    
    #include <stxxl/bits/config.h>
    #include <stxxl/bits/namespace.h>
    
    #if STXXL_STD_THREADS
     #include <mutex>
    #elif STXXL_BOOST_THREADS
     #include <boost/thread/mutex.hpp>
    #elif STXXL_POSIX_THREADS
     #include <pthread.h>
    
     #include <stxxl/bits/noncopyable.h>
     #include <stxxl/bits/common/error_handling.h>
    #else
     #error "Thread implementation not detected."
    #endif
    
    STXXL_BEGIN_NAMESPACE
    
    #if STXXL_STD_THREADS
    
    typedef std::mutex mutex;
    
    #elif STXXL_BOOST_THREADS
    
    typedef boost::mutex mutex;
    
    #elif STXXL_POSIX_THREADS
    
    class mutex : private noncopyable
    {
        //! mutex handle
        pthread_mutex_t m_mutex;
    
    public:
        //! construct unlocked mutex
        mutex()
        {
            STXXL_CHECK_PTHREAD_CALL(pthread_mutex_init(&m_mutex, NULL));
        }
        //! destroy mutex handle
        ~mutex()
        {
            // try simple delete first
            int res = pthread_mutex_destroy(&m_mutex);
            if (res == 0) return;
    
            // try to lock and unlock mutex
            res = pthread_mutex_trylock(&m_mutex);
    
            if (res == 0 || res == EBUSY) {
                STXXL_CHECK_PTHREAD_CALL(pthread_mutex_unlock(&m_mutex));
            } else {
                STXXL_THROW_ERRNO2(resource_error, "pthread_mutex_trylock() failed", res);
            }
    
            STXXL_CHECK_PTHREAD_CALL(pthread_mutex_destroy(&m_mutex));
        }
        //! lock mutex, may block
        void lock()
        {
            STXXL_CHECK_PTHREAD_CALL(pthread_mutex_lock(&m_mutex));
        }
        //! unlock mutex
        void unlock()
        {
            STXXL_CHECK_PTHREAD_CALL(pthread_mutex_unlock(&m_mutex));
        }
        //! return platform specific handle
        pthread_mutex_t & native_handle()
        {
            return m_mutex;
        }
    };
    
    #endif
    
    #if STXXL_STD_THREADS
    
    typedef std::unique_lock<std::mutex> scoped_mutex_lock;
    
    #elif STXXL_BOOST_THREADS
    
    typedef boost::mutex::scoped_lock scoped_mutex_lock;
    
    #else
    
    //! Aquire a lock that's valid until the end of scope.
    class scoped_mutex_lock
    {
        //! mutex reference
        mutex& m_mutex;
    
        //! marker if already unlocked by this thread (needs no synchronization)
        bool is_locked;
    
    public:
        //! lock mutex
        scoped_mutex_lock(mutex& m)
            : m_mutex(m), is_locked(true)
        {
            m_mutex.lock();
        }
        //! unlock mutex hold when object goes out of scope.
        ~scoped_mutex_lock()
        {
            unlock();
        }
        //! unlock mutex hold prematurely
        void unlock()
        {
            if (is_locked) {
                is_locked = false;
                m_mutex.unlock();
            }
        }
        //! return platform specific handle
        pthread_mutex_t & native_handle()
        {
            return m_mutex.native_handle();
        }
    };
    
    #endif
    
    STXXL_END_NAMESPACE
    
    #endif // !STXXL_COMMON_MUTEX_HEADER
    
    

    And just for reference here is the sample test program I'm trying to compile:

    #include "mainwindow.h"
    #include <QApplication>
    
    /*int main(int argc, char *argv[])
    {
        QApplication a(argc, argv);
        MainWindow w;
        w.show();
    
        return a.exec();
    
    }*/
    
    #include <iostream>
    #include <limits>
    
    #include <stxxl/vector>
    #include <stxxl/random>
    #include <stxxl/sort>
    
    struct my_less_int : std::less<int>
    {
        int min_value() const { return std::numeric_limits<int>::min(); }
        int max_value() const { return std::numeric_limits<int>::max(); }
    };
    
    int main()
    {
        // create vector
        stxxl::VECTOR_GENERATOR<int>::result vector;
    
        // fill vector with random integers
        {
            stxxl::scoped_print_timer
                timer("write random numbers", 100 * 1024 * 1024 * sizeof(int));
    
            stxxl::random_number32 random;
    
            for (size_t i = 0; i < 100 * 1024 * 1024; ++i) {
                vector.push_back(random());
            }
        }
    
        // sort vector using 16 MiB RAM
        {
            stxxl::scoped_print_timer
                timer("sorting random numbers", 100 * 1024 * 1024 * sizeof(int));
    
            stxxl::sort(vector.begin(), vector.end(), my_less_int(), 16 * 1024 * 1024);
        }
    
        // output first and last items:
        std::cout << vector.size() << " items sorted ranging from "
                  << vector.front() << " to " << vector.back() << std::endl;
    
        return 0;
    }
    

    Oh and maybe it's useful to know how my .pro file looks:

    #-------------------------------------------------
    #
    # Project created by QtCreator 2016-02-11T18:45:57
    #
    #-------------------------------------------------
    
    QT       += core gui
    
    greaterThan(QT_MAJOR_VERSION, 4): QT += widgets
    
    TARGET = TestProject
    TEMPLATE = app
    
    
    SOURCES += main.cpp\
            mainwindow.cpp
    
    HEADERS  += mainwindow.h
    
    FORMS    += mainwindow.ui
    
    win32:CONFIG(release, debug|release): LIBS += -L$$PWD/../../stxxl-1.4.1/build/lib/release/ -lstxxl
    else:win32:CONFIG(debug, debug|release): LIBS += -L$$PWD/../../stxxl-1.4.1/build/lib/debug/ -lstxxl
    else:unix: LIBS += -L$$PWD/../../stxxl-1.4.1/build/lib/ -lstxxl
    
    INCLUDEPATH += $$PWD/../../stxxl-1.4.1/include
    DEPENDPATH += $$PWD/../../stxxl-1.4.1/include
    
    

  • Lifetime Qt Champion

    Yes that's a mistake, config.h.in is a template that will be processed by CMake to generate config.h that is likely somewhere in the build folder of stxxl



  • Ah that makes sense! I found the actual configfile and the errors are reduced down to two instances of:

    D:\Coding\c++\stxxl-1.4.1\include\stxxl\bits\common\counting_ptr.h:417: Error: '_InterlockedIncrement' was not declared in this scope
    _InterlockedIncrement(&m_reference_count);
    ^

    They appear in the file "counting_ptr.h" (I only pasted the relevant part):

    public:
        //! Call whenever setting a pointer to the object
        void inc_reference() const
        {
    #if STXXL_MSVC
            _InterlockedIncrement(&m_reference_count);
    #else
            __sync_add_and_fetch(&m_reference_count, +1);
    #endif
        }
    
        //! Call whenever resetting (i.e. overwriting) a pointer to the object.
        //! IMPORTANT: In case of self-assignment, call AFTER inc_reference().
        //! \return if the object has to be deleted (i.e. if it's reference count dropped to zero)
        bool dec_reference() const
        {
    #if STXXL_MSVC
            return (_InterlockedDecrement(&m_reference_count) == 0);
    #else
            return (__sync_add_and_fetch(&m_reference_count, -1) == 0);
    #endif
        }
    

  • Lifetime Qt Champion

    That's when using your MinGW Qt build ?



  • Yes! Desktop QT 5.5.1 MinGW 32bit


  • Lifetime Qt Champion

    You can't mix and match C++ libraries built with different compilers on Windows. Either build stxxl with MinGW or install Qt for the same Visual Studio version you used to build stxxl.

    Note that it's not specific to MinGW vs Visual Studio nor Qt. Visual Studio compilers are not compatible one with the other so you have to build everything with the same compiler.



  • Aww... * sadface * gotta figure out how to build it with mingw then...


  • Lifetime Qt Champion

    Is using MinGW mandatory ? If not and since you already have stxxl built with Visual Studio, why not use it ? You don't have to use the IDE at all.



  • You mean I can use Visual Studios compiler inside the QT creator?
    Because I really don't want to use visual studios IDE if avoidable since IntelliSense is awfully slow!


  • Lifetime Qt Champion

    That's the whole goal of Qt Creator: use it on all supported platform. Qt Creator is an IDE and will use the compiler matching the Qt you installed (provided you installed said compiler)



  • Wicked! It worked * big smile * thank you very much!
    Now what do I need to pay attention to when releasing the application?


  • Lifetime Qt Champion

    You're welcome !

    To provide all dependencies.

    windeployqt will help with setting up a folder structure to distribute your application.

    You should create an installer so it will be easier to distribute your application.



  • Wonderful! At first it threw me a 0xc000007b error but after some cleaning up and recompiling and redeploying it worked like a charm! Thank you again very much! I would have been absolutely lost without your help :D


  • Lifetime Qt Champion

    You're welcome !

    Happy coding :)


Log in to reply
 

Looks like your connection to Qt Forum was lost, please wait while we try to reconnect.