Qt with BSON

Marek

Hi all,

Can someone recommend good solution for BSON with Qt ? I'm targetting Android and iOS platform.
I Need to exchange data with some device which is using BSON.

Best Regards
Marek

mrjj

Hi
I can recommend
https://github.com/nlohmann/json
It also supports BSON but not the full spec.
But should be fast to test if its enough for your device.
Its a wonderful JSON library. (IMHO)

Marek

@mrjj Thanks it looks well documented.
I only need to exchange some data that was previuosly send as ASCII, so I hope I don't need full spec.

Best,
Marek

mrjj

@Marek
Its extremely well documented and was very nice to work with.
Also has special c++ syntax that is very handy to make test with.

json j2 = {
  {"pi", 3.141},
  {"happy", true},
  {"name", "Niels"},
  {"nothing", nullptr},
  {"answer", {
    {"everything", 42}
  }},
  {"list", {1, 0, 2}},
  {"object", {
    {"currency", "USD"},
    {"value", 42.99}
  }}
};

On top of that it was pretty fast too.
Well im not sure what is actually missing but i think it can serialise to bSON most of JSON so hopefully its
fine.

Update:
Btw, Qt JSON can also save as binary. So if you can also alter the sender, you can use that also.

Marek

@mrjj Qt JSON binary format is something else.
According to this blog Q&A https://blog.qt.io/blog/2018/05/31/serialization-in-and-with-qt/ at the end, Qt JSON binary data is not BSON it is some internal format.

Best,
Marek

mrjj

@Marek
Hi
Yes, i know. its just binary.
I would just mention it - just in case. :)

Marek

@mrjj project you recommended works fine, quick integration, nice

Unfortunatelly I have problem, device I'm communicating with does not use JSON, they are using BSON without JSON.
I have compiled libbson for android and I'm including static lib into my project it compiles fine.

Question is: since this is C library, I will have to use wrappers for each function ?
like:

extern "C" {
#include "bson/bson.h"
};

Best,
Marek

mrjj

@Marek
Hi
you mean libbson is a c library ?
In that case, most likely yes, use extern.

Marek

@mrjj
Hi
To clarify my problem, https://github.com/nlohmann/json has some limitations, not all data types are converted from BSON to JSON for instance binary and int64 are not
data_types link

So I need to use libbson and it works on Linux, all cool, I have problem with android.
When I compile I got errors:

/opt/mongo-c-driver/src/libbson/src/bson/bson.c:0: error: undefined reference to 'stderr'
/opt/mongo-c-driver/src/libbson/src/bson/bson.c:0: error: undefined reference to 'stderr'
/opt/mongo-c-driver/src/libbson/src/bson/bson.c:0: error: undefined reference to 'stderr'
/opt/mongo-c-driver/src/libbson/src/bson/bson.c:0: error: undefined reference to 'stderr'
/opt/mongo-c-driver/src/libbson/src/bson/bson-context.c:235: error: undefined reference to 'rand_r'
/opt/mongo-c-driver/src/libbson/src/bson/bson-context.c:235: error: undefined reference to 'rand_r'
/opt/mongo-c-driver/src/libbson/src/bson/bson-context.c:235: error: undefined reference to 'rand_r'
clang++: error: linker command failed with exit code 1 (use -v to see invocation)

They show up when I actually start to use code like:

if ((b = bson_new_from_data (reinterpret_cast<uint8_t*>(data.data()), data.size()))) {
....

They don't show up when I just include header and make variables

extern "C" {
#include "bson/forwarding/bson.h"
}
bson_t *b;
bson_iter_t iter;

I was trying NDK r19c and r20
I have seen someone saying about Unified Headers but it was with NDK 15

I'm compiling libbson with this command:

cmake . -DCMAKE_SYSTEM_NAME=Android -DCMAKE_TOOLCHAIN_FILE=/opt/Android/Ndk/android-ndk-r19c/build/cmake/android.toolchain.cmake -DANDROID_PLATFORM=android-28
-- Check for working C compiler: /opt/Android/Ndk/android-ndk-r19c/toolchains/llvm/prebuilt/linux-x86_64/bin/clang
-- Check for working C compiler: /opt/Android/Ndk/android-ndk-r19c/toolchains/llvm/prebuilt/linux-x86_64/bin/clang -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- No CMAKE_BUILD_TYPE selected, defaulting to RelWithDebInfo
file VERSION_CURRENT contained BUILD_VERSION 1.15.0-20190711+gitf34ff4aa6b
  -- Using bundled libbson
libbson version (from VERSION_CURRENT file): 1.15.0-pre
-- Check if the system is big endian
-- Searching 16 bit integer
-- Looking for sys/types.h
-- Looking for sys/types.h - found
-- Looking for stdint.h
-- Looking for stdint.h - found
-- Looking for stddef.h
-- Looking for stddef.h - found
-- Check size of unsigned short
-- Check size of unsigned short - done
-- Using unsigned short
-- Check if the system is big endian - little endian
-- Looking for snprintf
-- Looking for snprintf - found
-- Looking for reallocf
-- Looking for reallocf - not found
-- Performing Test BSON_HAVE_TIMESPEC
-- Performing Test BSON_HAVE_TIMESPEC - Success
--     struct timespec found
-- Looking for gmtime_r
-- Looking for gmtime_r - found
-- Looking for rand_r
-- Looking for rand_r - found
-- Looking for strings.h
-- Looking for strings.h - found
-- Looking for clock_gettime
-- Looking for clock_gettime - found
-- Looking for strnlen
-- Looking for strnlen - found
-- Looking for stdbool.h
-- Looking for stdbool.h - found
-- Looking for SYS_gettid
-- Looking for SYS_gettid - found
-- Looking for syscall
-- Looking for syscall - found
-- Performing Test HAVE_ATOMIC_32_ADD_AND_FETCH
-- Performing Test HAVE_ATOMIC_32_ADD_AND_FETCH - Success
-- Performing Test HAVE_ATOMIC_64_ADD_AND_FETCH
-- Performing Test HAVE_ATOMIC_64_ADD_AND_FETCH - Success
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Looking for pthread_create
-- Looking for pthread_create - found
-- Found Threads: TRUE  
libmongoc version (from VERSION_CURRENT file): 1.15.0-pre
-- Searching for zlib CMake packages
-- Found ZLIB: /opt/Android/Ndk/android-ndk-r19c/toolchains/llvm/prebuilt/linux-x86_64/sysroot/usr/lib/arm-linux-androideabi/libz.a (found version "1.2.7") 
--   zlib found version "1.2.7"
--   zlib include path "/opt/Android/Ndk/android-ndk-r19c/toolchains/llvm/prebuilt/linux-x86_64/sysroot/usr/include"
--   zlib libraries "/opt/Android/Ndk/android-ndk-r19c/toolchains/llvm/prebuilt/linux-x86_64/sysroot/usr/lib/arm-linux-androideabi/libz.a"
-- Looking for include file unistd.h
-- Looking for include file unistd.h - found
-- Looking for include file stdarg.h
-- Looking for include file stdarg.h - found
-- Could NOT find OpenSSL, try to set the path to OpenSSL root folder in the system variable OPENSSL_ROOT_DIR (missing:  OPENSSL_LIBRARIES OPENSSL_INCLUDE_DIR) 
-- Searching for sasl/sasl.h
--   Not found (specify -DCMAKE_INCLUDE_PATH=/path/to/sasl/include for SASL support)
-- Searching for libsasl2
--   Not found (specify -DCMAKE_LIBRARY_PATH=/path/to/sasl/lib for SASL support)
-- Check size of socklen_t
-- Check size of socklen_t - done
-- Looking for res_nsearch
-- Looking for res_nsearch - not found
-- Looking for res_search
-- Looking for res_search - found
-- Looking for sched_getcpu
-- Looking for sched_getcpu - not found
-- Detected parameters: accept (int, struct sockaddr *, socklen_t *)
-- Searching for compression library header snappy-c.h
--   Not found (specify -DCMAKE_INCLUDE_PATH=/path/to/snappy/include for Snappy compression)
-- No ICU library found, SASLPrep disabled for SCRAM-SHA-256 authentication.
-- If ICU is installed in a non-standard directory, define ICU_ROOT as the ICU installation path.
-- SSL disabled
-- SASL disabled
-- Configuring done
-- Generating done
-- Build files have been written to: /home/franki/apps/qt/mongo-c-driver

Any idea how to solve this problem?

Best,
Marek

Marek

Maybe this is not good idea but I have added:

#include <stdio.h>
#undef stderr
FILE *stderr = NULL;

to bson.c and now I get rid of undefined stderr
When I set in CMakeList.txt in libbson

set (BSON_HAVE_RAND_R 0)

There are different errors:

/opt/mongo-c-driver/src/libbson/src/bson/bson-context.c:303: error: undefined reference to 'srand'
/opt/mongo-c-driver/src/libbson/src/bson/bson-context.c:230: error: undefined reference to 'rand'
/opt/mongo-c-driver/src/libbson/src/bson/bson-context.c:230: error: undefined reference to 'rand'
/opt/mongo-c-driver/src/libbson/src/bson/bson-context.c:230: error: undefined reference to 'rand'

So something is not linking properly for Android.

Marek

I have also commented out rand_r and set current_time as result from rand_r, for now.
I assume I need to report problem to mongoDb.

Best,
Marek

Morbius

I also tried the JSON for Modern C++ . I thought I wanted BSON, but after looking at all the wasted space in the resultant binary file, I tried CBOR, but too wasted too much space. MessagePack turned out to be nicely compact. I had a minor mysterious error if I created it with an initializer list, and then tried to assign a new datum via a string index, e.g.:

j["foo"] = "bar";

It turns out it must be one or the other. It is well-maintained and up to date. I found it easier to use the one big include. I would use an official MessagePack API, but the vanilla C++ one requires Boost++ also, and there are about six other APIs which only use MessagePack output format. This one can use multiple output formats. I might use QJsonObject, but the lack of support for storing binaries as binary was a non-starter; and the much more compact format of MessagePack, without a CPU-expensive re-encoding of binary format was a winner for me. I hope in the future, QJsonObject will support MessagePack and non-reencoded binary fields.