Building protobuff libs for Android 64 bit (ARM64-v8a) Qt 5.12.4

[Pradeep P N]

Hi @Clem3nt

Please share me the error logs. I will check and update you.
Which proto version you are using ?

[Pradeep P N]

Hi @SGaist
Sure, i will try to make the instructions shorter & post it

Thank you.

[Chapay]

@Clem3nt

f...............k
I spent 2 days :( with this issue

But now can build any android arch from macOS!
Sorry I use ndk 16 and not tested for you version (ndk 19)

#!/bin/sh

ANDROID_NDK_NAME="android-ndk-r16b"
PROTO_VERSION="v3.11.4"

PROTO_DIR=`pwd`/protobuf
ANDROID_RESULT_DIR=`pwd`/libprotobuf/android
NDK_ROOT=`pwd`/ndk

ANDROID_NDK_DIR="$NDK_ROOT/$ANDROID_NDK_NAME"

export ANDROID="/Volumes/Storage/dvlp/android/sdk/sdk"
export ANDROID_HOME="/Volumes/Storage/dvlp/android/sdk/sdk"

# Env variable for Android NDK.
export ANDROID_NDK="$ANDROID_NDK_DIR"
export ANDROID_NDK_HOME="$ANDROID_NDK_DIR"

# Env variable for Android cmake.
export ANDROID_CMAKE_ROOT="$ANDROID_HOME/cmake/3.6.4111459/"
export ANDROID_CMAKE="$ANDROID_CMAKE_ROOT/bin/cmake"

if [ -d "$ANDROID_NDK" ]; then
	echo "EXIST: $ANDROID_NDK"
else
	mkdir -p $NDK_ROOT
	cd $NDK_ROOT	
	# For Mac OS, change the NDK download link accordingly.
	wget "https://dl.google.com/android/repository/$ANDROID_NDK_NAME-darwin-x86_64.zip"
	unzip $ANDROID_NDK_NAME-darwin-x86_64.zip
fi

echo "******************************************************"
echo "start checkout https://github.com/google/protobuf.git"
echo "******************************************************"
if [ -d "$PROTO_DIR" ]; then
	cd $PROTO_DIR
	git fetch origin
	git reset --hard origin/master
	git pull 
	git checkout -b $PROTO_VERSION $PROTO_VERSION
	git submodule update --init --recursive
else
	git clone https://github.com/google/protobuf.git "$PROTO_DIR"	
	cd $PROTO_DIR
	git checkout -b $PROTO_VERSION $PROTO_VERSION
	git submodule update --init --recursive
fi

echo "******************************************************"
echo "end checkout https://github.com/google/protobuf.git"
echo "******************************************************"

function cmakeBuid {
	export ARCH="$1"

	echo "********************" 	
	echo "start $ARCH"
	echo "********************"

	cd "$PROTO_DIR/cmake"
	mkdir -p $ARCH
	cd $ARCH
	
	INSTALL_DIR="$ANDROID_RESULT_DIR/$ARCH"
	mkdir -p $INSTALL

	echo "********************************************************" 	
	echo "start cmake for $ARCH ....."
	echo "********************************************************"

	$ANDROID_CMAKE \
		-Dprotobuf_BUILD_PROTOC_BINARIES=OFF \
	    -Dprotobuf_BUILD_SHARED_LIBS=OFF \
	    -Dprotobuf_BUILD_EXAMPLES=OFF \
	   	-Dprotobuf_BUILD_TESTS=OFF \
	    -DCMAKE_BUILD_TYPE=Release \
	    -DCMAKE_TOOLCHAIN_FILE=$ANDROID_NDK_DIR/build/cmake/android.toolchain.cmake \
	    -DCMAKE_INSTALL_PREFIX=$INSTALL_DIR \
	    -DANDROID_NDK=$ANDROID_NDK \
	    -DANDROID_TOOLCHAIN=clang \
	    -DANDROID_ABI=$ARCH \
	    -DANDROID_NATIVE_API_LEVEL=21 \
	    -DANDROID_STL=c++_shared \
	    -DANDROID_LINKER_FLAGS="-llog -lz -lc++_static" \
	    -DANDROID_CPP_FEATURES="rtti exceptions" \
	    ..

	$ANDROID_CMAKE --build .    

	echo "********************************************************" 	
	echo "end cmake for $ARCH ....."
	echo "********************************************************"	
	
	make install

	echo "***** start: copy results *****"
	if [ -d "$ANDROID_RESULT_DIR/lib/include" ]; then
		rm -rf "$ANDROID_RESULT_DIR/lib/include"
	fi	
	
	if [ -d "$ANDROID_RESULT_DIR/lib/$ARCH" ]; then
		rm -rf "$ANDROID_RESULT_DIR/lib/$ARCH"
	fi		
	
	mkdir -p "$ANDROID_RESULT_DIR/lib/$ARCH"

	cp "$INSTALL_DIR/lib/libprotobuf-lite.a" "$ANDROID_RESULT_DIR/lib/$ARCH"
	cp "$INSTALL_DIR/lib/libprotobuf.a" "$ANDROID_RESULT_DIR/lib/$ARCH"
	cp -r "$INSTALL_DIR/include" "$ANDROID_RESULT_DIR/include"	
	echo "***** end: copy results *****"

	rm -rf "$INSTALL_DIR"

	echo "********************" 	
	echo "end $ARCH"
	echo "********************"
}

cmakeBuid "armeabi-v7a"
cmakeBuid "arm64-v8a"
cmakeBuid "x86"
cmakeBuid "x86_64"

[Chapay]

helpfull link:
https://tech.pic-collage.com/how-to-cross-compile-google-protobuf-lite-for-android-977df3b7f20c

command to get avaliable cmake params (stackowerflow):

cd "../protobufr-repo-dir/cmake"
mkdir build
cd build
cmake ..
cmake -LA | awk '{if(f)print} /-- Cache values/{f=1}'

[Pradeep P N]

@Chapay
I had built it with NDK r20b on Ubuntu 16.04 LTS
And Sorry i was travelling and hence could not respond or update.

So you were success in building it right ?

[Chapay]

@Pradeep-P-N

Yes. I used cmake instead configure script

[Pradeep P N]

Build Instructions on Ubuntu 16.04 LTS

Building Android Tool Chain :

• Few Dependencies :
  sudo apt-get install autoconf automake libtool curl make g++ unzip python gcc git

• Download NDK - Latest Stable Version ( I used NDK r20b, which was latest during i built it )

• You may want to append export NDK=$HOME/<path-to-NDK> to your ~/.bashrc
echo $NDK to verify NDK path from above.

• Creating the Standalone toolchain.
  The NDK folder contains default toolchains, or we can build it - Standalone Toolchains (Obsolete)
  Example

     $NDK/build/tools/make_standalone_toolchain.py \
           --arch arm64 \
           --api 28 \
           --stl=libc++ \
           --install-dir=$HOME/<path-to-build-dir>

• You may want to append export PATH=$HOME/<path-to-build-dir/bin:$PATH to your ~/.bashrc
echo $PATH to verify toolchain path.

Building Google’s Protobuf library :

• Clone protobuf

  mkdir $HOME/Android/NDK/google
 
  cd $HOME/Android/NDK/google

  git clone https://github.com/protocolbuffers/protobuf.git

  cd protobuf

  git submodule update --init --recursive

# Checkout if specific version is needed, example as shown below, else ignore the below command 
 git checkout -b v3.5.2 v3.5.2

• Build a native (x86) version of the protobuf libraries and compiler (protoc)
  sudo apt-get install curl autoconf libtool build-essential g++

In protobuf folder

./autogen.sh

 mkdir x86_build
 cd x86_build

# x86_pb_install folder should be created manually at below given path then run the below cmd

 ../configure --prefix=$HOME/Android/NDK/google/x86_pb_install 

 make install –j4

 cd ..

Libraries will been installed in: $HOME/Android/NDK/google/x86_pb_install/lib
protoc will be in: $HOME/Android/NDK/google/x86_pb_install/bin
include files will be in: $HOME/Android/NDK/google/x86_pb_install/include

• Build the arm64 version of the protobuf libraries:

 mkdir arm64_build

 cd arm64_build

 # arm64_pb_install folder should be created manually at below given path then run the below cmd

  CC=aarch64-linux-android-clang \
  CXX=aarch64-linux-android-clang++ \
  CFLAGS="-fPIE -fPIC" LDFLAGS="-pie -llog" \
     ../configure --host=aarch64-linux-android \
     --prefix=$HOME/Android/NDK/google/arm64_pb_install \
     --with-protoc=$HOME/Android/NDK/google/x86_pb_install/bin/protoc

  make install –j4

  cd ..

Libraries will been installed in: $HOME/Android/NDK/google/arm64_pb_install/lib
protoc will be in: $HOME/Android/NDK/google/arm64_pb_install/bin
include files will be in: $HOME/Android/NDK/google/arm64_pb_install/include

• Compiling Your Protocol Buffers
  protoc -I=$SRC_DIR --cpp_out=$DST_DIR $SRC_DIR/file_name.proto

@SGaist
Hope this helps

@Chapay, @Clem3nt
If possible please try this and update me.

[Chapay]

@Pradeep-P-N Thanks I will try

[mbruel]

@Pradeep-P-N

Thanks for this.
It's working fine.
Little remark from Nov 2020, the script make_standalone_toolchain.py is no longer needed:

#$ $NDK/build/tools/make_standalone_toolchain.py --arch arm64 --api 28 --stl=libc++ --install-dir=/home/bruel/apps/protobuf/build
#WARNING:__main__:make_standalone_toolchain.py is no longer necessary. The
#$NDK/toolchains/llvm/prebuilt/linux-x86_64/bin directory contains target-specific scripts that perform
#the same task. For example, instead of:
#
#    $ python $NDK/build/tools/make_standalone_toolchain.py \
#        --arch arm64 --api 28 --install-dir toolchain
#    $ toolchain/bin/clang++ src.cpp
#
#Instead use:
#
#    $ $NDK/toolchains/llvm/prebuilt/linux-x86_64/bin/aarch64-linux-android28-clang++ src.cpp

So for arm64, you can go directly like this
CC=$NDK/toolchains/llvm/prebuilt/linux-x86_64/bin/aarch64-linux-android$ANDROID_API-clang CXX=$NDK/toolchains/llvm/prebuilt/linux-x86_64/bin/aarch64-linux-android$ANDROID_API-clang++ CFLAGS="-fPIE -fPIC" LDFLAGS="-pie -llog" ../configure --host=aarch64-linux-android --prefix=$APPS_DIR/protobuf/release/arm64 --with-protoc=~/apps/protobuf/release/x86_64/bin/protoc

For those interested, here is my script to build on Linux for x86_64, arm64 and armv7a.

[Pradeep P N]

Hi @mbruel

That's cool,
I had a critical time & had to go with make_standalone_toolchain.py at that time.

Thanks for the updated script on this Building protobuff libs for Android 64 bit (ARM64-v8a) Qt 5.12.4.

[mbruel]

In fact with the method I've posted just above, I can build the Android arm64 library but when I deploy it to a device it's crashing with this:

E AndroidRuntime: FATAL EXCEPTION: qtMainLoopThread
E AndroidRuntime: Process: fr.mbruel.ClementineRemote, PID: 24487
E AndroidRuntime: java.lang.UnsatisfiedLinkError: dlopen failed: cannot locate symbol "_Unwind_Resume" referenced by "/data/app/fr.mbruel.ClementineRemote-eCT926MVQi5HkO3nFNVzEg==/lib/arm64/libprotobuf.so"...
E AndroidRuntime: 	at java.lang.Runtime.load0(Runtime.java:938)
E AndroidRuntime: 	at java.lang.System.load(System.java:1631)
E AndroidRuntime: 	at org.qtproject.qt5.android.QtNative$3.run(QtNative.java:348)
E AndroidRuntime: 	at org.qtproject.qt5.android.QtThread$2.run(QtThread.java:87)
E AndroidRuntime: 	at org.qtproject.qt5.android.QtThread$1.run(QtThread.java:61)
E AndroidRuntime: 	at java.lang.Thread.run(Thread.java:919)

Any idea why?
I'm using protobuf 3.13. I've tried the latest 3.17 but I've the same issue...
I'm building using this command:

NDK=~/android/android-ndk-r21
ANDROID_API=29

CC=$NDK/toolchains/llvm/prebuilt/linux-x86_64/bin/aarch64-linux-android$ANDROID_API-clang CXX=$NDK/toolchains/llvm/prebuilt/linux-x86_64/bin/aarch64-linux-android$ANDROID_API-clang++ CFLAGS="-fPIE -fPIC" LDFLAGS="-pie -llog" ../configure --host=aarch64-linux-android --prefix=~/apps/protobuf/release/arm64 --with-protoc=~/apps/protobuf/release/x86_64/bin/protoc

[mbruel]

@mbruel said in Building protobuff libs for Android 64 bit (ARM64-v8a) Qt 5.12.4:

cannot locate symbol "_Unwind_Resume"

I've updated the NDK to the r21e and the crash is gone! \o/