Building protobuff libs for Android 64 bit (ARM64-v8a) Qt 5.12.4

[Pradeep P N]

Hi @SGaist

I dont have these folders in the toolchain which i generated for NDK 20 & NDK 19
I am also confused with the new folder structure in updated NDK code.

[SGaist]

What folder are you talking about ?

[Pradeep P N]

@SGaist
NDK_ROOT/sources/cxx-stl has only llvm-libc++ & llvm-llibc++abi
We need the clang for Qt 5.12.4
And i want to build it for Android API 28 which is only supported from NDK 19

[Pradeep P N]

@SGaist

Below is the script i have for building it.

export PREFIX=$HOME/ARM64_Protobuf/protobuf/
export PATH=$HOME/Android/NDK19/aarch64-linux-android-4.9/bin:$PATH
export SYSROOT=$HOME/Android/NDK19/aarch64-linux-android-4.9/sysroot
export CC="aarch64-linux-android-clang --sysroot $SYSROOT"
export CXX="aarch64-linux-android-clang++ --sysroot $SYSROOT"

../configure \
--prefix=$PREFIX \
--host=aarch64-linux-android \
--with-sysroot="${SYSROOT}" \
--enable-shared \
--enable-cross-compile \
--with-protoc=protoc \
CFLAGS="-march=arm64-v8a -D__ANDROID_API__=28" \
CXXFLAGS="-frtti -fexceptions -march=arm64-v8a -D__ANDROID_API__=28" \
LIBS="-llog -lz -lc++_static"

make -j 2

make install

and i have the error below as

checking whether to enable maintainer-specific portions of Makefiles... yes
checking build system type... x86_64-pc-linux-gnu
checking host system type... aarch64-unknown-linux-android
checking target system type... aarch64-unknown-linux-android
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for aarch64-linux-android-strip... aarch64-linux-android-strip
checking for a thread-safe mkdir -p... /bin/mkdir -p
checking for gawk... no
checking for mawk... mawk
checking whether make sets $(MAKE)... yes
checking whether make supports nested variables... yes
checking whether UID '1000' is supported by ustar format... yes
checking whether GID '1000' is supported by ustar format... yes
checking how to create a ustar tar archive... gnutar
checking whether make supports nested variables... (cached) yes
checking for aarch64-linux-android-gcc... aarch64-linux-android-clang --sysroot /home/Android/NDK19/aarch64-linux-android-4.9/sysroot
checking whether the C compiler works... no
configure: error: in `/home/ARM64_Protobuf/protobuf/build':
configure: error: C compiler cannot create executables
See `config.log' for more details
make: *** No targets specified and no makefile found.  Stop.
make: *** No rule to make target `install'.  Stop.

[SGaist]

What does the config.log tell you ?

[Pradeep P N]

@SGaist Thanks for the support.

I was able to build the library & protoc for Android ARM 64 v8a using Ubuntu 16.04 LTS.
I am moving it to Solved. If some one comes up with the same issue/problem I can support on that.

[SGaist]

Glad you succeeded. It would be nice to have a write down of what you did to be able to achieve the build.

[Clem3nt]

Hello @Pradeep-P-N,

I'm trying to build protobuf for arm64 and I have the same error. How did you solve the problem?

Thank you

[Pradeep P N]

Hi @Clem3nt

Please share me the error logs. I will check and update you.
Which proto version you are using ?

[Pradeep P N]

Hi @SGaist
Sure, i will try to make the instructions shorter & post it

Thank you.

[Chapay]

@Clem3nt

f...............k
I spent 2 days :( with this issue

But now can build any android arch from macOS!
Sorry I use ndk 16 and not tested for you version (ndk 19)

#!/bin/sh

ANDROID_NDK_NAME="android-ndk-r16b"
PROTO_VERSION="v3.11.4"

PROTO_DIR=`pwd`/protobuf
ANDROID_RESULT_DIR=`pwd`/libprotobuf/android
NDK_ROOT=`pwd`/ndk

ANDROID_NDK_DIR="$NDK_ROOT/$ANDROID_NDK_NAME"

export ANDROID="/Volumes/Storage/dvlp/android/sdk/sdk"
export ANDROID_HOME="/Volumes/Storage/dvlp/android/sdk/sdk"

# Env variable for Android NDK.
export ANDROID_NDK="$ANDROID_NDK_DIR"
export ANDROID_NDK_HOME="$ANDROID_NDK_DIR"

# Env variable for Android cmake.
export ANDROID_CMAKE_ROOT="$ANDROID_HOME/cmake/3.6.4111459/"
export ANDROID_CMAKE="$ANDROID_CMAKE_ROOT/bin/cmake"

if [ -d "$ANDROID_NDK" ]; then
	echo "EXIST: $ANDROID_NDK"
else
	mkdir -p $NDK_ROOT
	cd $NDK_ROOT	
	# For Mac OS, change the NDK download link accordingly.
	wget "https://dl.google.com/android/repository/$ANDROID_NDK_NAME-darwin-x86_64.zip"
	unzip $ANDROID_NDK_NAME-darwin-x86_64.zip
fi

echo "******************************************************"
echo "start checkout https://github.com/google/protobuf.git"
echo "******************************************************"
if [ -d "$PROTO_DIR" ]; then
	cd $PROTO_DIR
	git fetch origin
	git reset --hard origin/master
	git pull 
	git checkout -b $PROTO_VERSION $PROTO_VERSION
	git submodule update --init --recursive
else
	git clone https://github.com/google/protobuf.git "$PROTO_DIR"	
	cd $PROTO_DIR
	git checkout -b $PROTO_VERSION $PROTO_VERSION
	git submodule update --init --recursive
fi

echo "******************************************************"
echo "end checkout https://github.com/google/protobuf.git"
echo "******************************************************"

function cmakeBuid {
	export ARCH="$1"

	echo "********************" 	
	echo "start $ARCH"
	echo "********************"

	cd "$PROTO_DIR/cmake"
	mkdir -p $ARCH
	cd $ARCH
	
	INSTALL_DIR="$ANDROID_RESULT_DIR/$ARCH"
	mkdir -p $INSTALL

	echo "********************************************************" 	
	echo "start cmake for $ARCH ....."
	echo "********************************************************"

	$ANDROID_CMAKE \
		-Dprotobuf_BUILD_PROTOC_BINARIES=OFF \
	    -Dprotobuf_BUILD_SHARED_LIBS=OFF \
	    -Dprotobuf_BUILD_EXAMPLES=OFF \
	   	-Dprotobuf_BUILD_TESTS=OFF \
	    -DCMAKE_BUILD_TYPE=Release \
	    -DCMAKE_TOOLCHAIN_FILE=$ANDROID_NDK_DIR/build/cmake/android.toolchain.cmake \
	    -DCMAKE_INSTALL_PREFIX=$INSTALL_DIR \
	    -DANDROID_NDK=$ANDROID_NDK \
	    -DANDROID_TOOLCHAIN=clang \
	    -DANDROID_ABI=$ARCH \
	    -DANDROID_NATIVE_API_LEVEL=21 \
	    -DANDROID_STL=c++_shared \
	    -DANDROID_LINKER_FLAGS="-llog -lz -lc++_static" \
	    -DANDROID_CPP_FEATURES="rtti exceptions" \
	    ..

	$ANDROID_CMAKE --build .    

	echo "********************************************************" 	
	echo "end cmake for $ARCH ....."
	echo "********************************************************"	
	
	make install

	echo "***** start: copy results *****"
	if [ -d "$ANDROID_RESULT_DIR/lib/include" ]; then
		rm -rf "$ANDROID_RESULT_DIR/lib/include"
	fi	
	
	if [ -d "$ANDROID_RESULT_DIR/lib/$ARCH" ]; then
		rm -rf "$ANDROID_RESULT_DIR/lib/$ARCH"
	fi		
	
	mkdir -p "$ANDROID_RESULT_DIR/lib/$ARCH"

	cp "$INSTALL_DIR/lib/libprotobuf-lite.a" "$ANDROID_RESULT_DIR/lib/$ARCH"
	cp "$INSTALL_DIR/lib/libprotobuf.a" "$ANDROID_RESULT_DIR/lib/$ARCH"
	cp -r "$INSTALL_DIR/include" "$ANDROID_RESULT_DIR/include"	
	echo "***** end: copy results *****"

	rm -rf "$INSTALL_DIR"

	echo "********************" 	
	echo "end $ARCH"
	echo "********************"
}

cmakeBuid "armeabi-v7a"
cmakeBuid "arm64-v8a"
cmakeBuid "x86"
cmakeBuid "x86_64"

[Chapay]

helpfull link:
https://tech.pic-collage.com/how-to-cross-compile-google-protobuf-lite-for-android-977df3b7f20c

command to get avaliable cmake params (stackowerflow):

cd "../protobufr-repo-dir/cmake"
mkdir build
cd build
cmake ..
cmake -LA | awk '{if(f)print} /-- Cache values/{f=1}'

[Pradeep P N]

@Chapay
I had built it with NDK r20b on Ubuntu 16.04 LTS
And Sorry i was travelling and hence could not respond or update.

So you were success in building it right ?

[Chapay]

@Pradeep-P-N

Yes. I used cmake instead configure script

[Pradeep P N]

Build Instructions on Ubuntu 16.04 LTS

Building Android Tool Chain :

• Few Dependencies :
  sudo apt-get install autoconf automake libtool curl make g++ unzip python gcc git

• Download NDK - Latest Stable Version ( I used NDK r20b, which was latest during i built it )

• You may want to append export NDK=$HOME/<path-to-NDK> to your ~/.bashrc
echo $NDK to verify NDK path from above.

• Creating the Standalone toolchain.
  The NDK folder contains default toolchains, or we can build it - Standalone Toolchains (Obsolete)
  Example

     $NDK/build/tools/make_standalone_toolchain.py \
           --arch arm64 \
           --api 28 \
           --stl=libc++ \
           --install-dir=$HOME/<path-to-build-dir>

• You may want to append export PATH=$HOME/<path-to-build-dir/bin:$PATH to your ~/.bashrc
echo $PATH to verify toolchain path.

Building Google’s Protobuf library :

• Clone protobuf

  mkdir $HOME/Android/NDK/google
 
  cd $HOME/Android/NDK/google

  git clone https://github.com/protocolbuffers/protobuf.git

  cd protobuf

  git submodule update --init --recursive

# Checkout if specific version is needed, example as shown below, else ignore the below command 
 git checkout -b v3.5.2 v3.5.2

• Build a native (x86) version of the protobuf libraries and compiler (protoc)
  sudo apt-get install curl autoconf libtool build-essential g++

In protobuf folder

./autogen.sh

 mkdir x86_build
 cd x86_build

# x86_pb_install folder should be created manually at below given path then run the below cmd

 ../configure --prefix=$HOME/Android/NDK/google/x86_pb_install 

 make install –j4

 cd ..

Libraries will been installed in: $HOME/Android/NDK/google/x86_pb_install/lib
protoc will be in: $HOME/Android/NDK/google/x86_pb_install/bin
include files will be in: $HOME/Android/NDK/google/x86_pb_install/include

• Build the arm64 version of the protobuf libraries:

 mkdir arm64_build

 cd arm64_build

 # arm64_pb_install folder should be created manually at below given path then run the below cmd

  CC=aarch64-linux-android-clang \
  CXX=aarch64-linux-android-clang++ \
  CFLAGS="-fPIE -fPIC" LDFLAGS="-pie -llog" \
     ../configure --host=aarch64-linux-android \
     --prefix=$HOME/Android/NDK/google/arm64_pb_install \
     --with-protoc=$HOME/Android/NDK/google/x86_pb_install/bin/protoc

  make install –j4

  cd ..

Libraries will been installed in: $HOME/Android/NDK/google/arm64_pb_install/lib
protoc will be in: $HOME/Android/NDK/google/arm64_pb_install/bin
include files will be in: $HOME/Android/NDK/google/arm64_pb_install/include

• Compiling Your Protocol Buffers
  protoc -I=$SRC_DIR --cpp_out=$DST_DIR $SRC_DIR/file_name.proto

@SGaist
Hope this helps

@Chapay, @Clem3nt
If possible please try this and update me.

[Chapay]

@Pradeep-P-N Thanks I will try

[mbruel]

@Pradeep-P-N

Thanks for this.
It's working fine.
Little remark from Nov 2020, the script make_standalone_toolchain.py is no longer needed:

#$ $NDK/build/tools/make_standalone_toolchain.py --arch arm64 --api 28 --stl=libc++ --install-dir=/home/bruel/apps/protobuf/build
#WARNING:__main__:make_standalone_toolchain.py is no longer necessary. The
#$NDK/toolchains/llvm/prebuilt/linux-x86_64/bin directory contains target-specific scripts that perform
#the same task. For example, instead of:
#
#    $ python $NDK/build/tools/make_standalone_toolchain.py \
#        --arch arm64 --api 28 --install-dir toolchain
#    $ toolchain/bin/clang++ src.cpp
#
#Instead use:
#
#    $ $NDK/toolchains/llvm/prebuilt/linux-x86_64/bin/aarch64-linux-android28-clang++ src.cpp

So for arm64, you can go directly like this
CC=$NDK/toolchains/llvm/prebuilt/linux-x86_64/bin/aarch64-linux-android$ANDROID_API-clang CXX=$NDK/toolchains/llvm/prebuilt/linux-x86_64/bin/aarch64-linux-android$ANDROID_API-clang++ CFLAGS="-fPIE -fPIC" LDFLAGS="-pie -llog" ../configure --host=aarch64-linux-android --prefix=$APPS_DIR/protobuf/release/arm64 --with-protoc=~/apps/protobuf/release/x86_64/bin/protoc

For those interested, here is my script to build on Linux for x86_64, arm64 and armv7a.

[Pradeep P N]

Hi @mbruel

That's cool,
I had a critical time & had to go with make_standalone_toolchain.py at that time.

Thanks for the updated script on this Building protobuff libs for Android 64 bit (ARM64-v8a) Qt 5.12.4.

[mbruel]

In fact with the method I've posted just above, I can build the Android arm64 library but when I deploy it to a device it's crashing with this:

E AndroidRuntime: FATAL EXCEPTION: qtMainLoopThread
E AndroidRuntime: Process: fr.mbruel.ClementineRemote, PID: 24487
E AndroidRuntime: java.lang.UnsatisfiedLinkError: dlopen failed: cannot locate symbol "_Unwind_Resume" referenced by "/data/app/fr.mbruel.ClementineRemote-eCT926MVQi5HkO3nFNVzEg==/lib/arm64/libprotobuf.so"...
E AndroidRuntime: 	at java.lang.Runtime.load0(Runtime.java:938)
E AndroidRuntime: 	at java.lang.System.load(System.java:1631)
E AndroidRuntime: 	at org.qtproject.qt5.android.QtNative$3.run(QtNative.java:348)
E AndroidRuntime: 	at org.qtproject.qt5.android.QtThread$2.run(QtThread.java:87)
E AndroidRuntime: 	at org.qtproject.qt5.android.QtThread$1.run(QtThread.java:61)
E AndroidRuntime: 	at java.lang.Thread.run(Thread.java:919)

Any idea why?
I'm using protobuf 3.13. I've tried the latest 3.17 but I've the same issue...
I'm building using this command:

NDK=~/android/android-ndk-r21
ANDROID_API=29

CC=$NDK/toolchains/llvm/prebuilt/linux-x86_64/bin/aarch64-linux-android$ANDROID_API-clang CXX=$NDK/toolchains/llvm/prebuilt/linux-x86_64/bin/aarch64-linux-android$ANDROID_API-clang++ CFLAGS="-fPIE -fPIC" LDFLAGS="-pie -llog" ../configure --host=aarch64-linux-android --prefix=~/apps/protobuf/release/arm64 --with-protoc=~/apps/protobuf/release/x86_64/bin/protoc

[mbruel]

@mbruel said in Building protobuff libs for Android 64 bit (ARM64-v8a) Qt 5.12.4:

cannot locate symbol "_Unwind_Resume"

I've updated the NDK to the r21e and the crash is gone! \o/