Building protobuff libs for Android 64 bit (ARM64-v8a) Qt 5.12.4

[Pradeep P N]

Hallo,

I am trying to build the protobuff library for android using NDK r20b.
I could not find proper solution for arm64-v8a all the links available are armv7-a

I have generated the tool chain for aarch64_linux_android which is 64bit support on Qt Android (AFAIK).

Below are few links i followed through and customized the script as per my requirement.

• https://github.com/protocolbuffers/protobuf/issues/5279#issuecomment-449727577
• https://github.com/protocolbuffers/protobuf/issues/5893#issuecomment-475841203
• https://gist.github.com/helayzhang/9034454#file-build_protobuf_android-sh

Using aarch64_linux_android from tool chain build.

 export PATH=$HOME/Android/aarch64_linux_android/bin:$PATH
 export SYSROOT=$HOME/Android/aarch64_linux_android/sysroot
 export CC="arm-linux-androideabi-gcc --sysroot $SYSROOT"
 export CXX="arm-linux-androideabi-g++ --sysroot $SYSROOT"
 export CXXSTL=$NDK/sources/cxx-stl/gnu-libstdc++/4.6

 ./configure --prefix=$(pwd)/build \
 --host=aarch64_linux_android \
 --with-sysroot=$SYSROOT \
 --disable-shared \
 --enable-cross-compile \
 --with-protoc=protoc \
 CFLAGS="-march=lib64" \
 CXXFLAGS="-march=lib64 -I$CXXSTL/include -I$CXXSTL/libs/arm64-v8a/include"

export CXXSTL=$NDK/sources/cxx-stl/gnu-libstdc++/4.6 we don't have this with ndkr20b

Can anyone please help me to build the 64bit protobuff library for Android arm64-v8a ?

Thanks in advance.

[mvuori]

I don't know anything about protobuff, but if you don't specifically need NDK 20, I'd try with 19, as at least Qt 5.12.3 doesn't seem to like v 20.

[Pradeep P N]

Thanks @mvuori

The issue is same with NDK 19.

I can build it for below NDK 19 as they have sources/cxx-stl/gnu-libstdc++/4.6 but they come with armeabi-v7a which is for 32 bit (AFAIK).
We need aarch64-linux-android for android 64 bit.

Also the folder structure has completely changed from NDK 19.

[SGaist]

Hi

From a quick look, your -march values are wrong. There's no architecture named lib64. The correct value is more likely to be arm64-v8a.

[Pradeep P N]

Hi @SGaist

I dont have these folders in the toolchain which i generated for NDK 20 & NDK 19
I am also confused with the new folder structure in updated NDK code.

[SGaist]

What folder are you talking about ?

[Pradeep P N]

@SGaist
NDK_ROOT/sources/cxx-stl has only llvm-libc++ & llvm-llibc++abi
We need the clang for Qt 5.12.4
And i want to build it for Android API 28 which is only supported from NDK 19

[Pradeep P N]

@SGaist

Below is the script i have for building it.

export PREFIX=$HOME/ARM64_Protobuf/protobuf/
export PATH=$HOME/Android/NDK19/aarch64-linux-android-4.9/bin:$PATH
export SYSROOT=$HOME/Android/NDK19/aarch64-linux-android-4.9/sysroot
export CC="aarch64-linux-android-clang --sysroot $SYSROOT"
export CXX="aarch64-linux-android-clang++ --sysroot $SYSROOT"

../configure \
--prefix=$PREFIX \
--host=aarch64-linux-android \
--with-sysroot="${SYSROOT}" \
--enable-shared \
--enable-cross-compile \
--with-protoc=protoc \
CFLAGS="-march=arm64-v8a -D__ANDROID_API__=28" \
CXXFLAGS="-frtti -fexceptions -march=arm64-v8a -D__ANDROID_API__=28" \
LIBS="-llog -lz -lc++_static"

make -j 2

make install

and i have the error below as

checking whether to enable maintainer-specific portions of Makefiles... yes
checking build system type... x86_64-pc-linux-gnu
checking host system type... aarch64-unknown-linux-android
checking target system type... aarch64-unknown-linux-android
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for aarch64-linux-android-strip... aarch64-linux-android-strip
checking for a thread-safe mkdir -p... /bin/mkdir -p
checking for gawk... no
checking for mawk... mawk
checking whether make sets $(MAKE)... yes
checking whether make supports nested variables... yes
checking whether UID '1000' is supported by ustar format... yes
checking whether GID '1000' is supported by ustar format... yes
checking how to create a ustar tar archive... gnutar
checking whether make supports nested variables... (cached) yes
checking for aarch64-linux-android-gcc... aarch64-linux-android-clang --sysroot /home/Android/NDK19/aarch64-linux-android-4.9/sysroot
checking whether the C compiler works... no
configure: error: in `/home/ARM64_Protobuf/protobuf/build':
configure: error: C compiler cannot create executables
See `config.log' for more details
make: *** No targets specified and no makefile found.  Stop.
make: *** No rule to make target `install'.  Stop.

[SGaist]

What does the config.log tell you ?

[Pradeep P N]

@SGaist Thanks for the support.

I was able to build the library & protoc for Android ARM 64 v8a using Ubuntu 16.04 LTS.
I am moving it to Solved. If some one comes up with the same issue/problem I can support on that.

[SGaist]

Glad you succeeded. It would be nice to have a write down of what you did to be able to achieve the build.

[Clem3nt]

Hello @Pradeep-P-N,

I'm trying to build protobuf for arm64 and I have the same error. How did you solve the problem?

Thank you

[Pradeep P N]

Hi @Clem3nt

Please share me the error logs. I will check and update you.
Which proto version you are using ?

[Pradeep P N]

Hi @SGaist
Sure, i will try to make the instructions shorter & post it

Thank you.

[Chapay]

@Clem3nt

f...............k
I spent 2 days :( with this issue

But now can build any android arch from macOS!
Sorry I use ndk 16 and not tested for you version (ndk 19)

#!/bin/sh

ANDROID_NDK_NAME="android-ndk-r16b"
PROTO_VERSION="v3.11.4"

PROTO_DIR=`pwd`/protobuf
ANDROID_RESULT_DIR=`pwd`/libprotobuf/android
NDK_ROOT=`pwd`/ndk

ANDROID_NDK_DIR="$NDK_ROOT/$ANDROID_NDK_NAME"

export ANDROID="/Volumes/Storage/dvlp/android/sdk/sdk"
export ANDROID_HOME="/Volumes/Storage/dvlp/android/sdk/sdk"

# Env variable for Android NDK.
export ANDROID_NDK="$ANDROID_NDK_DIR"
export ANDROID_NDK_HOME="$ANDROID_NDK_DIR"

# Env variable for Android cmake.
export ANDROID_CMAKE_ROOT="$ANDROID_HOME/cmake/3.6.4111459/"
export ANDROID_CMAKE="$ANDROID_CMAKE_ROOT/bin/cmake"

if [ -d "$ANDROID_NDK" ]; then
	echo "EXIST: $ANDROID_NDK"
else
	mkdir -p $NDK_ROOT
	cd $NDK_ROOT	
	# For Mac OS, change the NDK download link accordingly.
	wget "https://dl.google.com/android/repository/$ANDROID_NDK_NAME-darwin-x86_64.zip"
	unzip $ANDROID_NDK_NAME-darwin-x86_64.zip
fi

echo "******************************************************"
echo "start checkout https://github.com/google/protobuf.git"
echo "******************************************************"
if [ -d "$PROTO_DIR" ]; then
	cd $PROTO_DIR
	git fetch origin
	git reset --hard origin/master
	git pull 
	git checkout -b $PROTO_VERSION $PROTO_VERSION
	git submodule update --init --recursive
else
	git clone https://github.com/google/protobuf.git "$PROTO_DIR"	
	cd $PROTO_DIR
	git checkout -b $PROTO_VERSION $PROTO_VERSION
	git submodule update --init --recursive
fi

echo "******************************************************"
echo "end checkout https://github.com/google/protobuf.git"
echo "******************************************************"

function cmakeBuid {
	export ARCH="$1"

	echo "********************" 	
	echo "start $ARCH"
	echo "********************"

	cd "$PROTO_DIR/cmake"
	mkdir -p $ARCH
	cd $ARCH
	
	INSTALL_DIR="$ANDROID_RESULT_DIR/$ARCH"
	mkdir -p $INSTALL

	echo "********************************************************" 	
	echo "start cmake for $ARCH ....."
	echo "********************************************************"

	$ANDROID_CMAKE \
		-Dprotobuf_BUILD_PROTOC_BINARIES=OFF \
	    -Dprotobuf_BUILD_SHARED_LIBS=OFF \
	    -Dprotobuf_BUILD_EXAMPLES=OFF \
	   	-Dprotobuf_BUILD_TESTS=OFF \
	    -DCMAKE_BUILD_TYPE=Release \
	    -DCMAKE_TOOLCHAIN_FILE=$ANDROID_NDK_DIR/build/cmake/android.toolchain.cmake \
	    -DCMAKE_INSTALL_PREFIX=$INSTALL_DIR \
	    -DANDROID_NDK=$ANDROID_NDK \
	    -DANDROID_TOOLCHAIN=clang \
	    -DANDROID_ABI=$ARCH \
	    -DANDROID_NATIVE_API_LEVEL=21 \
	    -DANDROID_STL=c++_shared \
	    -DANDROID_LINKER_FLAGS="-llog -lz -lc++_static" \
	    -DANDROID_CPP_FEATURES="rtti exceptions" \
	    ..

	$ANDROID_CMAKE --build .    

	echo "********************************************************" 	
	echo "end cmake for $ARCH ....."
	echo "********************************************************"	
	
	make install

	echo "***** start: copy results *****"
	if [ -d "$ANDROID_RESULT_DIR/lib/include" ]; then
		rm -rf "$ANDROID_RESULT_DIR/lib/include"
	fi	
	
	if [ -d "$ANDROID_RESULT_DIR/lib/$ARCH" ]; then
		rm -rf "$ANDROID_RESULT_DIR/lib/$ARCH"
	fi		
	
	mkdir -p "$ANDROID_RESULT_DIR/lib/$ARCH"

	cp "$INSTALL_DIR/lib/libprotobuf-lite.a" "$ANDROID_RESULT_DIR/lib/$ARCH"
	cp "$INSTALL_DIR/lib/libprotobuf.a" "$ANDROID_RESULT_DIR/lib/$ARCH"
	cp -r "$INSTALL_DIR/include" "$ANDROID_RESULT_DIR/include"	
	echo "***** end: copy results *****"

	rm -rf "$INSTALL_DIR"

	echo "********************" 	
	echo "end $ARCH"
	echo "********************"
}

cmakeBuid "armeabi-v7a"
cmakeBuid "arm64-v8a"
cmakeBuid "x86"
cmakeBuid "x86_64"

[Chapay]

helpfull link:
https://tech.pic-collage.com/how-to-cross-compile-google-protobuf-lite-for-android-977df3b7f20c

command to get avaliable cmake params (stackowerflow):

cd "../protobufr-repo-dir/cmake"
mkdir build
cd build
cmake ..
cmake -LA | awk '{if(f)print} /-- Cache values/{f=1}'

[Pradeep P N]

@Chapay
I had built it with NDK r20b on Ubuntu 16.04 LTS
And Sorry i was travelling and hence could not respond or update.

So you were success in building it right ?

[Chapay]

@Pradeep-P-N

Yes. I used cmake instead configure script

[Pradeep P N]

Build Instructions on Ubuntu 16.04 LTS

Building Android Tool Chain :

• Few Dependencies :
  sudo apt-get install autoconf automake libtool curl make g++ unzip python gcc git

• Download NDK - Latest Stable Version ( I used NDK r20b, which was latest during i built it )

• You may want to append export NDK=$HOME/<path-to-NDK> to your ~/.bashrc
echo $NDK to verify NDK path from above.

• Creating the Standalone toolchain.
  The NDK folder contains default toolchains, or we can build it - Standalone Toolchains (Obsolete)
  Example

     $NDK/build/tools/make_standalone_toolchain.py \
           --arch arm64 \
           --api 28 \
           --stl=libc++ \
           --install-dir=$HOME/<path-to-build-dir>

• You may want to append export PATH=$HOME/<path-to-build-dir/bin:$PATH to your ~/.bashrc
echo $PATH to verify toolchain path.

Building Google’s Protobuf library :

• Clone protobuf

  mkdir $HOME/Android/NDK/google
 
  cd $HOME/Android/NDK/google

  git clone https://github.com/protocolbuffers/protobuf.git

  cd protobuf

  git submodule update --init --recursive

# Checkout if specific version is needed, example as shown below, else ignore the below command 
 git checkout -b v3.5.2 v3.5.2

• Build a native (x86) version of the protobuf libraries and compiler (protoc)
  sudo apt-get install curl autoconf libtool build-essential g++

In protobuf folder

./autogen.sh

 mkdir x86_build
 cd x86_build

# x86_pb_install folder should be created manually at below given path then run the below cmd

 ../configure --prefix=$HOME/Android/NDK/google/x86_pb_install 

 make install –j4

 cd ..

Libraries will been installed in: $HOME/Android/NDK/google/x86_pb_install/lib
protoc will be in: $HOME/Android/NDK/google/x86_pb_install/bin
include files will be in: $HOME/Android/NDK/google/x86_pb_install/include

• Build the arm64 version of the protobuf libraries:

 mkdir arm64_build

 cd arm64_build

 # arm64_pb_install folder should be created manually at below given path then run the below cmd

  CC=aarch64-linux-android-clang \
  CXX=aarch64-linux-android-clang++ \
  CFLAGS="-fPIE -fPIC" LDFLAGS="-pie -llog" \
     ../configure --host=aarch64-linux-android \
     --prefix=$HOME/Android/NDK/google/arm64_pb_install \
     --with-protoc=$HOME/Android/NDK/google/x86_pb_install/bin/protoc

  make install –j4

  cd ..

Libraries will been installed in: $HOME/Android/NDK/google/arm64_pb_install/lib
protoc will be in: $HOME/Android/NDK/google/arm64_pb_install/bin
include files will be in: $HOME/Android/NDK/google/arm64_pb_install/include

• Compiling Your Protocol Buffers
  protoc -I=$SRC_DIR --cpp_out=$DST_DIR $SRC_DIR/file_name.proto

@SGaist
Hope this helps

@Chapay, @Clem3nt
If possible please try this and update me.

[Chapay]

@Pradeep-P-N Thanks I will try