A repo to store all my guides/notes/knowledge/what i learned. https://silkevicious.codeberg.page
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

290 lines
16 KiB

<!DOCTYPE html>
<html lang="en">
<link rel="stylesheet" href="simple.min.css">
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>How to build an Android app and deploy an F-Droid repository</title>
<h1>How to build an Android app and deploy an F-Droid repository</h1>
<a href="index.html">Home</a>
<a href="projects.html">Projects</a>
<a href="posts.html">Posts</a>
<a href="about.html">About</a>
<blockquote>Welcome!<br />
These are our notes on how to build an Android app and deploy an F-Droid repository.<br />
This is also a reminder for the future me, when i forget steps (often).<br />
This guide is only possible with the collaboration of the Grey Eminence behind me, whose identity can't be disclosed.<br /><br />
<b>Personal suggestion:</b> if you have the chance and the possibility use a VM to do this.<br />
Android environment can require tweaks that you may not like for your main system, doing this in a VM can help, you can always nuke and redo in case things go nuts.<br />
I may explain my VM setup in a separate guide.<br /><br />
As always, you can do things in different ways, you can manually build things, you can build with F-Droid, etc...<br />
I'll try to be as linear as possible. Let's start!
<h2 id="AndroidSDK">Setup Android SDK</h2>
<p>This is a common prerequisite for both the path (manual build and F-Droid build) so don't skip yet.
<br /><br />
Install necessary packages
<pre><code>$ sudo apt install git wget unzip default-jdk-headless
Create a new folder for this project and cd into it
<pre><code>$ mkdir app_builds
$ cd app_builds
Download Android SDK and save it as <code>sdk.zip</code>:
<pre><code>$ wget https://dl.google.com/android/repository/commandlinetools-linux-6858069_latest.zip -O sdk.zip</code></pre>
<b>Note:</b> this is the current link as of writing this guide. This can change in future. Check <a href="https://developer.android.com/studio#command-tools">https://developer.android.com/studio#command-tools</a> for future updates.
<br />Now we have to extract the <code>sdk.zip</code> to <code>android_sdk/cmdline-tools/</code>
<br /><br />
Create the directories
<pre><code>$ mkdir -p android_sdk/cmdline-tools/</code></pre>
Extract the files
<pre><code>$ unzip sdk.zip -d android_sdk/cmdline-tools</code></pre>
Command line tools of the Android SDK expects to be in a specific directory, so let's give what it want:
<pre><code>$ mv android_sdk/cmdline-tools/cmdline-tools android_sdk/cmdline-tools/latest</code></pre>
Add <code>ANDROID_HOME</code> environment variable
<pre><code>$ echo "export ANDROID_HOME=~/app_builds/android_sdk" >> ~/.bashrc
$ source ~/.bashrc</code></pre>
<!--export ANDROID_HOME=~/app_builds/android_sdk
export ANDROID_NDK=~/app_builds/android_sdk
export ANDROID_SDK_ROOT=~/app_builds/android_sdk
export PATH=$PATH:$ANDROID_HOME/platform-tools-->
Add Java location to <code>PATH</code>
<pre><code>$ echo "export JAVA_HOME=/usr/lib/jvm/java-11-openjdk-amd64" >> ~/.profile
$ echo "export PATH=\$PATH:\$JAVA_HOME/bin" >> ~/.profile
$ source ~/.profile</code></pre>
Now we have to accept the licenses
<pre><code>$ ./android_sdk/cmdline-tools/latest/bin/sdkmanager --licenses</code></pre>
Once accepted see the package you can install with the command:
<pre><code>$ ./android_sdk/cmdline-tools/latest/bin/sdkmanager --list</code></pre>
At the moment of writing, the latest build tool version is 30.0.3. We need it to build the apps, so let's download it with:
<pre><code>$ ./android_sdk/cmdline-tools/latest/bin/sdkmanager --install "build-tools;30.0.3"</code></pre>
Prerequisites are done. Now, if you want to manual build apps, keep reading. If you want to build apps with F-Droid skip to <a href="#FDroidServer">F-Droid Server</a> Chapter.</p>
<h2 id="ManualBuild">Manual Android app build</h2>
<p>If you never built an app before, i suggest you to start with manually building an app.
This will let you learn a lot of things while troubleshooting errors you will encounter.<br /><br />
Let's take Fedilab as example for build. The procedure is very similar with almost any app.<br />
Clone Fedilab’s repository
<pre><code>$ git clone https://framagit.org/tom79/fedilab.git fedilab</code></pre>
Go to the cloned folder and build!
<pre><code>$ cd fedilab
$ ./gradlew assembleFdroid</code></pre>
I wrote <code>assembleFdroid</code> after the command. It's not the same for every app, for example for Blabber.im is <code>assembleGit</code>.<br />
Check the build.gradle file in your app's git repo to see how is called the flavour.<br />
Different flavour leads to different results: some flavours for example requires Google dependencies we may not like...<br />
It may happens the app has no flavours. In this case it's sufficient to execute
<pre><code>$ ./gradlew assemble</code></pre></p>
<h3 id="CreateKeyManual">Signature Key</h3>
<p><mark>This is needed just the first time.</mark>
<br />We need to generate our key to sign APK.
<pre><code>$ cd ~/app_builds/
$ keytool -genkey -v -keystore NAME.keystore -keyalg RSA -keysize 4096 -validity 10000 -alias NAME</code></pre>
It will ask you some infos, some can be left blank. After we have to convert it to PKCS12 format to fix the warning that just appeared.
<pre><code>$ keytool -importkeystore -srckeystore NAME.keystore -destkeystore NAME.keystore -deststoretype pkcs12</code></pre>
To make it quicker to sign the APKs i suggest you to add an alias in the end of .bashrc file.
<pre><code>$ nano ~/.bashrc</code></pre>
Paste and edit this
<pre><code>alias apksigner="~/app_builds/android_sdk/build-tools/30.0.3/apksigner sign --ks ~/app_builds/NAME.keystore --ks-key-alias NAME"</code></pre>
<pre><code>$ source ~/.bashrc</code></pre></p>
<h3 id="SignAPKs">Sign APKs</h3>
<p>Now sign the apk (continuing the Fedilab example)
<pre><code>cp app/build/outputs/apk/fdroid/release/app-fdroid-release-unsigned.apk fedilab.apk
~/app_builds/android_sdk/build-tools/30.0.3/apksigner sign --ks <pathto>/YOURALIAS.keystore --ks-key-alias YOURALIAS fedilab.apk</code></pre>
Or if you followed my suggestion of the alias just
<pre><code>$ apksigner fedilab.apk</code></pre>
Done! Copy it to your phone and install! Of course, since your signature is different from the F-Droid one (if you installed already it from their repo) you need to uninstall that first.</p>
<h3 id="ManualDaily">Daily</h3>
<p>Go in every folder for each app you want to build, get the latest code, build and sign.
<pre><code>$ cd fedilab
$ git pull
$ ./gradlew assembleFdroid
$ cp app/build/outputs/apk/fdroid/release/app-fdroid-release-unsigned.apk fedilab.apk
$ apksigner fedilab.apk</code></pre>
As you can see this can take some time if you have a lot of apps...</p>
<h2 id="FDroidServer">F-Droid Server</h2>
<p>Instead of manual build each app, you could automatise things a bit and use F-Droid server.<br />
This requires more initial setup, but hopefully you gain time daily, especially if you want to build a lot of apps.</p>
<h3 id="SetupFDroidServer">Setup</h3>
<p>We need few more packages</p>
<pre><code>$ sudo apt install python3-pip curl rsync</code></pre>
<p>If you aren't already there, move yourself into app_builds folder</p>
<pre><code>$ cd app_builds</code></pre>
<p>Clone the fdroidserver repo</p>
<pre><code>$ git clone https://gitlab.com/fdroid/fdroidserver.git _fdroidserver</code></pre>
<p>Install fdroidserver using pip</p>
<pre><code>$ pip3 install -e _fdroidserver</code></pre>
<p>Now run the command <code>fdroid</code> to check if it works. If not, try this</p>
<pre><code>$ source ~/.profile</code></pre>
<p>Now create a folder for this and cd into it</p>
<pre><code>$ mkdir fdroid
$ cd fdroid</code></pre>
<p>Initialize a fdroid working directory</p>
<pre><code>$ fdroid init</code></pre>
<p>Previous command will create a keystore, which is used for signing your F-Droid repository. We are going to remove it and manually create a new one. If you generated the key in the manual steps you can use that and skip this step.</p>
<pre><code>$ rm keystore.p12 keytool -genkey -keystore keystore.p12 -alias NAME -keyalg RSA -keysize 4096 -sigalg SHA256withRSA -validity 10000 -storetype pkcs12 -dname "CN=NAME, OU=F-Droid" -J-Duser.language=en</code></pre>
<p>Replace <code>NAME</code> with a name you like.
<br />
It will ask you for a password. Type a good password.
<br />
Edit the <code>config.yml</code> and update below values in it nano config.yml</p>
<pre><code>keystorepass: #password you used in previous step
keypass: #password you used in previous step
repo_keyalias: NAME #Replace NAME with the same value you used when creating the keystore
keydname: CN=NAME, OU=F-Droid #Replace NAME with the same value you used when creating the keystore</code></pre>
<p>Save and exit the text editor (press <kbd>CTRL+X</kbd> then press <kbd>Y</kbd> ). We're done!</p>
<h3 id="BuildFDroidServer">Build</h3>
<p>First we need to tell the F-Droid what to build. You have 2 ways.
<br /><br />
The easiest is fetch a metadata configuration from F-Droid Data repo of your app (if exists there).<br />
Copy metadata in <code>fdroid/metadata</code> folder. The metadata file name is something like <code>de.pixart.messenger.yml</code><br />
Then run this in <code>fdroid</code> folder</p>
<pre><code>$ fdroid checkupdates --allow-dirty --verbose --auto</code></pre>
<p>It will clone the repo to <code>fdroid/build</code> folder.
<br /><br />
<br /><br />
Use this command to import an app:</p>
<pre><code>$ fdroid import --url=http://address.of.project</code></pre>
<p>It will create a basic metadata file and you can edit it yourself.
<br /><br />
Now let's trigger the build with:</p>
<pre><code>$ fdroid build --latest --no-tarball --verbose de.pixart.messenger</code></pre>
<p>If you have multiple apps you can build them all with <code>--all</code> instead of app codename. If the app is already build (no version change), it skips the build of course.</p>
<pre><code>$ fdroid build --latest --no-tarball --verbose --all</code></pre>
<p>If it says "Build successful", <mark>congrats</mark>! You made it!<br />
If not, check what it says. Usually you just have to install some SDK packages. Check <a href="#Troubleshoot">Troubleshoot</a> Chapter to fix some error i encountered so far in my experience.</p>
<h3 id="SignFDroidServer">Sign</h3>
<p>Similar to the manual build, we have to sign APKs before we made them available.<br />
The apk you built are in the unsigned/ folder.</p>
<pre><code>$ ../android_sdk/build-tools/30.0.3/apksigner sign --ks keystore.p12 --ks-key-alias NAME unsigned/NAMEOFTHE.apk</code></pre>
<p>Replace <code>NAME</code> with a name you used to create the key and <code>NAMEOFTHE.apk</code> with the name of the file. Insert the password and done!.
Of course if you added an alias for apksigner it's way easier. See <a href="#CreateKeyManual">above</a></p>
<h3 id="PrepareForDeploy">Prepare for deploy</h3>
<p><mark>This is needed just the first time.</mark>
<br />Place an icon of your choice into /fdroid folder.<br />
I created a cool QR code with a logo inside with the url of my repo and my avatar here: https://www.logodesign.net/qrcode-generator
<br /><br />
Add or update these in <code>config.yml</code>
<pre><code>make_current_version_link: false
deploy_process_logs: false
repo_url: https://codeberg.org/silkevicious/apkrepo/raw/branch/master/fdroid/repo/
repo_name: My F-Droid Repo
repo_description: This is a repository of apps to be used with F-Droid.
archive_older: 0</code></pre>
I decided to make available via git my repo. You can also make available via normal website (i may cover this in future).<br />
Edit config.yml Add or update these in config.yml
<pre><code>git_mirror_size_limit: 100MB
identity_file: ~/.ssh/ssh_key_for_git
servergitmirrors: git@codeberg.org:silkevicious/apkrepo.git</code></pre>
<h3 id="Deploy">Deploy</h3>
Place APKs fdroid/repo, from your manually build folder or from the unsigned folder, then run in fdroid folder
<pre><code>$ cd fdroid
$ fdroid update</code></pre>
You should run with -c if you haven't copied metadata from fdroid so it can add a skeleton of metadatas to edit.
<pre><code>$ fdroid update -c</code></pre>
And now
<pre><code>$ fdroid deploy</code></pre>
If you add your repo to fdroid you can now download and install your apps. Great job!
<h3 id="ServerDaily">Daily</h3>
<p>Go in fdroid folder, build, sign and deploy
<pre><code>$ sudo apt update && apt upgrade -y
$ cd ~/app_builds/fdroid
$ fdroid checkupdates --allow-dirty --auto && fdroid build --latest --no-tarball --verbose --all</code></pre>
then sign if there are new builds (this is the only boring move), move to repo folder and
<pre><code>$ fdroid update && fdroid deploy</code></pre>
Sometimes check also if the fdroidserver got updates
<pre><code>$ cd ~/app_builds/_fdroidserver
$ git pull</code></pre>
Thanks to the -e parameter while installing it we don't have to do anything else.</p>
<h2 id="Troubleshoot">Troubleshooting</h2>
<h5>Packages missing</h5>
Some apps require extra packages to be installed on the system.
For example <code>npm</code> is required by Tutanota, <code>mercurial</code> by Klar and <code>maven</code> by Seafile. There can be others, depending what you build.
<pre><code>$ sudo apt install npm mercurial maven</code></pre>
<h5>NDK missing</h5>
Some apps require specific versions of NDK.
<pre><code>$ ./android_sdk/cmdline-tools/latest/bin/sdkmanager --list
$ ./android_sdk/cmdline-tools/latest/bin/sdkmanager --install "ndk;21.3.6528147"</code></pre>
Also add or update these in <code>config.yml</code>
r21d: $ANDROID_HOME/ndk/21.3.6528147/</code></pre>
For example i have some ndk in my config:
r17c: $ANDROID_HOME/ndk/17.2.4988734/
r20b: $ANDROID_HOME/ndk/20.1.5948944/
r21: $ANDROID_HOME/ndk/21.0.6113669/
r21d: $ANDROID_HOME/ndk/21.3.6528147/
r22: $ANDROID_HOME/ndk/22.0.7026061/</code></pre>
<h5>Java errors</h5>
Some apps gave me some java errors. They required the old Java 8. This instruction are for Debian, but with few edits are ok for any other distro.
<pre><code>$ sudo sh -c "echo 'deb http://deb.debian.org/debian/ stretch main' >> /etc/apt/sources.list.d/stretch.list"
$ sudo sh -c "echo '\ndeb http://security.debian.org/debian-security/ stretch/updates main' >> /etc/apt/sources.list.d/stretch.list"
$ sudo sh -c "printf 'Package: *\nPin: release a=stretch\nPin-Priority: 90\n' >> /etc/apt/preferences.d/limit-stretch"
$ sudo apt update && sudo apt install openjdk-8-jdk-headless</code></pre>
and edit <code>~/.profile</code>
and then
<pre><code>$ source ~/.profile</code></pre>
Also add or update these in <code>config.yml</code>
8: /usr/lib/jvm/java-8-openjdk-amd64/
11: /usr/lib/jvm/java-11-openjdk-amd64/</code></pre>
<h5>Warning or error about permission.</h5>
Happened to me after i nuked and rebuilt a VM. I moved files back and forth.
Check permission of ssh and config.yml keys.
<pre><code>$ chmod 0600 config.yml</code></pre>
<h5>Can't deploy on git because permission error</h5>
Happened to me the first time, with maiden VM.
start ssh agent and add your ssh key
<pre><code>$ eval "$(ssh-agent -s)"
$ ssh-add -K /Users/you/.ssh/id_rsa</code></pre>
You have to put your git host in the known hosts otherwise it will fail
<pre><code>$ git clone git@codeberg.org:silkevicious/apkrepo.git test</code></pre>
You may also want to config globally your email and name (not mandatory):
<pre><code>$ git config --global user.email "your@email.com"
$ git config --global user.name "Your Name"</code></pre>
<p>Made with &#10084; by <a href="https://codeberg.org/silkevicious">Sylke Vicious</a> - License <a href="http://www.wtfpl.net/"><img src="http://www.wtfpl.net/wp-content/uploads/2012/12/wtfpl-badge-4.png" width="80" height="15" alt="WTFPL" /></a></p>