diff --git a/README.md b/README.md new file mode 100644 index 0000000..26b0db4 --- /dev/null +++ b/README.md @@ -0,0 +1,234 @@ +# fdroid-repo-ci + +Shared tooling to sign a prebuilt APK, build a signed F-Droid repo index over +it, and publish the result to an S3-compatible bucket -- consumed as a +composite action by whichever repository currently holds the signing keys +and runs the publish. Which repository that is has changed more than once, +so nothing here names it; see "Not hard-coded" below. + +Same shape as [weblib-ci], and public for the same measured reason: a +private repository cannot be consumed cross-repo via `uses:` on this Gitea +instance (see weblib-ci's README) -- every cross-repo form fails while the +repo being referenced is private, and only the full-URL form works once it +is public. + +**Nothing secret belongs here, ever.** Keys, keystores, credentials, bucket +names and hostnames all live in the private repo that consumes this one and +arrive as inputs -- arguments and environment variables, never committed +here. + +## What is here + +| script | what it does | +|---|---| +| `sign-apk.sh` | check alignment, then sign a prebuilt unsigned release APK with a keystore the caller supplies | +| `fdroid-update.sh` | run `fdroid update` over a `repo/` directory of already-signed APKs and produce the signed index | +| `publish.sh` | upload the resulting repo tree to an S3-compatible bucket | +| `with-nixpkgs.sh` | run a command with one or more nixpkgs packages on PATH, pinned to the revision in `nixpkgs-pin` | +| `check_zip_alignment.py` | the alignment check `sign-apk.sh` runs before signing (see "Align before sign" below) | +| `action.yml` | the composite action a consumer includes to get all of the above | + +Deliberately **not** in scope: fetching the APK. That needs an upstream +credential and is the consumer's business, not the tooling's -- which also +keeps this repo from knowing which upstreams exist. + +All of it is standard library / plain bash plus the pinned tools below. They +are *run*, not built, so this repo has no flake -- see "Pinning nixpkgs" +below for what that costs and why it is still the right call here. + +## Using it + +```yaml +- uses: actions/checkout@v4 +- id: ci + uses: https://git.chaosbit.de/weblib/fdroid-repo-ci@main +- env: + APK_KEYSTORE_PASSWORD: ${{ secrets.APK_KEYSTORE_PASSWORD }} + run: | + bash "${{ steps.ci.outputs.path }}/sign-apk.sh" \ + app-release-unsigned.apk app-release.apk \ + "$APK_KEYSTORE" release-key APK_KEYSTORE_PASSWORD + +- env: + INDEX_KEYSTORE_PASSWORD: ${{ secrets.INDEX_KEYSTORE_PASSWORD }} + run: | + bash "${{ steps.ci.outputs.path }}/fdroid-update.sh" \ + "$FDROID_REPO_ROOT" "$INDEX_KEYSTORE" repo-key INDEX_KEYSTORE_PASSWORD + +- env: + AWS_ACCESS_KEY_ID: ${{ secrets.PUBLISH_ACCESS_KEY_ID }} + AWS_SECRET_ACCESS_KEY: ${{ secrets.PUBLISH_SECRET_ACCESS_KEY }} + run: | + bash "${{ steps.ci.outputs.path }}/publish.sh" \ + "$FDROID_REPO_ROOT/repo" "$BUCKET" "$S3_ENDPOINT_URL" +``` + +**Use the full URL, `https://git.chaosbit.de/weblib/fdroid-repo-ci@main`, not +`weblib/fdroid-repo-ci@main`.** Measured on weblib-ci (weblib-archive#44): +the bare `owner/repo` form resolves against the instance's default actions +URL rather than this host and fails outright. The full-URL form is also the +only one that works while a referenced repo is private -- irrelevant for +*this* repo, which is public on purpose, but relevant for whatever repo is +consuming it, since that one holds the keys. + +`$APK_KEYSTORE`/`$INDEX_KEYSTORE`/`$FDROID_REPO_ROOT`/`$BUCKET`/ +`$S3_ENDPOINT_URL` above are the consumer's own paths and values -- nothing +in this repo defaults or hints at them. Env var *names* like +`APK_KEYSTORE_PASSWORD` are the consumer's choice too, not a convention this +repo imposes: `sign-apk.sh` and `fdroid-update.sh` each take the name of the +environment variable to read a password from as an argument, specifically so +two different keystores in the same job (the APK key and the separate repo +index key -- see below) never have to share one variable name. + +## Align before sign, and how "no `zipalign`" was verified + +`zipalign` then `apksigner`. Signing first and aligning after is the old +`jarsigner`-era order and invalidates an `apksigner`-era (v2/v3) signature, +because that scheme signs the exact bytes of the packaged zip and alignment +padding changes those bytes. + +**`apksigner` is a standalone nixpkgs attribute** (`nixpkgs#apksigner`, +verified at 35.0.6). **`zipalign` is not** -- it exists only inside +`androidenv`'s composed build-tools, which pulls a full versioned Android SDK +and its licence machinery for one binary. Before accepting that as a +dependency, three things were measured rather than assumed: + +1. **Google's own docs** (developer.android.com/tools/zipalign): "If you + build using Android Studio, which uses the Android Gradle plugin (AGP), + this is done automatically. In this case, you should still use zipalign + to verify that the APK is aligned, but you don't need to align it." AGP is + what `flutter build apk` (or any Gradle-based Android build) drives + underneath, so this applies to any such build's output, not just + Android-Studio-initiated ones. +2. **`fdroidserver` 2.4.3's own source never calls `zipalign`, anywhere** -- + grepped across all 38 of its modules. Its APK-signature-copying code + (`apksigcopier.py`, used for reproducible-build verification, a different + flow than binary-repo mode) implements realignment itself in pure Python + against the zip format directly, rather than shelling out to the + `zipalign` binary. +3. **`apksigner` does not check or enforce alignment at all**, in either + direction -- measured directly: a deliberately misaligned fixture zip + (including a `.so` misaligned to the stricter 4096-byte boundary) was + signed and then verified with `apksigner verify --verbose`, and the + output carried no warning about alignment whatsoever. + +**Confidence: high** that a Gradle/AGP-built unsigned release APK is already +aligned, resting on (1) and (2) above. **Not independently confirmed** +against a real `flutter build apk` output run in this environment -- that +build was judged too expensive to attempt for this measurement. A sibling +repo's own CI proposal for building the Flutter app was rejected on the same +"the toolchain is too large to pull on every push" grounds this pin sidesteps +(composing a multi-platform Android SDK plus NDK pulls tens of gigabytes), +and the host available for this measurement had well under that in free +disk space. + +Point (3) is why this repo does not simply trust point (1) and move on: +since nothing downstream complains about misalignment either, a build that +stopped emitting aligned output would fail silently -- an APK that signs and +verifies fine but behaves badly (or refuses to install) on a real device. +`check_zip_alignment.py`, a few lines against the standard library `zipfile` +module, checks every uncompressed entry of the APK `sign-apk.sh` is asked to +sign, and refuses to sign one that fails it. If that check ever legitimately +trips, the fix is to add `androidenv`'s `zipalign` as a dependency here, not +to weaken or remove the check. + +## The repo index key is not the APK signing key + +Binary-repo mode (every APK arrives already signed) means `fdroid update` +reads each APK's existing signature and never touches an app's own signing +key. It signs the **index** with a separate key of its own -- `fdroid-update.sh` +takes its own keystore/alias/password arguments, deliberately unrelated to +`sign-apk.sh`'s. Conflating the two would mean one compromised key +undermines both the apps and the index that vouches for them. + +`fdroid-update.sh` writes `/config.yml` itself, on every run, +using fdroidserver's own `{env: VARNAME}` indirection for both passwords -- +the exact mechanism its `common.py` implements *specifically so config.yml +can be safely git-managed*: "it is a good idea to manage config.yml (WITHOUT +PASSWORDS!) in git." A caller never has to know or hand-write fdroidserver's +config format. + +## Pinning nixpkgs + +`fdroidserver`'s output feeds a *signed index*, so an unpinned or +channel-drifting nixpkgs is a worse trade here than for a disposable test +job. This repo carries a `nixpkgs-pin` file -- one line, a 40-character +nixpkgs revision -- and `with-nixpkgs.sh` refuses to run at all if that file +is missing or malformed, rather than silently falling back to an unpinned +`nix shell nixpkgs#foo`, which re-resolves the registry and fetches whatever +the channel currently points at. + +This is deliberately **not** the same mechanism as [weblib-ci]'s +`with-nixpkgs.sh`, which reads the *consuming* repo's `flake.lock`. This repo +has no flake (see weblib-ci#9 for the same tension in that repo: a +`flake.lock` existing only to hold one revision string), and per this repo's +own originating issue, the repo that consumes it holds signing keys and is +not guaranteed to be a flake either -- reading a caller's `flake.lock` the +way weblib-ci's script does would not even find one here. So the pin travels +with *this* repo instead, in a file named `nixpkgs-pin` on purpose: it is the +exact fallback weblib-ci#9 proposes adding to the shared script (flake.lock +present -> today's behaviour there; else `nixpkgs-pin` -> a bare revision; +else fail loudly). If that lands, this repo already speaks its format and +nothing here needs to change. + +A `nixpkgs-pin` file has no `nix flake update` -- it is bumped by hand. +Accepted here for the same reason weblib-ci#9 accepted it for a +stdlib-only test suite, generalised: the alternative (a flake that builds +nothing, just to hold a lockfile) is a lot of structure for one string, and a +genuinely broken pinned nixpkgs would be caught immediately by this repo's +own test suite. + +`fdroidserver` is in nixpkgs as `fdroidserver`, verified at 2.4.3. +`publish.sh` uses `awscli2` for the same S3-compatible upload every consumer +here needs, resolved through the same pin. + +## Testing + +```bash +bash run_tests.sh +``` + +42 assertions across five suites, ~20s on a warm nix store: + +* `test_check_zip_alignment.py` -- pure standard library, no nix, no + network at all; builds an aligned and a misaligned fixture zip by hand and + checks both directions, including the 4096-byte boundary specific to `.so` + entries. +* `test_with_nixpkgs.sh` -- offline; every way the pinning wrapper is + supposed to refuse rather than silently degrade (missing pin, malformed + pin, too few arguments). Does not exercise the happy path of actually + resolving a `github:nixos/nixpkgs/...` reference automatically, since that + needs network (or an already-warm local cache) the same way weblib-ci's + own `with-nixpkgs.sh` does, and that repo does not automate a test for it + either. It was checked by hand instead. +* `test_sign_apk.sh` and `test_fdroid_update.sh` -- full end-to-end runs + against a throwaway keystore generated fresh per run, real `apksigner` + and `fdroid update`. Needs the pinned toolchain to be resolvable (locally + cached, or fetched once over the network); nothing beyond that. +* `test_publish.sh` -- argument/credential validation and `--dry-run` + command construction, fully offline, plus one real invocation against a + closed local port (`127.0.0.1:1`, nothing listens there) to prove the + built command genuinely attempts a network call rather than failing on its + own arguments. The actual upload path is out of scope for an offline + suite, matching the constraint that only `publish.sh` needs network in the + first place. + +No `.gitea/workflows` runs this, on the same reasoning weblib-ci#9 recorded: +run it by hand before pushing. If that changes for weblib-ci it should +probably change here too, at the same time. + +## Not hard-coded + +No bucket name, repo URL, app id, keystore path, or channel name or count -- +a consumer may have one publish target or several, under whatever names it +likes; nothing here assumes "release" and "dev", or that there are exactly +two. No consumer is named anywhere either: the repository that runs the +publish has already changed twice during design, so naming one would be +wrong again shortly. + +The one exception is deliberate and unavoidable: **this repository's own +address** appears in the `uses:` examples above, because that is what a +consumer has to type. Those lines are what will need editing if this repo +moves organisation -- nothing else should. + +[weblib-ci]: https://git.chaosbit.de/weblib/weblib-ci diff --git a/action.yml b/action.yml new file mode 100644 index 0000000..c37f173 --- /dev/null +++ b/action.yml @@ -0,0 +1,39 @@ +# Makes sign-apk.sh, fdroid-update.sh, publish.sh and with-nixpkgs.sh +# available to a workflow and reports where they are. +# +# Consume it with the **full URL**, the same reason recorded in weblib-ci's +# README: `uses: owner/repo@main` resolves against the instance's default +# actions URL rather than this host, and only the full URL form works while +# the repository being referenced is private -- and the repository holding +# the signing keys that consumes this one is private on purpose. +# +# - uses: actions/checkout@v4 +# - id: ci +# uses: https://git.chaosbit.de/weblib/fdroid-repo-ci@main +# - run: bash "${{ steps.ci.outputs.path }}/sign-apk.sh" ... +# +# Nothing here reads a secret or names a bucket, a keystore or a consumer +# repo -- those all stay on the caller's side of this action's output path. +name: F-Droid repo publishing tools +description: >- + Shared, key-free tooling to align-check and sign a prebuilt APK, run + `fdroid update` over a binary repo, and publish the result to an + S3-compatible bucket. The caller supplies the keys, the bucket and the + APK; this action supplies the pinned toolchain and the scripts. + +outputs: + path: + description: >- + Directory holding with-nixpkgs.sh, nixpkgs-pin, sign-apk.sh, + fdroid-update.sh and publish.sh. + value: ${{ github.action_path }} + +runs: + using: composite + steps: + - shell: bash + run: | + echo "fdroid-repo-ci tools at ${{ github.action_path }}" + for f in with-nixpkgs.sh nixpkgs-pin sign-apk.sh fdroid-update.sh publish.sh; do + test -f "${{ github.action_path }}/$f" + done diff --git a/check_zip_alignment.py b/check_zip_alignment.py new file mode 100755 index 0000000..25d175e --- /dev/null +++ b/check_zip_alignment.py @@ -0,0 +1,96 @@ +#!/usr/bin/env python3 +"""Check that every uncompressed entry in a zip/APK starts at an aligned offset. + +Exists to *verify* rather than assume the zipalign decision recorded in +sign-apk.sh: this repo has no `zipalign` (it is not a standalone nixpkgs +attribute -- fdroid-repo-ci#1), on the strength of Android's own docs saying +a build made through the Android Gradle Plugin already aligns its zip entries +and that `zipalign` is only needed afterwards to check that, not to do it. +`apksigner` itself does not check alignment at all -- signing and verifying a +deliberately misaligned fixture produces no warning, measured while writing +this script -- so nothing downstream catches a regression here. This does. + +Standard library only (`zipfile`, `struct`), like the rest of this repo's +Python. No `zipalign` binary is invoked or required. + +Alignment only matters for *uncompressed* (STORED) entries: a compressed +entry is inflated into a fresh buffer before use, so its position in the zip +file is irrelevant. Two rules, matching what `zipalign`/`apksigner` apply: + +* `.so` (native libraries) must start on a 4096-byte boundary, so the + loader can mmap them directly. +* everything else stored must start on a 4-byte boundary. + +`zipfile.ZipInfo` records the *central directory's* header offset and extra +field, which is not always identical to the *local* file header at that +offset -- the two can carry different extra-field lengths. Reading the local +header directly, the same way apksigcopier's realignment code does, is what +makes this check exact rather than approximate. +""" + +import argparse +import struct +import sys +import zipfile + +LOCAL_HEADER_SIGNATURE = b"PK\x03\x04" + + +def local_data_offset(fh, header_offset): + """Where this entry's data actually starts, per its local file header.""" + fh.seek(header_offset) + sig = fh.read(4) + if sig != LOCAL_HEADER_SIGNATURE: + raise ValueError(f"no local file header at offset {header_offset} " + f"(got {sig!r})") + fh.seek(header_offset + 26) + name_len, extra_len = struct.unpack(" \ +# [] +# +# is the directory *containing* repo/ -- fdroidserver's own +# layout, not repo/ itself. This script writes /config.yml itself +# on every run (see below), so an existing one is overwritten; nothing else +# under is touched beyond what `fdroid update` writes into repo/. +# +# / name environment variables holding the +# passwords, the same convention as sign-apk.sh and for the same reason: an +# argument is visible in the job log and in `ps`, a password must not be. If +# is omitted, the keystore password is reused as the key +# password. +# +# Needs no network (fdroid-repo-ci#1) -- it only reads repo/ and signs. +# Pins fdroidserver, the JDK (fdroidserver shells out to jarsigner/keytool +# for the index signature) and apksigner (for the per-entry v1 signature in +# newer index formats) via with-nixpkgs.sh/nixpkgs-pin, next to this script. +set -euo pipefail + +here=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd) + +if [ "$#" -lt 4 ]; then + cat >&2 <<'USAGE' +usage: fdroid-update.sh \ + [] [extra fdroid update args...] +USAGE + exit 2 +fi + +root=$1 +keystore=$2 +alias_name=$3 +ks_pass_env=$4 +shift 4 + +key_pass_env=$ks_pass_env +if [ "$#" -gt 0 ] && [[ $1 != -* ]]; then + key_pass_env=$1 + shift +fi + +extra_args=("$@") + +[ -d "$root/repo" ] || { echo "fdroid-update.sh: no repo/ under $root" >&2; exit 2; } +[ -f "$keystore" ] || { echo "fdroid-update.sh: no such keystore: $keystore" >&2; exit 2; } +: "${!ks_pass_env:?fdroid-update.sh: environment variable $ks_pass_env (keystore password) is not set}" +: "${!key_pass_env:?fdroid-update.sh: environment variable $key_pass_env (key password) is not set}" + +# fdroidserver's config.yml supports "{env: VARNAME}" in place of any string +# value specifically so a config that names a keystore and its passwords can +# still be free of secrets -- fdroidserver's own common.py: "it is a good +# idea to manage config.yml (WITHOUT PASSWORDS!) in git". Writing it here +# means a caller never has to hand-write fdroidserver's config format, and +# this repo -- not every consumer -- is the one place that has to track it if +# it changes. +cat > "$root/config.yml" </repo [] +# +# Bucket, endpoint and prefix are all arguments, never baked in here: this +# repo does not know, and per fdroid-repo-ci#1 must never come to know, which +# bucket or host any consumer publishes to. Credentials come from the +# standard AWS_ACCESS_KEY_ID/AWS_SECRET_ACCESS_KEY/AWS_SESSION_TOKEN +# environment variables that `aws` itself reads -- nothing here parses or +# forwards them as arguments. +# +# Region is required by the `aws` CLI even against a non-AWS S3-compatible +# backend that ignores it; set AWS_DEFAULT_REGION if the endpoint cares, +# otherwise the placeholder below is never actually consulted. +# +# Uses `aws s3 sync`, not `cp --recursive`: re-publishing re-uploads only +# what changed. Does **not** pass `--delete` by default -- an F-Droid repo +# normally keeps every APK it has ever served (older index entries can still +# point at them), so silently pruning the remote on every publish is the +# wrong default. Pass `--delete` as an extra argument if a caller genuinely +# wants the remote pruned to match local. +# +# `fdroid update` needs no network; this is the one script in this repo that +# does (fdroid-repo-ci#1), and is accordingly the one script whose happy path +# the test suite cannot exercise against a real bucket. `--dry-run` prints +# the exact command this script would run -- including the `with-nixpkgs.sh` +# pin -- and exits without touching the network, which is what the test +# suite asserts against. (A PATH-stubbed `aws` cannot do this job: `nix +# shell` puts the packages it was asked for at the *front* of PATH, ahead of +# anything already there, so a stub earlier on the caller's PATH would simply +# be shadowed by the real `aws` from awscli2.) Separately, the test suite +# also runs this for real against a closed local port, to confirm the +# constructed command is well-formed and genuinely attempts a network call +# rather than failing on argument parsing. +# +# Pins awscli2 via with-nixpkgs.sh/nixpkgs-pin, next to this script. +set -euo pipefail + +here=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd) + +if [ "$#" -lt 3 ]; then + cat >&2 <<'USAGE' +usage: publish.sh [--dry-run] [] [extra aws s3 sync args...] + +Credentials: AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY (and optionally +AWS_SESSION_TOKEN) must already be exported -- never passed as arguments. +USAGE + exit 2 +fi + +dry_run=false +if [ "$1" = "--dry-run" ]; then + dry_run=true + shift +fi + +local_dir=$1 +bucket=$2 +endpoint=$3 +shift 3 + +prefix="" +if [ "$#" -gt 0 ] && [[ $1 != -* ]]; then + prefix=$1 + shift +fi + +extra_args=("$@") + +[ -d "$local_dir" ] || { echo "publish.sh: no such directory: $local_dir" >&2; exit 2; } +: "${AWS_ACCESS_KEY_ID:?publish.sh: AWS_ACCESS_KEY_ID is not set}" +: "${AWS_SECRET_ACCESS_KEY:?publish.sh: AWS_SECRET_ACCESS_KEY is not set}" + +dest="s3://$bucket" +if [ -n "$prefix" ]; then + dest="$dest/${prefix#/}" +fi + +cmd=(bash "$here/with-nixpkgs.sh" awscli2 aws s3 sync "$local_dir" "$dest" + --endpoint-url "$endpoint" + --region "${AWS_DEFAULT_REGION:-us-east-1}" + "${extra_args[@]}") + +if [ "$dry_run" = true ]; then + printf '%q ' "${cmd[@]}" + printf '\n' + exit 0 +fi + +echo "publish.sh: syncing $local_dir -> $dest (endpoint $endpoint)" +exec "${cmd[@]}" diff --git a/run_tests.sh b/run_tests.sh new file mode 100755 index 0000000..56a8484 --- /dev/null +++ b/run_tests.sh @@ -0,0 +1,70 @@ +#!/usr/bin/env bash +# +# Runs every test in this repo and prints one passed/skipped/failed total. +# Not wired to any `.gitea/workflows` -- see the README for why, the same +# reasoning weblib-ci#9 recorded for that repo: run it by hand before +# pushing. +# +# bash run_tests.sh +# +# check_zip_alignment.py's own suite needs only python3 (via with-nixpkgs.sh, +# since none is assumed to be on PATH); everything else additionally needs +# apksigner/fdroidserver/jdk21/awscli2, resolved through the same pin. All of +# it stays on loopback/local-store -- nothing here reaches a real host, +# bucket or key. +set -uo pipefail +here=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd) + +total_pass=0 +total_skip=0 +total_fail=0 +any_hard_fail=0 + +run_python_suite() { + echo "=== test_check_zip_alignment.py ===" + local out + if out=$(bash "$here/with-nixpkgs.sh" python3 python3 "$here/test_check_zip_alignment.py" -v 2>&1); then + local n + n=$(echo "$out" | grep -oE '^Ran [0-9]+ test' | grep -oE '[0-9]+') + total_pass=$((total_pass + ${n:-0})) + echo "$out" | tail -3 + else + total_fail=$((total_fail + 1)) + any_hard_fail=1 + echo "$out" + fi + echo +} + +run_shell_suite() { + local script=$1 + echo "=== $script ===" + local out rc=0 + out=$(bash "$here/$script" 2>&1) || rc=$? + echo "$out" + local p s f + p=$(echo "$out" | grep -oE '^passed=[0-9]+' | grep -oE '[0-9]+' | tail -1) + s=$(echo "$out" | grep -oE 'skipped=[0-9]+' | grep -oE '[0-9]+' | tail -1) + f=$(echo "$out" | grep -oE 'failed=[0-9]+' | grep -oE '[0-9]+' | tail -1) + if [ -z "$p" ] && [ -z "$f" ]; then + # the script itself crashed before reaching report_and_exit + total_fail=$((total_fail + 1)) + any_hard_fail=1 + else + total_pass=$((total_pass + ${p:-0})) + total_skip=$((total_skip + ${s:-0})) + total_fail=$((total_fail + ${f:-0})) + [ "${f:-0}" -ne 0 ] && any_hard_fail=1 + fi + echo +} + +run_python_suite +run_shell_suite test_with_nixpkgs.sh +run_shell_suite test_sign_apk.sh +run_shell_suite test_fdroid_update.sh +run_shell_suite test_publish.sh + +echo "====================================" +echo "TOTAL: passed=$total_pass skipped=$total_skip failed=$total_fail" +exit "$any_hard_fail" diff --git a/sign-apk.sh b/sign-apk.sh new file mode 100755 index 0000000..c732b1c --- /dev/null +++ b/sign-apk.sh @@ -0,0 +1,117 @@ +#!/usr/bin/env bash +# +# Align-check and sign a prebuilt, unsigned release APK with a keystore the +# caller supplies. Exists because binary-repo F-Droid mode never builds +# anything itself (fdroid-repo-ci#1) -- something still has to turn a Gradle +# build's output into a signed APK before `fdroid update` can read its +# certificate. +# +# KEYSTORE_PASSWORD=... bash sign-apk.sh \ +# unsigned.apk signed.apk keystore.jks release-key KEYSTORE_PASSWORD +# +# Positional args: +# []. The last two are *names* of +# environment variables to read the passwords from, not the passwords +# themselves -- an argument shows up in the job log and in `ps`, a keystore +# password must not. If is omitted, the keystore password is +# reused as the key password, which is the common case for a keystore holding +# one key. Naming the variable rather than fixing it lets a caller with +# several keystores (an APK key here, a separate repo index key for +# fdroid-update.sh) avoid a collision without this repo inventing a +# convention for it. +# +# Any arguments after the fifth/sixth are forwarded verbatim to both +# `apksigner sign` and `apksigner verify`, e.g. `--min-sdk-version`/ +# `--max-sdk-version` to bypass real manifest parsing for a fixture that has +# no valid binary AndroidManifest.xml -- see the test suite. +# +# ## Alignment, measured rather than assumed (fdroid-repo-ci#1) +# +# `zipalign` is not a standalone nixpkgs attribute -- only `androidenv`'s +# composed build-tools carry it, and that pulls a full versioned Android SDK +# for one binary. Before adding that dependency: +# +# developer.android.com/tools/zipalign states plainly that a build made +# through the Android Gradle Plugin -- which `flutter build apk` drives +# underneath, same as a plain Android Studio build -- already aligns its zip +# entries via zipflinger, and that `zipalign` is only needed afterwards to +# *verify* that, not to perform it. `apksigner` does not check or complain +# about alignment either way: signing and verifying a deliberately misaligned +# fixture here produced no warning from `apksigner verify --verbose` at all -- +# measured while writing this script, not inferred from the docs. +# +# So this script does not depend on `zipalign`, but it does not take the AGP +# claim on faith either: `check_zip_alignment.py`, a few lines against the +# stdlib `zipfile` module, checks every *uncompressed* entry -- the only ones +# alignment affects -- before signing, and refuses to sign an APK that fails +# it, rather than silently producing one that behaves badly on a device. If +# that check ever trips on a real build, the fix is to pull in androidenv's +# zipalign, not to weaken the check. +# +# Confidence in "AGP already aligns its output" is high (Google's own docs, +# corroborated by fdroidserver 2.4.3's source never shelling out to zipalign +# anywhere at all) but NOT independently confirmed here against a real +# `flutter build apk` output -- that build was judged too expensive to run +# for this measurement (weblib-viewer#5 already declined a CI job on the same +# toolchain for the same reason, and this host had ~7.5G free against a +# multi-platform Android SDK + NDK). check_zip_alignment.py is the safety net +# for that gap: it runs against every APK this script is asked to sign, not +# just the fixtures in the test suite. +# +# Pins apksigner via with-nixpkgs.sh/nixpkgs-pin, next to this script. +set -euo pipefail + +here=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd) + +if [ "$#" -lt 5 ]; then + cat >&2 <<'USAGE' +usage: sign-apk.sh \ + [] [extra apksigner args...] + + and name environment variables holding the +passwords -- they are never passed as arguments. +USAGE + exit 2 +fi + +unsigned=$1 +signed=$2 +keystore=$3 +alias_name=$4 +ks_pass_env=$5 +shift 5 + +key_pass_env=$ks_pass_env +if [ "$#" -gt 0 ] && [[ $1 != -* ]]; then + key_pass_env=$1 + shift +fi + +extra_args=("$@") + +[ -f "$unsigned" ] || { echo "sign-apk.sh: no such file: $unsigned" >&2; exit 2; } +[ -f "$keystore" ] || { echo "sign-apk.sh: no such keystore: $keystore" >&2; exit 2; } +: "${!ks_pass_env:?sign-apk.sh: environment variable $ks_pass_env (keystore password) is not set}" +: "${!key_pass_env:?sign-apk.sh: environment variable $key_pass_env (key password) is not set}" + +echo "sign-apk.sh: checking zip alignment of $unsigned" +bash "$here/with-nixpkgs.sh" python3 python3 "$here/check_zip_alignment.py" "$unsigned" + +echo "sign-apk.sh: signing $unsigned -> $signed" +# apksigner's --ks-pass/--key-pass take "env:VARNAME", reading the variable's +# *current* value at apksigner's own invocation, not ours -- pass the names +# through so nothing beyond this script ever holds the password in an +# argument list. +bash "$here/with-nixpkgs.sh" apksigner apksigner sign \ + --ks "$keystore" \ + --ks-key-alias "$alias_name" \ + --ks-pass "env:$ks_pass_env" \ + --key-pass "env:$key_pass_env" \ + --out "$signed" \ + "${extra_args[@]}" \ + "$unsigned" + +echo "sign-apk.sh: verifying $signed" +bash "$here/with-nixpkgs.sh" apksigner apksigner verify --print-certs \ + "${extra_args[@]}" \ + "$signed" diff --git a/test_check_zip_alignment.py b/test_check_zip_alignment.py new file mode 100755 index 0000000..1e77dd6 --- /dev/null +++ b/test_check_zip_alignment.py @@ -0,0 +1,118 @@ +#!/usr/bin/env python3 +"""Offline tests for check_zip_alignment.py. + +Standard library only, no network, no nix, no external binary -- this is the +one piece of this repo that needs none of the pinned toolchain, so it is +tested at plain `python3` speed rather than through with-nixpkgs.sh. + +Builds both a genuinely aligned fixture and a deliberately misaligned one +with nothing but `zipfile`/`struct`, and runs the checker as a *subprocess* +(not an import) against both, because that is how sign-apk.sh actually calls +it -- a test that only imports the function would miss an argv or exit-code +regression. + + python3 test_check_zip_alignment.py +""" + +import os +import struct +import subprocess +import sys +import tempfile +import unittest +import zipfile + +HERE = os.path.dirname(os.path.abspath(__file__)) +CHECKER = os.path.join(HERE, "check_zip_alignment.py") + + +def add_stored(zf, name, data, align=None): + """Write a STORED entry, padded to `align` bytes if given. + + Mirrors what a real zip-aligning tool does: pad the "extra field" of the + local header so the entry's *data* (not its header) starts on the + boundary. `align=None` writes the entry with no padding at all, which is + what a plain zipfile.writestr call already does by default -- the + negative-control case that must fail the check. + """ + info = zipfile.ZipInfo(name) + info.compress_type = zipfile.ZIP_STORED + if align: + header_len = 30 + len(name.encode()) + data_start_if_no_extra = zf.fp.tell() + header_len + pad = (-data_start_if_no_extra) % align + if pad: + if pad < 4: + pad += align + extra_len = pad - 4 + info.extra = struct.pack("/dev/null 2>&1 +assert_file_exists "throwaway index keystore was created" index.jks + +echo "signing a fixture APK with sign-apk.sh, to sit in repo/..." +bash "$here/with-nixpkgs.sh" jdk21 keytool -genkeypair \ + -keystore app.jks -storepass apppass123 -keypass apppass123 \ + -alias appkey -keyalg RSA -keysize 2048 -validity 1 \ + -dname "CN=App, OU=Test, O=Test, L=Test, S=Test, C=US" >/dev/null 2>&1 +bash "$here/with-nixpkgs.sh" python3 python3 - "$work" <<'PYEOF' +import struct, sys, zipfile + +def add_stored(zf, name, data, align=4): + info = zipfile.ZipInfo(name) + info.compress_type = zipfile.ZIP_STORED + header_len = 30 + len(name.encode()) + start = zf.fp.tell() + header_len + pad = (-start) % align + if pad: + if pad < 4: + pad += align + extra_len = pad - 4 + info.extra = struct.pack("/dev/null +assert_file_exists "fixture APK signed and placed in repo/" repo/com.example.testapp_1.apk + +echo "--- fault: no repo/ directory is refused ---" +assert_failure "fdroid-update.sh refuses a root with no repo/" \ + bash "$here/fdroid-update.sh" "$work/does-not-have-repo" index.jks indexkey IDX_KS_PASS + +echo "--- fault: missing password env var is refused ---" +assert_failure "fdroid-update.sh refuses when the password env var is unset" \ + env -u IDX_KS_PASS bash "$here/fdroid-update.sh" "$work" index.jks indexkey IDX_KS_PASS + +echo "--- revert: a real run, needs no network ---" +export IDX_KS_PASS=idxpass123 +assert_success "fdroid-update.sh runs fdroid update successfully" \ + bash "$here/fdroid-update.sh" "$work" index.jks indexkey IDX_KS_PASS + +assert_file_exists "index-v2.json exists and is non-empty" repo/index-v2.json +assert_file_exists "index-v1.jar (signed index) exists and is non-empty" repo/index-v1.jar +assert_file_exists "entry.jar (signed entry, apksigner v1) exists and is non-empty" repo/entry.jar + +echo "--- index-v2.json is valid JSON, not just a non-empty file ---" +assert_success "index-v2.json parses as JSON" \ + bash "$here/with-nixpkgs.sh" python3 python3 -c "import json; json.load(open('repo/index-v2.json'))" + +echo "--- index-v1.jar is a validly signed jar (its own signature, not apksigner's) ---" +assert_success "jarsigner -verify accepts the signed index" \ + bash "$here/with-nixpkgs.sh" jdk21 jarsigner -verify repo/index-v1.jar + +echo "--- config.yml never holds a literal password ---" +assert_failure "config.yml has no literal password (uses {env: ...} instead)" \ + grep -q "idxpass123" "$work/config.yml" + +report_and_exit diff --git a/test_lib.sh b/test_lib.sh new file mode 100644 index 0000000..43cfe29 --- /dev/null +++ b/test_lib.sh @@ -0,0 +1,66 @@ +# Shared assertion helpers for the shell test scripts. Sourced, not run. +# +# Each test_*.sh increments PASS_COUNT/FAIL_COUNT/SKIP_COUNT as it goes and +# prints one line per check, so a caller sees passed/skipped/failed as +# separate numbers rather than a single pass/fail verdict -- a skip must +# never read as a pass. + +PASS_COUNT=0 +FAIL_COUNT=0 +SKIP_COUNT=0 + +pass() { PASS_COUNT=$((PASS_COUNT + 1)); echo "PASS: $1"; } +fail() { FAIL_COUNT=$((FAIL_COUNT + 1)); echo "FAIL: $1" >&2; } +skip() { SKIP_COUNT=$((SKIP_COUNT + 1)); echo "SKIP: $1"; } + +assert_success() { + # assert_success