Files
fdroid-repo-ci/README.md
T
claudeandbit 19e55c4a53 publish.sh: verify credentials and upload the index last; add next-version-code.sh
Closes #5
Closes #6

An F-Droid repo is a signed index plus the files it names, and a partial
upload could previously leave them disagreeing: aws s3 sync walks the tree
in its own order, so an interrupted publish could land the index before the
APKs it names, breaking the repo for every client until the upload finished
or was rolled back. publish.sh now uploads in two passes, unconditionally --
every non-index file first, the index (index-v1.json(.asc), index-v2.json,
entry.json, entry.jar) last -- so an interruption anywhere in pass 1 simply
leaves the repo describing the old state a little longer, never a mixture.
Credentials are also verified with one real authenticated call
(head-bucket) before either pass, rather than merely checking the
environment variables are non-empty, since an unusable-but-present
credential failing halfway through is exactly how a partial upload happens.

next-version-code.sh is the one implementation of the versionCode rule
(candidate = minutes since 2026-01-01Z, clamped to strictly exceed whatever
is already published for that applicationId), shared by CI and manual
builds so there is exactly one place this can drift. bit pointed out
(appended to #6) that the clamp is monotonic against *observed* state, not
monotonic by construction: two jobs starting in the same minute can both
read the same published value and race to publish the same versionCode on
different APKs. Nothing can prevent that wasted build, so publish.sh now
also refuses to overwrite an already-existing APK object before pass 1 --
turning the collision from a silent, permanent wrong state into a loud
failure at the last moment the information is reliable.

Tested offline (83 assertions: pure-stdlib suites for both scripts, the
real CLIs through with-nixpkgs.sh, and a small local HEAD-only S3 stub
proving the credential check is a real authenticated call and the
APK-collision refusal fires only when it should) and live against
fdroid-weblib-dev (7 more: a real publish observed mid-flight via a
same-process synchronization hook to prove the index is genuinely absent
while the file is already present, both objects reading back
byte-identical, and a second publish refused against a real
already-published APK with its bytes left unchanged).

Co-authored-by: bit <bit@das-labor.org>
2026-09-09 08:00:45 +00:00

18 KiB

fdroid-repo-ci

Shared tooling to sign a prebuilt APK, build a signed F-Droid repo index over it, and publish the result to an S3-compatible bucket -- consumed as a composite action by whichever repository currently holds the signing keys and runs the publish. Which repository that is has changed more than once, so nothing here names it; see "Not hard-coded" below.

Same shape as weblib-ci, and public for the same measured reason: a private repository cannot be consumed cross-repo via uses: on this Gitea instance (see weblib-ci's README) -- every cross-repo form fails while the repo being referenced is private, and only the full-URL form works once it is public.

Nothing secret belongs here, ever. Keys, keystores, credentials, bucket names and hostnames all live in the private repo that consumes this one and arrive as inputs -- arguments and environment variables, never committed here.

What is here

script what it does
next-version-code.sh print the next monotonic versionCode for an application id, read from a repo index -- called before the build, CI and a manual build alike (see "One versionCode rule" below)
sign-apk.sh check alignment, then sign a prebuilt unsigned release APK with a keystore the caller supplies
fdroid-update.sh run fdroid update over a repo/ directory of already-signed APKs and produce the signed index
publish.sh upload the resulting repo tree to an S3-compatible bucket -- files first, index last, and refuses to overwrite an already-published APK (see "Publish ordering and safety" below)
with-nixpkgs.sh run a command with one or more nixpkgs packages on PATH, pinned to the revision in nixpkgs-pin
check_zip_alignment.py the alignment check sign-apk.sh runs before signing (see "Align before sign" below)
next_version_code.py the stdlib-only implementation next-version-code.sh wraps (index read, arithmetic)
action.yml the composite action a consumer includes to get all of the above

Deliberately not in scope: fetching the APK. That needs an upstream credential and is the consumer's business, not the tooling's -- which also keeps this repo from knowing which upstreams exist.

All of it is standard library / plain bash plus the pinned tools below. They are run, not built, so this repo has no flake -- see "Pinning nixpkgs" below for what that costs and why it is still the right call here.

Using it

- uses: actions/checkout@v4
- id: ci
  uses: https://git.chaosbit.de/weblib/fdroid-repo-ci@main

- run: |
    version_code=$(bash "${{ steps.ci.outputs.path }}/next-version-code.sh" \
      "$FDROID_REPO_INDEX_URL" "$APPLICATION_ID")
    echo "version_code=$version_code" >> "$GITHUB_ENV"
    # ... feed $version_code into whatever builds app-release-unsigned.apk

- env:
    APK_KEYSTORE_PASSWORD: ${{ secrets.APK_KEYSTORE_PASSWORD }}
  run: |
    bash "${{ steps.ci.outputs.path }}/sign-apk.sh" \
      app-release-unsigned.apk app-release.apk \
      "$APK_KEYSTORE" release-key APK_KEYSTORE_PASSWORD

- env:
    INDEX_KEYSTORE_PASSWORD: ${{ secrets.INDEX_KEYSTORE_PASSWORD }}
  run: |
    bash "${{ steps.ci.outputs.path }}/fdroid-update.sh" \
      "$FDROID_REPO_ROOT" "$INDEX_KEYSTORE" repo-key INDEX_KEYSTORE_PASSWORD

- env:
    AWS_ACCESS_KEY_ID: ${{ secrets.PUBLISH_ACCESS_KEY_ID }}
    AWS_SECRET_ACCESS_KEY: ${{ secrets.PUBLISH_SECRET_ACCESS_KEY }}
  run: |
    bash "${{ steps.ci.outputs.path }}/publish.sh" \
      "$FDROID_REPO_ROOT/repo" "$BUCKET" "$S3_ENDPOINT_URL"

Use the full URL, https://git.chaosbit.de/weblib/fdroid-repo-ci@main, not weblib/fdroid-repo-ci@main. Measured on weblib-ci (weblib-archive#44): the bare owner/repo form resolves against the instance's default actions URL rather than this host and fails outright. The full-URL form is also the only one that works while a referenced repo is private -- irrelevant for this repo, which is public on purpose, but relevant for whatever repo is consuming it, since that one holds the keys.

$APK_KEYSTORE/$INDEX_KEYSTORE/$FDROID_REPO_ROOT/$BUCKET/ $S3_ENDPOINT_URL/$FDROID_REPO_INDEX_URL/$APPLICATION_ID above are the consumer's own paths and values -- nothing in this repo defaults or hints at them. Env var names like APK_KEYSTORE_PASSWORD are the consumer's choice too, not a convention this repo imposes: sign-apk.sh and fdroid-update.sh each take the name of the environment variable to read a password from as an argument, specifically so two different keystores in the same job (the APK key and the separate repo index key -- see below) never have to share one variable name.

One versionCode rule, shared by CI and a manual build (fdroid-repo-ci#6)

Two implementations of "the next version code" is the exact failure mode next-version-code.sh exists to avoid, so both CI and a manual build call this one script rather than each computing their own:

candidate  = whole minutes since 2026-01-01T00:00:00Z
published  = highest versionCode already in the repo index for this applicationId
result     = max(candidate, published + 1)

Minutes, not seconds or raw Unix time: versionCode is a signed 32-bit int, and this scheme does not exhaust it for 4,082 years, against 67 years for seconds and 11 for Unix time. The clamp against published exists because minute granularity means two builds started in the same minute would otherwise compute the same number -- and it also covers a clock stepping backwards or an old commit being rebuilt, both of which would otherwise produce a number lower than one a device has already seen.

The clamp is monotonic against observed state, not monotonic by construction. It does not close the race where two jobs start in the same minute, both read the same published value, both compute the same next versionCode, and both build and publish -- the window is the whole build, not just the clock read, since the version code has to be baked into the APK before it exists. Nothing in next-version-code.sh can prevent that. publish.sh's APK-existence check (below) is the other half of this: it cannot stop the wasted build, but it turns the collision from a silent, permanent wrong state into a loud failure at the last moment, when the information is finally reliable.

Takes an index-v2.json URL or local file path, and an application id; prints the integer and nothing else, so a caller does versionCode=$(next-version-code.sh "$URL" "$APPLICATION_ID"). The URL form needs no credentials, since both this repo and the F-Droid repo it reads are public -- the point of weblib-viewer#16's "buildable when CI is down". Fails loudly (non-zero exit, nothing on stdout) if the index cannot be read; an application id with no entry at all in the index is not an error -- that is simply the first-ever publish for a channel, and yields the bare candidate.

Publish ordering and safety (fdroid-repo-ci#5, fdroid-repo-ci#6)

An F-Droid repo is a signed index plus the files it names. publish.sh uploads in two passes, unconditionally: every non-index file first, the index (index-v1.json(.asc), index-v2.json, entry.json, entry.jar, matched only at the top level of the tree) last. A publish interrupted anywhere in the first pass leaves the repo describing the old state a little longer; it can never leave an index naming files that 404. Credentials are verified with a real authenticated call (head-bucket) before either pass, rather than merely checking that the environment variables are non-empty -- an unusable-but-present credential failing halfway through is exactly how a partial (and potentially index-first) upload happens.

Before pass 1, publish.sh also checks every local *.apk's destination key with head-object and refuses the whole publish if any of them already exists -- the safeguard next-version-code.sh's clamp cannot provide on its own (see above). This does not prevent the wasted build behind a same-minute collision; it prevents the collision from silently corrupting the repo.

Align before sign, and how "no zipalign" was verified

zipalign then apksigner. Signing first and aligning after is the old jarsigner-era order and invalidates an apksigner-era (v2/v3) signature, because that scheme signs the exact bytes of the packaged zip and alignment padding changes those bytes.

apksigner is a standalone nixpkgs attribute (nixpkgs#apksigner, verified at 35.0.6). zipalign is not -- it exists only inside androidenv's composed build-tools, which pulls a full versioned Android SDK and its licence machinery for one binary. Before accepting that as a dependency, three things were measured rather than assumed:

  1. Google's own docs (developer.android.com/tools/zipalign): "If you build using Android Studio, which uses the Android Gradle plugin (AGP), this is done automatically. In this case, you should still use zipalign to verify that the APK is aligned, but you don't need to align it." AGP is what flutter build apk (or any Gradle-based Android build) drives underneath, so this applies to any such build's output, not just Android-Studio-initiated ones.
  2. fdroidserver 2.4.3's own source never calls zipalign, anywhere -- grepped across all 38 of its modules. Its APK-signature-copying code (apksigcopier.py, used for reproducible-build verification, a different flow than binary-repo mode) implements realignment itself in pure Python against the zip format directly, rather than shelling out to the zipalign binary.
  3. apksigner does not check or enforce alignment at all, in either direction -- measured directly: a deliberately misaligned fixture zip (including a .so misaligned to the stricter 4096-byte boundary) was signed and then verified with apksigner verify --verbose, and the output carried no warning about alignment whatsoever.

Confidence: high that a Gradle/AGP-built unsigned release APK is already aligned, resting on (1) and (2) above. Not independently confirmed against a real flutter build apk output run in this environment -- that build was judged too expensive to attempt for this measurement. A sibling repo's own CI proposal for building the Flutter app was rejected on the same "the toolchain is too large to pull on every push" grounds this pin sidesteps (composing a multi-platform Android SDK plus NDK pulls tens of gigabytes), and the host available for this measurement had well under that in free disk space.

Point (3) is why this repo does not simply trust point (1) and move on: since nothing downstream complains about misalignment either, a build that stopped emitting aligned output would fail silently -- an APK that signs and verifies fine but behaves badly (or refuses to install) on a real device. check_zip_alignment.py, a few lines against the standard library zipfile module, checks every uncompressed entry of the APK sign-apk.sh is asked to sign, and refuses to sign one that fails it. If that check ever legitimately trips, the fix is to add androidenv's zipalign as a dependency here, not to weaken or remove the check.

The repo index key is not the APK signing key

Binary-repo mode (every APK arrives already signed) means fdroid update reads each APK's existing signature and never touches an app's own signing key. It signs the index with a separate key of its own -- fdroid-update.sh takes its own keystore/alias/password arguments, deliberately unrelated to sign-apk.sh's. Conflating the two would mean one compromised key undermines both the apps and the index that vouches for them.

fdroid-update.sh writes <repo-root>/config.yml itself, on every run, using fdroidserver's own {env: VARNAME} indirection for both passwords -- the exact mechanism its common.py implements specifically so config.yml can be safely git-managed: "it is a good idea to manage config.yml (WITHOUT PASSWORDS!) in git." A caller never has to know or hand-write fdroidserver's config format.

Pinning nixpkgs

fdroidserver's output feeds a signed index, so an unpinned or channel-drifting nixpkgs is a worse trade here than for a disposable test job. This repo carries a nixpkgs-pin file -- one line, a 40-character nixpkgs revision -- and with-nixpkgs.sh refuses to run at all if that file is missing or malformed, rather than silently falling back to an unpinned nix shell nixpkgs#foo, which re-resolves the registry and fetches whatever the channel currently points at.

This is deliberately not the same mechanism as weblib-ci's with-nixpkgs.sh, which reads the consuming repo's flake.lock. This repo has no flake (see weblib-ci#9 for the same tension in that repo: a flake.lock existing only to hold one revision string), and per this repo's own originating issue, the repo that consumes it holds signing keys and is not guaranteed to be a flake either -- reading a caller's flake.lock the way weblib-ci's script does would not even find one here. So the pin travels with this repo instead, in a file named nixpkgs-pin on purpose: it is the exact fallback weblib-ci#9 proposes adding to the shared script (flake.lock present -> today's behaviour there; else nixpkgs-pin -> a bare revision; else fail loudly). If that lands, this repo already speaks its format and nothing here needs to change.

A nixpkgs-pin file has no nix flake update -- it is bumped by hand. Accepted here for the same reason weblib-ci#9 accepted it for a stdlib-only test suite, generalised: the alternative (a flake that builds nothing, just to hold a lockfile) is a lot of structure for one string, and a genuinely broken pinned nixpkgs would be caught immediately by this repo's own test suite.

fdroidserver is in nixpkgs as fdroidserver, verified at 2.4.3. publish.sh uses awscli2 for the same S3-compatible upload every consumer here needs, resolved through the same pin.

Testing

bash run_tests.sh

83 assertions across seven suites offline (~20s on a warm nix store), plus 5 more that only run against a real bucket (see below):

  • test_check_zip_alignment.py -- pure standard library, no nix, no network at all; builds an aligned and a misaligned fixture zip by hand and checks both directions, including the 4096-byte boundary specific to .so entries.
  • test_next_version_code.py -- pure standard library too, testing next_version_code.py as a subprocess against fixture index-v2.json files written to a temp dir: no prior build, a lower/equal/higher prior (the clock-went-backwards case), several versions at once, a version entry with no versionCode, an application id missing from the index entirely (not an error), and an unreadable/malformed index (fails loudly, nothing on stdout). "now" is pinned via an undocumented env var seam so the expected candidate is a fixed number.
  • test_with_nixpkgs.sh -- offline; every way the pinning wrapper is supposed to refuse rather than silently degrade (missing pin, malformed pin, too few arguments). Does not exercise the happy path of actually resolving a github:nixos/nixpkgs/... reference automatically, since that needs network (or an already-warm local cache) the same way weblib-ci's own with-nixpkgs.sh does, and that repo does not automate a test for it either. It was checked by hand instead.
  • test_sign_apk.sh and test_fdroid_update.sh -- full end-to-end runs against a throwaway keystore generated fresh per run, real apksigner and fdroid update. Needs the pinned toolchain to be resolvable (locally cached, or fetched once over the network); nothing beyond that.
  • test_next_version_code.sh -- the real next-version-code.sh CLI, through with-nixpkgs.sh: argument count, a local index file end to end, a missing application id, a missing/malformed local file, and an unreachable URL (http://127.0.0.1:1, the same closed-port trick test_publish.sh uses) -- proving the wrapper's argv/exit-code/stdout contract, not just the Python module's arithmetic.
  • test_publish.sh -- argument/credential validation, --dry-run command construction (now two commands, files then index, and asserted in that order), and a real invocation against a closed local port (127.0.0.1:1), fully offline. Also runs test_s3_stub.py, a same-repo, HEAD-only local HTTP stub, to prove the credential check is a real authenticated call (a wrong bucket is rejected) and that an already-published APK is refused before any upload, with a negative control proving a non-colliding APK does not trip the same check. The actual upload path -- and the two-pass ordering of real uploads, which nothing offline can see -- is exercised only by the opt-in live round trip below, matching the constraint that only publish.sh needs network in the first place.

test_publish.sh's live section (skipped by default; needs FDROID_CI_LIVE_BUCKET, FDROID_CI_LIVE_ENDPOINT and real credentials) adds 5 more assertions against a real bucket: the round trip itself; the index object provably absent while the file object is already present at the one point in the middle of a real publish that the ordering guarantee depends on (observed via a same-process synchronization hook publish.sh exposes only for this, PUBLISH_SH_TEST_HOOK_BETWEEN_PASSES -- deliberately not a poll racing a background upload from outside, since that is exactly the flakiness fdroid-repo-ci#5 warned against); both objects reading back byte-identical; a second publish refused when it would overwrite a real, already-published APK; and that APK's bytes unchanged afterwards.

No .gitea/workflows runs this, on the same reasoning weblib-ci#9 recorded: run it by hand before pushing. If that changes for weblib-ci it should probably change here too, at the same time.

Not hard-coded

No bucket name, repo URL, app id, keystore path, or channel name or count -- a consumer may have one publish target or several, under whatever names it likes; nothing here assumes "release" and "dev", or that there are exactly two. No consumer is named anywhere either: the repository that runs the publish has already changed twice during design, so naming one would be wrong again shortly.

The one exception is deliberate and unavoidable: this repository's own address appears in the uses: examples above, because that is what a consumer has to type. Those lines are what will need editing if this repo moves organisation -- nothing else should.