fdroid-repo-ci/README.md

# fdroid-repo-ci

Shared tooling to sign a prebuilt APK, build a signed F-Droid repo index over
it, and publish the result to an S3-compatible bucket -- consumed as a
composite action by whichever repository currently holds the signing keys
and runs the publish. Which repository that is has changed more than once,
so nothing here names it; see "Not hard-coded" below.

Same shape as [weblib-ci], and public for the same measured reason: a
private repository cannot be consumed cross-repo via `uses:` on this Gitea
instance (see weblib-ci's README) -- every cross-repo form fails while the
repo being referenced is private, and only the full-URL form works once it
is public.

**Nothing secret belongs here, ever.** Keys, keystores, credentials, bucket
names and hostnames all live in the private repo that consumes this one and
arrive as inputs -- arguments and environment variables, never committed
here.

## What is here

| script | what it does |
|---|---|
| `next-version-code.sh` | print the next monotonic `versionCode` for an application id and a commit's committer date, read from a repo index -- called before the build, CI and a manual build alike (see "One versionCode rule" below) |
| `sign-apk.sh` | check alignment, then sign a prebuilt unsigned release APK with a keystore the caller supplies |
| `fdroid-update.sh` | run `fdroid update` over a `repo/` directory of already-signed APKs and produce the signed index |
| `publish.sh` | upload the resulting repo tree to an S3-compatible bucket -- files first, index last, and refuses to overwrite an already-published APK (see "Publish ordering and safety" below) |
| `with-nixpkgs.sh` | run a command with one or more nixpkgs packages on PATH, pinned to the revision in `nixpkgs-pin` |
| `check_zip_alignment.py` | the alignment check `sign-apk.sh` runs before signing (see "Align before sign" below) |
| `next_version_code.py` | the stdlib-only implementation `next-version-code.sh` wraps (index read, arithmetic) |
| `action.yml` | the composite action a consumer includes to get all of the above |

Deliberately **not** in scope: fetching the APK. That needs an upstream
credential and is the consumer's business, not the tooling's -- which also
keeps this repo from knowing which upstreams exist.

All of it is standard library / plain bash plus the pinned tools below. They
are *run*, not built, so this repo has no flake -- see "Pinning nixpkgs"
below for what that costs and why it is still the right call here.

## Using it

```yaml
- uses: actions/checkout@v4
- id: ci
  uses: https://git.chaosbit.de/weblib/fdroid-repo-ci@main

- run: |
    version_code=$(bash "${{ steps.ci.outputs.path }}/next-version-code.sh" \
      "$FDROID_REPO_INDEX_URL" "$APPLICATION_ID" "$(git log -1 --format=%ct)")
    echo "version_code=$version_code" >> "$GITHUB_ENV"
    # ... feed $version_code into whatever builds app-release-unsigned.apk

- env:
    APK_KEYSTORE_PASSWORD: ${{ secrets.APK_KEYSTORE_PASSWORD }}
  run: |
    bash "${{ steps.ci.outputs.path }}/sign-apk.sh" \
      app-release-unsigned.apk app-release.apk \
      "$APK_KEYSTORE" release-key APK_KEYSTORE_PASSWORD

- env:
    INDEX_KEYSTORE_PASSWORD: ${{ secrets.INDEX_KEYSTORE_PASSWORD }}
  run: |
    bash "${{ steps.ci.outputs.path }}/fdroid-update.sh" \
      "$FDROID_REPO_ROOT" "$INDEX_KEYSTORE" repo-key INDEX_KEYSTORE_PASSWORD

- env:
    AWS_ACCESS_KEY_ID: ${{ secrets.PUBLISH_ACCESS_KEY_ID }}
    AWS_SECRET_ACCESS_KEY: ${{ secrets.PUBLISH_SECRET_ACCESS_KEY }}
  run: |
    bash "${{ steps.ci.outputs.path }}/publish.sh" \
      "$FDROID_REPO_ROOT/repo" "$BUCKET" "$S3_ENDPOINT_URL"
```

**Use the full URL, `https://git.chaosbit.de/weblib/fdroid-repo-ci@main`, not
`weblib/fdroid-repo-ci@main`.** Measured on weblib-ci (weblib-archive#44):
the bare `owner/repo` form resolves against the instance's default actions
URL rather than this host and fails outright. The full-URL form is also the
only one that works while a referenced repo is private -- irrelevant for
*this* repo, which is public on purpose, but relevant for whatever repo is
consuming it, since that one holds the keys.

`$APK_KEYSTORE`/`$INDEX_KEYSTORE`/`$FDROID_REPO_ROOT`/`$BUCKET`/
`$S3_ENDPOINT_URL`/`$FDROID_REPO_INDEX_URL`/`$APPLICATION_ID` above are the
consumer's own paths and values -- nothing in this repo defaults or hints at
them. Env var *names* like `APK_KEYSTORE_PASSWORD` are the consumer's choice
too, not a convention this repo imposes: `sign-apk.sh` and `fdroid-update.sh`
each take the name of the environment variable to read a password from as an
argument, specifically so two different keystores in the same job (the APK
key and the separate repo index key -- see below) never have to share one
variable name.

## One `versionCode` rule, shared by CI and a manual build (fdroid-repo-ci#6)

Two implementations of "the next version code" is the exact failure mode
`next-version-code.sh` exists to avoid, so both CI and a manual build call
this one script rather than each computing their own. The candidate is
derived from the **committer date of the commit being built**, not the
build clock -- deterministic per commit, rather than depending on when a
runner happened to pick the job up -- and clamped on **both** sides:

```
candidate  = whole minutes since 2026-01-01T00:00:00Z, of the commit's committer date
candidate  = min(candidate, minutes(now) + TOLERANCE)             # forwards
published  = highest versionCode already in the repo index for this applicationId
result     = max(candidate, published + 1)                        # backwards
```

Minutes, not seconds or raw Unix time: `versionCode` is a signed 32-bit int,
and this scheme does not exhaust it for 4,082 years, against 67 years for
seconds and 11 for Unix time. Granularity is no longer what keeps
same-commit rebuilds from colliding, though -- that is a concurrency-group
plus refuse-on-existing-APK problem now (see below) -- so minutes stand
purely as a wide, cheap headroom choice, not a load-bearing part of the
concurrency story.

**Backwards.** Rebase, amend and cherry-pick can all produce a commit older
than one already published. The rule uses the commit's **committer** date,
not its author date, specifically because of this: author date survives a
rebase unchanged, so a rebased branch would keep re-emitting its original
(now stale) timestamp and the backwards clamp would never see it move.
Committer date changes on every rewrite, which is what lets `published + 1`
actually catch it.

**Forwards.** A commit's timestamp is attacker- and typo-controlled, and the
number this rule produces only ever rises -- so a bad one can never be
walked back down. A commit dated a century ahead burns decades of headroom
permanently; worse, a value placed just *below* the int32 ceiling is still
accepted by Android, still publishes, and then **no later build can ever
exceed it** -- the app becomes permanently un-updatable, recoverable only by
shipping a new `applicationId` and asking every user to reinstall by hand.
`TOLERANCE` only has to absorb honest clock skew between the committer's
machine and the runner -- a few hours is generous, a day would still be
harmless, since its only job is keeping the number near the present. A
commit that lands beyond it is **clamped and a warning is printed, never
rejected outright**: rejecting would block a legitimate build over a few
minutes of innocent skew, and the clamp already removes the actual harm (a
permanently reserved version-code range), so a hard failure would buy
nothing further.

**The clamp is monotonic against *observed* state, not monotonic by
construction.** It does not close the race where two jobs start in the same
minute, both read the same `published` value, both compute the same next
versionCode, and both build and publish -- the window is the whole build,
not just the clock read, since the version code has to be baked into the APK
before it exists. Nothing in `next-version-code.sh` can prevent that.
`publish.sh`'s APK-existence check (below) is the other half of this: it
cannot stop the wasted build, but it turns the collision from a silent,
permanent wrong state into a loud failure at the last moment, when the
information is finally reliable. Concurrent same-commit builds are now also
addressed at the workflow level with a concurrency group
(cancel-in-progress) -- measured working on this Gitea instance -- which is
an optimization on top of the refusal, not a replacement for it.

Takes an index-v2.json URL or local file path, an application id, and the
committer timestamp of the commit being built as Unix seconds (exactly what
`git log -1 --format=%ct` prints); prints the integer and nothing else, so a
caller does
`versionCode=$(next-version-code.sh "$URL" "$APPLICATION_ID" "$(git log -1 --format=%ct)")`.
The script takes the timestamp as an argument rather than shelling out to
git itself, which keeps it pure and testable -- the caller already knows
its own commit. The URL form needs no credentials, since both this repo and
the F-Droid repo it reads are public -- the point of weblib-viewer#16's
"buildable when CI is down". Fails loudly (non-zero exit, nothing on
stdout) if the index cannot be read; an application id with no entry at all
in the index is not an error -- that is simply the first-ever publish for a
channel, and yields the bare (clamped) candidate.

## Publish ordering and safety (fdroid-repo-ci#5, fdroid-repo-ci#6)

An F-Droid repo is a signed index plus the files it names. `publish.sh`
uploads in two passes, unconditionally: every non-index file first, the
index (`index-v1.json(.asc)`, `index-v2.json`, `entry.json`, `entry.jar`,
matched only at the top level of the tree) last. A publish interrupted
anywhere in the first pass leaves the repo describing the old state a little
longer; it can never leave an index naming files that 404. Credentials are
verified with a real authenticated call (`head-bucket`) before either pass,
rather than merely checking that the environment variables are non-empty --
an unusable-but-present credential failing halfway through is exactly how a
partial (and potentially index-first) upload happens.

Before pass 1, `publish.sh` also checks every local `*.apk`'s destination
key with `head-object` and refuses the whole publish if any of them already
exists -- the safeguard `next-version-code.sh`'s clamp cannot provide on its
own (see above). This does not prevent the wasted build behind a same-minute
collision; it prevents the collision from silently corrupting the repo.
Skipped entirely for a plain `--index-only` invocation (below), since that
pass can never upload a `*.apk` in the first place.

### `--files-only` and `--index-only`

The two passes can also be run as two separate invocations:

```bash
bash publish.sh --files-only "$FDROID_REPO_ROOT/repo" "$BUCKET" "$S3_ENDPOINT_URL"
# ... build finishes, or a later job resumes here ...
bash publish.sh --index-only "$FDROID_REPO_ROOT/repo" "$BUCKET" "$S3_ENDPOINT_URL"
```

With neither flag, both passes run in one call, files then index -- the
default, and the only way to reach "index before files" is to invoke
`--index-only` on its own, which is opt-in misuse, never the default. This
is real, independently useful behaviour, not just a testing seam: an
interrupted publish can resume at the index step without re-uploading every
APK, and a caller can stage files now and publish the index later.

It is also what makes the ordering guarantee testable **by construction**:
run `--files-only`, assert the bucket has the file and not the index yet,
run `--index-only`, assert the index then appears -- two genuinely separate
invocations, with the bucket inspected for real in between. An earlier
version of this script proved the same property with a same-process
synchronization hook that `eval`'d an environment variable between the two
passes; that was removed in favour of these flags; a script that runs with
a release signing key and S3 credentials in its environment, in a public
repository, should not carry a documented path from env-var influence to
arbitrary code execution, however unlikely, when a real feature closes the
same gap for free.

## Align before sign, and how "no `zipalign`" was verified

`zipalign` then `apksigner`. Signing first and aligning after is the old
`jarsigner`-era order and invalidates an `apksigner`-era (v2/v3) signature,
because that scheme signs the exact bytes of the packaged zip and alignment
padding changes those bytes.

**`apksigner` is a standalone nixpkgs attribute** (`nixpkgs#apksigner`,
verified at 35.0.6). **`zipalign` is not** -- it exists only inside
`androidenv`'s composed build-tools, which pulls a full versioned Android SDK
and its licence machinery for one binary. Before accepting that as a
dependency, three things were measured rather than assumed:

1. **Google's own docs** (developer.android.com/tools/zipalign): "If you
   build using Android Studio, which uses the Android Gradle plugin (AGP),
   this is done automatically. In this case, you should still use zipalign
   to verify that the APK is aligned, but you don't need to align it." AGP is
   what `flutter build apk` (or any Gradle-based Android build) drives
   underneath, so this applies to any such build's output, not just
   Android-Studio-initiated ones.
2. **`fdroidserver` 2.4.3's own source never calls `zipalign`, anywhere** --
   grepped across all 38 of its modules. Its APK-signature-copying code
   (`apksigcopier.py`, used for reproducible-build verification, a different
   flow than binary-repo mode) implements realignment itself in pure Python
   against the zip format directly, rather than shelling out to the
   `zipalign` binary.
3. **`apksigner` does not check or enforce alignment at all**, in either
   direction -- measured directly: a deliberately misaligned fixture zip
   (including a `.so` misaligned to the stricter 4096-byte boundary) was
   signed and then verified with `apksigner verify --verbose`, and the
   output carried no warning about alignment whatsoever.

**Confidence: high** that a Gradle/AGP-built unsigned release APK is already
aligned, resting on (1) and (2) above. **Not independently confirmed**
against a real `flutter build apk` output run in this environment -- that
build was judged too expensive to attempt for this measurement. A sibling
repo's own CI proposal for building the Flutter app was rejected on the same
"the toolchain is too large to pull on every push" grounds this pin sidesteps
(composing a multi-platform Android SDK plus NDK pulls tens of gigabytes),
and the host available for this measurement had well under that in free
disk space.

Point (3) is why this repo does not simply trust point (1) and move on:
since nothing downstream complains about misalignment either, a build that
stopped emitting aligned output would fail silently -- an APK that signs and
verifies fine but behaves badly (or refuses to install) on a real device.
`check_zip_alignment.py`, a few lines against the standard library `zipfile`
module, checks every uncompressed entry of the APK `sign-apk.sh` is asked to
sign, and refuses to sign one that fails it. If that check ever legitimately
trips, the fix is to add `androidenv`'s `zipalign` as a dependency here, not
to weaken or remove the check.

## The repo index key is not the APK signing key

Binary-repo mode (every APK arrives already signed) means `fdroid update`
reads each APK's existing signature and never touches an app's own signing
key. It signs the **index** with a separate key of its own -- `fdroid-update.sh`
takes its own keystore/alias/password arguments, deliberately unrelated to
`sign-apk.sh`'s. Conflating the two would mean one compromised key
undermines both the apps and the index that vouches for them.

`fdroid-update.sh` writes `<repo-root>/config.yml` itself, on every run,
using fdroidserver's own `{env: VARNAME}` indirection for both passwords --
the exact mechanism its `common.py` implements *specifically so config.yml
can be safely git-managed*: "it is a good idea to manage config.yml (WITHOUT
PASSWORDS!) in git." A caller never has to know or hand-write fdroidserver's
config format.

## Pinning nixpkgs

`fdroidserver`'s output feeds a *signed index*, so an unpinned or
channel-drifting nixpkgs is a worse trade here than for a disposable test
job. This repo carries a `nixpkgs-pin` file -- one line, a 40-character
nixpkgs revision -- and `with-nixpkgs.sh` refuses to run at all if that file
is missing or malformed, rather than silently falling back to an unpinned
`nix shell nixpkgs#foo`, which re-resolves the registry and fetches whatever
the channel currently points at.

This is deliberately **not** the same mechanism as [weblib-ci]'s
`with-nixpkgs.sh`, which reads the *consuming* repo's `flake.lock`. This repo
has no flake (see weblib-ci#9 for the same tension in that repo: a
`flake.lock` existing only to hold one revision string), and per this repo's
own originating issue, the repo that consumes it holds signing keys and is
not guaranteed to be a flake either -- reading a caller's `flake.lock` the
way weblib-ci's script does would not even find one here. So the pin travels
with *this* repo instead, in a file named `nixpkgs-pin` on purpose: it is the
exact fallback weblib-ci#9 proposes adding to the shared script (flake.lock
present -> today's behaviour there; else `nixpkgs-pin` -> a bare revision;
else fail loudly). If that lands, this repo already speaks its format and
nothing here needs to change.

A `nixpkgs-pin` file has no `nix flake update` -- it is bumped by hand.
Accepted here for the same reason weblib-ci#9 accepted it for a
stdlib-only test suite, generalised: the alternative (a flake that builds
nothing, just to hold a lockfile) is a lot of structure for one string, and a
genuinely broken pinned nixpkgs would be caught immediately by this repo's
own test suite.

`fdroidserver` is in nixpkgs as `fdroidserver`, verified at 2.4.3.
`publish.sh` uses `awscli2` for the same S3-compatible upload every consumer
here needs, resolved through the same pin.

## Testing

```bash
bash run_tests.sh
```

98 assertions across seven suites offline (~20s on a warm nix store, 4
deliberately skipped -- see below), plus 13 more that only run against a
real bucket:

* `test_check_zip_alignment.py` -- pure standard library, no nix, no
  network at all; builds an aligned and a misaligned fixture zip by hand and
  checks both directions, including the 4096-byte boundary specific to `.so`
  entries.
* `test_next_version_code.py` -- pure standard library too, testing
  `next_version_code.py` as a subprocess against fixture index-v2.json files
  written to a temp dir: a normal commit yields its own minute, a commit
  older than the published maximum still yields published+1 (backwards --
  rebase/amend/cherry-pick), a future-dated commit within tolerance passes
  through untouched, one beyond tolerance is clamped to roughly now *and
  warns* (forwards -- a bad clock or a malicious/typo'd date), a commit
  dated near the int32 ceiling (year 2999) does not produce a near-int32
  result, several published versions at once, a version entry with no
  versionCode, an application id missing from the index entirely (not an
  error), and an unreadable/malformed index (fails loudly, nothing on
  stdout). A dedicated property test also checks, across every one of those
  scenarios in one place, that the result always strictly exceeds every
  versionCode already in the fixture -- the property that actually matters,
  which a future refactor could break while each individual case still
  looks right. Both the commit's committer date and "now" (used only for
  the forwards clamp) are pinned independently, the latter via an
  undocumented env var seam, so every expected number is fixed rather than
  depending on whatever minute the suite happens to run in.
* `test_with_nixpkgs.sh` -- offline; every way the pinning wrapper is
  supposed to refuse rather than silently degrade (missing pin, malformed
  pin, too few arguments). Does not exercise the happy path of actually
  resolving a `github:nixos/nixpkgs/...` reference automatically, since that
  needs network (or an already-warm local cache) the same way weblib-ci's
  own `with-nixpkgs.sh` does, and that repo does not automate a test for it
  either. It was checked by hand instead.
* `test_sign_apk.sh` and `test_fdroid_update.sh` -- full end-to-end runs
  against a throwaway keystore generated fresh per run, real `apksigner`
  and `fdroid update`. Needs the pinned toolchain to be resolvable (locally
  cached, or fetched once over the network); nothing beyond that.
* `test_next_version_code.sh` -- the real `next-version-code.sh` CLI, through
  `with-nixpkgs.sh`: argument count (now three: index, application id,
  committer timestamp), a local index file end to end, a first-ever publish
  for an application id, a far-future committer date clamped with a warning
  on stderr, a non-numeric committer timestamp refused rather than coerced,
  a missing/malformed local file, and an unreachable URL
  (`http://127.0.0.1:1`, the same closed-port trick `test_publish.sh` uses)
  -- proving the wrapper's argv/exit-code/stdout contract, not just the
  Python module's arithmetic.
* `test_publish.sh` -- argument/credential validation, `--dry-run` command
  construction (both the default two-command form, files then index, and
  each of `--files-only`/`--index-only` printing exactly one command, plus
  their mutual exclusivity), and a real invocation against a closed local
  port (`127.0.0.1:1`), fully offline. Also runs `test_s3_stub.py`, a
  same-repo, HEAD-only local HTTP stub, to prove the credential check is a
  real authenticated call (a wrong bucket is rejected), that an
  already-published APK is refused before any upload, that `--index-only`
  alone never runs that check at all (it cannot upload an APK), and a
  negative control proving a non-colliding APK does not trip the check. The
  actual upload path -- and the two-pass *ordering* of real uploads, which
  nothing offline can see -- is exercised only by the opt-in live round trip
  below, matching the constraint that only `publish.sh` needs network in the
  first place.

`test_publish.sh`'s live section (skipped by default; needs
`FDROID_CI_LIVE_BUCKET`, `FDROID_CI_LIVE_ENDPOINT` and real credentials) adds
13 more assertions against a real bucket:

* the round trip itself, both objects reading back byte-identical;
* upload ordering proven **by construction**: `--files-only` run alone,
  then the bucket inspected for real (file present, index absent), then
  `--index-only` run alone, then the bucket inspected again (index now
  present) -- two genuinely separate invocations, nothing racing, no
  synchronization hook, no `eval` (see "`--files-only` and `--index-only`"
  above for why that matters in a public repo);
* a fault-injection contrast proving the flags have real teeth: invoking
  `--index-only` **alone**, against a prefix `--files-only` never touched,
  really does publish an index naming a file that is not there -- the exact
  state the whole ordering guarantee exists to prevent -- while the
  *default* path (neither flag, one call) never leaves that state, which
  holds by construction (the default's first phase is the identical command
  array `--files-only` runs) and is checked here by its final state;
* a second publish refused when it would overwrite a real, already-published
  APK, and that APK's bytes unchanged afterwards.

No `.gitea/workflows` runs this, on the same reasoning weblib-ci#9 recorded:
run it by hand before pushing. If that changes for weblib-ci it should
probably change here too, at the same time.

## Not hard-coded

No bucket name, repo URL, app id, keystore path, or channel name or count --
a consumer may have one publish target or several, under whatever names it
likes; nothing here assumes "release" and "dev", or that there are exactly
two. No consumer is named anywhere either: the repository that runs the
publish has already changed twice during design, so naming one would be
wrong again shortly.

The one exception is deliberate and unavoidable: **this repository's own
address** appears in the `uses:` examples above, because that is what a
consumer has to type. Those lines are what will need editing if this repo
moves organisation -- nothing else should.

[weblib-ci]: https://git.chaosbit.de/weblib/weblib-ci