weblib/weblib-viewer#95: every version in the published index claimed it was added at the moment of the *last* publish, so an F-Droid client's version list showed five separate builds all added within seconds of each other. Measured on the live dev repo: index-v2.json gave versionCodes 365139/365516/365666/365710/366099 the added dates 05:50:06/08/10/12/13 on 2026-09-12, while the bucket's own Last-Modified for those objects says they were actually published on 2026-09-11 at 13:53, 20:09, 22:37, 23:23 and 2026-09-12 05:50. It is not an mtime problem, and --use-date-from-apk does not fix it. fdroidserver takes the date from common.KnownApks, whose __init__ loads it out of an already existing repo/index-v2.json (and archive/index-v2.json), and whose recordapk falls back to datetime.now() for any filename it has not seen; --use-date-from-apk only changes that fallback to the local file's mtime, which for a re-downloaded APK is when it was downloaded. KnownApks' own docstring says there is nowhere else to look: the index is "currently the only way to permanently store the 'updated' date of APKs". A CI job that rebuilds repo/ on a fresh runner -- which is exactly what weblib-viewer#32's retention scheme does when it re-downloads the previous APKs -- therefore starts with an empty KnownApks, and every build in it counts as new. So hand fdroid-update.sh the previously published index, optionally, as FDROID_PREVIOUS_INDEX -- a local path, because fetching it needs to know the bucket and per fdroid-repo-ci#1 that stays on the consumer's side; this script still needs no network. preserve_added.py stages it where KnownApks looks, and afterwards *checks* that every build present in both indexes really did keep its date, failing the run if not. That is not belt-and-braces: fdroidserver's own read_added_date_from_all_apks carries a TODO about reading these dates differently, so a future version could drop the behaviour this depends on, and a silent return to flattened history is the failure mode that took a human noticing to catch the first time. Set-but-missing is a hard failure rather than a shrug, and so is a file that is not a parseable index-v2.json (a truncated download, an S3 NoSuchKey error document): KnownApks has no error handling of its own and would abort mid-library with a traceback that never names the file, and a consumer whose fetch silently produced nothing would otherwise flatten its whole history on every publish while looking healthy. A first-ever publish expresses "there is no previous index" by leaving the variable unset. Verified end to end without publishing anything, against the five real APKs and a copy of the live dev index-v2.json pulled over HTTPS: - no previous index (today's behaviour), all five re-stamped: 366099 06:23:19, 365710 06:23:18, 365666 06:23:17, 365516 06:23:15, 365139 06:23:14 -- a five-second spread, all of it "now". - previous index minus 366099, i.e. the real "four retained plus one genuinely new" case: 365139/365516/365666/365710 keep 05:50:06/08/10/12 exactly, 366099 gets 06:24:01, and lastUpdated follows the new build. - a second run on a fresh repo/ with the APK mtimes deliberately touched to now: all five unchanged, "5 already-published build(s) kept their original 'added' timestamp, 0 new". test_preserve_added.py (12 assertions, stdlib only) covers the drift detection in both directions, including the negative control that a drifted date fails; test_fdroid_update.sh covers the shell contract. run_tests.sh: 123 passed, 4 skipped, 0 failed. A weblib-viewer change is needed too -- its workflows have to fetch last publish's index and set FDROID_PREVIOUS_INDEX -- and is a separate PR in that repo. Co-authored-by: bit <bit@das-labor.org>
36 KiB
fdroid-repo-ci
Shared tooling to build a signed F-Droid repo index over already-signed APKs and publish the result to an S3-compatible bucket -- consumed as a composite action by whichever repository currently holds the (index) signing key and runs the publish. Which repository that is has changed more than once, so nothing here names it; see "Not hard-coded" below.
The builder signs; this repo only distributes. Settled on
weblib-viewer#19: an APK can reach a device via a git release and via
F-Droid, and updates from either route have to install over each other,
which needs the same app signature on both -- so signing has to happen
once, at the build, not again here. This is also simply how F-Droid's
binary-repo mode works: fdroid update reads each APK's existing
signature and never creates one (see "The repo index key is not the APK
signing key" below). sign-apk.sh exists only for the exceptional consumer
whose build genuinely emits an unsigned artifact -- it is not part of the
default pipeline and action.yml does not call it.
This repository is never handed an app signing key. The only key it
ever receives is the repo index key, which signs index-v1.jar/
entry.jar and identifies the repo itself to a client (pinned via
?fingerprint= in the repo URL) -- never an individual app. A workflow
change that starts passing an APK keystore or its password into this repo's
scripts (other than as an explicit, opt-in sign-apk.sh argument for the
unsigned-artifact exception above) is a sign the invariant has been broken,
not a new normal.
Same shape as weblib-ci, and public for the same measured reason: a
private repository cannot be consumed cross-repo via uses: on this Gitea
instance (see weblib-ci's README) -- every cross-repo form fails while the
repo being referenced is private, and only the full-URL form works once it
is public.
Nothing secret belongs here, ever. Keys, keystores, credentials, bucket names and hostnames all live in the private repo that consumes this one and arrive as inputs -- arguments and environment variables, never committed here.
What is here
| script | what it does |
|---|---|
next-version-code.sh |
print the next monotonic versionCode for an application id and a commit's committer date, read from a repo index -- called before the build, CI and a manual build alike (see "One versionCode rule" below) |
sign-apk.sh |
the exception, not the default flow (see above) -- check alignment, then sign a prebuilt unsigned release APK with a keystore the caller supplies, for a consumer whose build does not already produce a signed APK |
fdroid-update.sh |
run fdroid update over a repo/ directory of already-signed APKs and produce the signed index |
publish.sh |
upload the resulting repo tree to an S3-compatible bucket -- files first, index last, and refuses to overwrite an already-published APK (see "Publish ordering and safety" below) |
with-nixpkgs.sh |
run a command with one or more nixpkgs packages on PATH, pinned to the revision in nixpkgs-pin |
check_zip_alignment.py |
the alignment check sign-apk.sh runs before signing (see "Align before sign" below) |
next_version_code.py |
the stdlib-only implementation next-version-code.sh wraps (index read, arithmetic) |
preserve_added.py |
stage last publish's index-v2.json so fdroid update reuses its per-build added dates, and check afterwards that it did (see "Per-build added timestamps" below) |
action.yml |
the composite action a consumer includes to get all of the above |
Deliberately not in scope: fetching the APK. That needs an upstream credential and is the consumer's business, not the tooling's -- which also keeps this repo from knowing which upstreams exist.
All of it is standard library / plain bash plus the pinned tools below. They are run, not built, so this repo has no flake -- see "Pinning nixpkgs" below for what that costs and why it is still the right call here.
Using it
This is the default flow: the APK arrives at this action already signed
by the builder (weblib-viewer#19), so the pipeline only computes the next
versionCode, indexes and publishes. It never touches an app signing key --
only the repo index key, in the fdroid-update.sh step.
- uses: actions/checkout@v4
- id: ci
uses: https://git.chaosbit.de/weblib/fdroid-repo-ci@main
- run: |
version_code=$(bash "${{ steps.ci.outputs.path }}/next-version-code.sh" \
"$FDROID_REPO_INDEX_URL" "$APPLICATION_ID" "$(git log -1 --format=%ct)")
echo "version_code=$version_code" >> "$GITHUB_ENV"
# ... feed $version_code into whatever builds the already-signed
# app-release.apk
- env:
INDEX_KEYSTORE_PASSWORD: ${{ secrets.INDEX_KEYSTORE_PASSWORD }}
FDROID_REPO_URL: https://fdroid-weblib.s3w.dev.chaosbit.de/repo
FDROID_REPO_NAME: weblib
FDROID_REPO_DESCRIPTION: "weblib's F-Droid repository"
# Last publish's index, fetched by the consumer (only it knows the
# bucket). Leave UNSET for a channel's first-ever publish; see
# "Per-build `added` timestamps" below.
FDROID_PREVIOUS_INDEX: /tmp/previous-index-v2.json
run: |
bash "${{ steps.ci.outputs.path }}/fdroid-update.sh" \
"$FDROID_REPO_ROOT" "$INDEX_KEYSTORE" repo-key INDEX_KEYSTORE_PASSWORD
- env:
AWS_ACCESS_KEY_ID: ${{ secrets.PUBLISH_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.PUBLISH_SECRET_ACCESS_KEY }}
run: |
bash "${{ steps.ci.outputs.path }}/publish.sh" \
"$FDROID_REPO_ROOT/repo" "$BUCKET" "$S3_ENDPOINT_URL" repo
The trailing repo is publish.sh's <prefix> argument -- it has to put
the tree under a URL path ending in /repo to match FDROID_REPO_URL above
(see "FDROID_REPO_URL must end in the literal suffix /repo" below).
Use the full URL, https://git.chaosbit.de/weblib/fdroid-repo-ci@main, not
weblib/fdroid-repo-ci@main. Measured on weblib-ci (weblib-archive#44):
the bare owner/repo form resolves against the instance's default actions
URL rather than this host and fails outright. The full-URL form is also the
only one that works while a referenced repo is private -- irrelevant for
this repo, which is public on purpose, but relevant for whatever repo is
consuming it, since that one holds the index key.
$INDEX_KEYSTORE/$FDROID_REPO_ROOT/$BUCKET/$S3_ENDPOINT_URL/
$FDROID_REPO_INDEX_URL/$APPLICATION_ID above are the consumer's own
paths and values -- nothing in this repo defaults or hints at them. The env
var name INDEX_KEYSTORE_PASSWORD is the consumer's choice too, not a
convention this repo imposes: fdroid-update.sh takes the name of the
environment variable to read the password from as an argument. Name it
distinctly from whatever holds the APK key's password even if, today, both
happen to resolve to the same underlying keystore/secret (see "The repo
index key is not the APK signing key" below) -- that way separating the two
keys later is a value change on the consumer's side, not a rename of this
convention.
FDROID_REPO_URL and FDROID_REPO_NAME are required environment
variables, not arguments -- unlike the password env var names above,
these are the actual values (not secrets), so they are passed directly
rather than through an indirection. fdroid-update.sh refuses to run
without them, naming the missing one (fdroid-repo-ci#11): a repo that
fdroid update builds without repo_url/repo_name set publishes
successfully while advertising fdroidserver's own placeholder template
(MyFirstFDroidRepo.org, "My First F-Droid Repo Demo") -- silently wrong,
which is worse than a failed job. FDROID_REPO_DESCRIPTION is optional and
has no fallback if omitted (fdroidserver's own default, which is the same
placeholder problem, one field narrower).
FDROID_REPO_URL must end in the literal suffix /repo. This is not a
convention of this repo -- fdroidserver's own common.py refuses to run
any command once config.yml sets repo_url at all, unless it ends in
/repo (repo_url needs to end with /repo), and there is no flag to turn
that check off. Whatever value is set here is also embedded verbatim as
repo.address in index-v2.json, so it has to be the actual URL path
index-v2.json is served under, not merely a plausible-looking one:
confirm with <repo_url>/index-v2.json resolving, don't guess.
Concretely, that means publish.sh's own <prefix> argument has to be used
to put the tree under a URL path ending in /repo -- e.g. publish with
prefix repo so the bucket serves .../repo/index-v2.json, and set
FDROID_REPO_URL to https://<bucket>.<domain>/repo to match. Publishing
to the bucket root with no prefix (as this repo's consumer did before this
was noticed) cannot satisfy fdroidserver's own requirement at all: any
repo_url set there either lacks the suffix (and fdroid update refuses to
run) or has the suffix and points at a path the bucket never actually
serves (and a client's "Add to F-Droid" 404s on index-v2.json) -- the
/repo prefix is not optional decoration, it is what makes repo_url
and the published tree agree at all.
Per-build added timestamps (FDROID_PREVIOUS_INDEX, weblib/weblib-viewer#95)
A repo that rebuilds repo/ from scratch on every run must hand
fdroid-update.sh the index it published last time, or every build in it
gets re-stamped with the current wall clock. That is
weblib/weblib-viewer#95: an F-Droid client's version list showed five
separate builds all "added" within seconds of each other, all at the moment
of the most recent publish.
It is not an mtime problem and --use-date-from-apk does not fix it.
fdroidserver takes the date from common.KnownApks, which loads it out of
an already existing index:
for part in ('repo', 'archive'):
path = os.path.join(part, 'index-v2.json')
if os.path.isfile(path):
...
filename = version["file"]["name"][1:]
date = datetime.fromtimestamp(version["added"] // 1000, tz=timezone.utc)
self.apks[filename] = date
and falls back to the wall clock for any filename it has not seen:
if apkName not in self.apks:
if default_date is None:
default_date = datetime.now(timezone.utc)
self.apks[apkName] = default_date
return self.apks[apkName]
KnownApks' own docstring says why there is nowhere else to look: the index
is "currently the only way to permanently store the 'updated' date of APKs".
A CI job on a fresh runner starts with no repo/index-v2.json, so
KnownApks starts empty and everything, including APKs re-downloaded by a
retention scheme, counts as new. (--use-date-from-apk only changes the
fallback from now() to the local file's mtime, which for a re-downloaded
file is the time it was downloaded -- no better.)
So: fetch the previously published index-v2.json and point
FDROID_PREVIOUS_INDEX at the local copy. fdroid-update.sh stages it
where KnownApks looks, runs the update, and then checks that every
build present in both indexes really did keep its date, failing the job if
not -- fdroidserver's own read_added_date_from_all_apks carries a TODO
about reading these dates differently, so this is verified rather than
assumed.
- Unset it for a first-ever publish. A path that is set but does not exist is a hard failure, deliberately: a consumer whose fetch silently produced nothing would otherwise flatten its entire history on every publish and look perfectly healthy doing it.
- A file that is not a parseable
index-v2.json(a truncated download, an S3NoSuchKeyerror document) is refused beforefdroid updatestarts, naming the path --KnownApkshas no error handling of its own and would abort mid-library with a traceback that never mentions the file. - A genuinely new build is untouched and still gets "now"; that is the one date that should be the current time.
lastUpdatedkeeps meaning the real last update: fdroidserver derives it as the newestaddedacross the app's builds.- Fetching it is the consumer's job, not this repo's, for the same reason
fetching the APK is: only the consumer knows the bucket, and per
fdroid-repo-ci#1 this repo must not come to know.
fdroid-update.shitself still needs no network.
One caveat worth stating rather than discovering: the app-level
metadata.added is derived by fdroidserver as the oldest added among the
builds currently in the index, so it moves forward as old APKs age out of a
retention window. That is inherent to retention -- the index cannot
reference an APK that is no longer published -- and is not what #95 is
about.
The exception: a consumer whose build does not already sign
sign-apk.sh is not part of the flow above and action.yml does not
call it. It exists only for a consumer whose build genuinely emits an
unsigned APK and has nowhere else to sign it -- an app signing key cannot be
rotated, so a build that produces an unsigned artifact and relies on this
step is a real (if narrow) case, not a shortcut around the settled model.
Insert it between the build and the fdroid-update.sh step above:
- env:
APK_KEYSTORE_PASSWORD: ${{ secrets.APK_KEYSTORE_PASSWORD }}
run: |
bash "${{ steps.ci.outputs.path }}/sign-apk.sh" \
app-release-unsigned.apk app-release.apk \
"$APK_KEYSTORE" release-key APK_KEYSTORE_PASSWORD
Using it at all means this repo's job is handed an app signing key for that run, which is exactly the thing "This repository is never handed an app signing key" (above) warns against as the default. Treat a workflow that includes this step as the documented exception, and prefer fixing the build to emit a signed APK over reaching for it.
One versionCode rule, shared by CI and a manual build (fdroid-repo-ci#6)
Two implementations of "the next version code" is the exact failure mode
next-version-code.sh exists to avoid, so both CI and a manual build call
this one script rather than each computing their own. The candidate is
derived from the committer date of the commit being built, not the
build clock -- deterministic per commit, rather than depending on when a
runner happened to pick the job up -- and clamped on both sides:
candidate = whole minutes since 2026-01-01T00:00:00Z, of the commit's committer date
candidate = min(candidate, minutes(now) + TOLERANCE) # forwards
published = highest versionCode already in the repo index for this applicationId
result = max(candidate, published + 1) # backwards
Minutes, not seconds or raw Unix time: versionCode is a signed 32-bit int,
and this scheme does not exhaust it for 4,082 years, against 67 years for
seconds and 11 for Unix time. Granularity is no longer what keeps
same-commit rebuilds from colliding, though -- that is a concurrency-group
plus refuse-on-existing-APK problem now (see below) -- so minutes stand
purely as a wide, cheap headroom choice, not a load-bearing part of the
concurrency story.
Backwards. Rebase, amend and cherry-pick can all produce a commit older
than one already published. The rule uses the commit's committer date,
not its author date, specifically because of this: author date survives a
rebase unchanged, so a rebased branch would keep re-emitting its original
(now stale) timestamp and the backwards clamp would never see it move.
Committer date changes on every rewrite, which is what lets published + 1
actually catch it.
Forwards. A commit's timestamp is attacker- and typo-controlled, and the
number this rule produces only ever rises -- so a bad one can never be
walked back down. A commit dated a century ahead burns decades of headroom
permanently; worse, a value placed just below the int32 ceiling is still
accepted by Android, still publishes, and then no later build can ever
exceed it -- the app becomes permanently un-updatable, recoverable only by
shipping a new applicationId and asking every user to reinstall by hand.
TOLERANCE only has to absorb honest clock skew between the committer's
machine and the runner -- a few hours is generous, a day would still be
harmless, since its only job is keeping the number near the present. A
commit that lands beyond it is clamped and a warning is printed, never
rejected outright: rejecting would block a legitimate build over a few
minutes of innocent skew, and the clamp already removes the actual harm (a
permanently reserved version-code range), so a hard failure would buy
nothing further.
The clamp is monotonic against observed state, not monotonic by
construction. It does not close the race where two jobs start in the same
minute, both read the same published value, both compute the same next
versionCode, and both build and publish -- the window is the whole build,
not just the clock read, since the version code has to be baked into the APK
before it exists. Nothing in next-version-code.sh can prevent that.
publish.sh's APK-existence check (below) is the other half of this: it
cannot stop the wasted build, but it turns the collision from a silent,
permanent wrong state into a loud failure at the last moment, when the
information is finally reliable. Concurrent same-commit builds are now also
addressed at the workflow level with a concurrency group
(cancel-in-progress) -- measured working on this Gitea instance -- which is
an optimization on top of the refusal, not a replacement for it.
Takes an index-v2.json URL or local file path, an application id, and the
committer timestamp of the commit being built as Unix seconds (exactly what
git log -1 --format=%ct prints); prints the integer and nothing else, so a
caller does
versionCode=$(next-version-code.sh "$URL" "$APPLICATION_ID" "$(git log -1 --format=%ct)").
The script takes the timestamp as an argument rather than shelling out to
git itself, which keeps it pure and testable -- the caller already knows
its own commit. The URL form needs no credentials, since both this repo and
the F-Droid repo it reads are public -- the point of weblib-viewer#16's
"buildable when CI is down". Fails loudly (non-zero exit, nothing on
stdout) if the index cannot be read; an application id with no entry at all
in the index is not an error -- that is simply the first-ever publish for a
channel, and yields the bare (clamped) candidate.
Publish ordering and safety (fdroid-repo-ci#5, fdroid-repo-ci#6)
An F-Droid repo is a signed index plus the files it names. publish.sh
uploads in two passes, unconditionally: every non-index file first, the
index (index-v1.json(.asc), index-v2.json, entry.json, entry.jar,
matched only at the top level of the tree) last. A publish interrupted
anywhere in the first pass leaves the repo describing the old state a little
longer; it can never leave an index naming files that 404. Credentials are
verified with a real authenticated call (head-bucket) before either pass,
rather than merely checking that the environment variables are non-empty --
an unusable-but-present credential failing halfway through is exactly how a
partial (and potentially index-first) upload happens.
Before pass 1, publish.sh also checks every local *.apk's destination
key with head-object and refuses the whole publish if any of them already
exists -- the safeguard next-version-code.sh's clamp cannot provide on its
own (see above). This does not prevent the wasted build behind a same-minute
collision; it prevents the collision from silently corrupting the repo.
Skipped entirely for a plain --index-only invocation (below), since that
pass can never upload a *.apk in the first place.
--files-only and --index-only
The two passes can also be run as two separate invocations:
bash publish.sh --files-only "$FDROID_REPO_ROOT/repo" "$BUCKET" "$S3_ENDPOINT_URL"
# ... build finishes, or a later job resumes here ...
bash publish.sh --index-only "$FDROID_REPO_ROOT/repo" "$BUCKET" "$S3_ENDPOINT_URL"
With neither flag, both passes run in one call, files then index -- the
default, and the only way to reach "index before files" is to invoke
--index-only on its own, which is opt-in misuse, never the default. This
is real, independently useful behaviour, not just a testing seam: an
interrupted publish can resume at the index step without re-uploading every
APK, and a caller can stage files now and publish the index later.
It is also what makes the ordering guarantee testable by construction:
run --files-only, assert the bucket has the file and not the index yet,
run --index-only, assert the index then appears -- two genuinely separate
invocations, with the bucket inspected for real in between. An earlier
version of this script proved the same property with a same-process
synchronization hook that eval'd an environment variable between the two
passes; that was removed in favour of these flags; a script that runs with
a release signing key and S3 credentials in its environment, in a public
repository, should not carry a documented path from env-var influence to
arbitrary code execution, however unlikely, when a real feature closes the
same gap for free.
Align before sign, and how "no zipalign" was verified
zipalign then apksigner. Signing first and aligning after is the old
jarsigner-era order and invalidates an apksigner-era (v2/v3) signature,
because that scheme signs the exact bytes of the packaged zip and alignment
padding changes those bytes.
apksigner is a standalone nixpkgs attribute (nixpkgs#apksigner,
verified at 35.0.6). zipalign is not -- it exists only inside
androidenv's composed build-tools, which pulls a full versioned Android SDK
and its licence machinery for one binary. Before accepting that as a
dependency, three things were measured rather than assumed:
- Google's own docs (developer.android.com/tools/zipalign): "If you
build using Android Studio, which uses the Android Gradle plugin (AGP),
this is done automatically. In this case, you should still use zipalign
to verify that the APK is aligned, but you don't need to align it." AGP is
what
flutter build apk(or any Gradle-based Android build) drives underneath, so this applies to any such build's output, not just Android-Studio-initiated ones. fdroidserver2.4.3's own source never callszipalign, anywhere -- grepped across all 38 of its modules. Its APK-signature-copying code (apksigcopier.py, used for reproducible-build verification, a different flow than binary-repo mode) implements realignment itself in pure Python against the zip format directly, rather than shelling out to thezipalignbinary.apksignerdoes not check or enforce alignment at all, in either direction -- measured directly: a deliberately misaligned fixture zip (including a.somisaligned to the stricter 4096-byte boundary) was signed and then verified withapksigner verify --verbose, and the output carried no warning about alignment whatsoever.
Confidence: high that a Gradle/AGP-built unsigned release APK is already
aligned, resting on (1) and (2) above. Not independently confirmed
against a real flutter build apk output run in this environment -- that
build was judged too expensive to attempt for this measurement. A sibling
repo's own CI proposal for building the Flutter app was rejected on the same
"the toolchain is too large to pull on every push" grounds this pin sidesteps
(composing a multi-platform Android SDK plus NDK pulls tens of gigabytes),
and the host available for this measurement had well under that in free
disk space.
Point (3) is why this repo does not simply trust point (1) and move on:
since nothing downstream complains about misalignment either, a build that
stopped emitting aligned output would fail silently -- an APK that signs and
verifies fine but behaves badly (or refuses to install) on a real device.
check_zip_alignment.py, a few lines against the standard library zipfile
module, checks every uncompressed entry of the APK sign-apk.sh is asked to
sign, and refuses to sign one that fails it. If that check ever legitimately
trips, the fix is to add androidenv's zipalign as a dependency here, not
to weaken or remove the check.
The repo index key is not the APK signing key
Binary-repo mode (every APK arrives already signed) means fdroid update
reads each APK's existing signature and never touches an app's own signing
key. It signs the index with a separate key of its own -- fdroid-update.sh
takes its own keystore/alias/password arguments, deliberately unrelated to
sign-apk.sh's. Conflating the two would mean one compromised key
undermines both the apps and the index that vouches for them.
This is also, concretely, the boundary of what this repo ever holds: it
receives the repo index key, and nothing here is the app signing key
except through the narrow, opt-in sign-apk.sh exception documented above.
The index key identifies this repo to a client, which pins it via
?fingerprint= in the repo URL; the app key identifies each app to Android
and, unlike the index key, can never be rotated once a device has installed
against it.
As of 2026-09-09 (bit), the index key is for now the same keystore as the
APK key -- they are expected to be split later. That is exactly why
fdroid-update.sh and sign-apk.sh take independently-named keystore/alias/
password arguments rather than sharing one: a consumer's workflow should
already name the index key's secrets (e.g. INDEX_KEYSTORE,
INDEX_KEYSTORE_PASSWORD) distinctly from the APK key's (e.g.
APK_KEYSTORE, APK_KEYSTORE_PASSWORD), even while both names resolve to
the same underlying secret today. Splitting the keys later is then a value
change on the consumer's side -- a new keystore, a new secret -- not a
rename of any argument or config key here.
Generate the index key as RSA 4096, never 2048. F-Droid's own
documentation suggests 2048 for a repo signing key; that is not followed
here on purpose (bit, 2026-09-09) -- use 4096 regardless of what a keytool -genkeypair/fdroid init example elsewhere shows. Recorded here so the
choice is not "corrected" back to F-Droid's default by someone going only
off those docs.
fdroid-update.sh writes <repo-root>/config.yml itself, on every run,
using fdroidserver's own {env: VARNAME} indirection for both passwords --
the exact mechanism its common.py implements specifically so config.yml
can be safely git-managed: "it is a good idea to manage config.yml (WITHOUT
PASSWORDS!) in git." A caller never has to know or hand-write fdroidserver's
config format. It also writes repo_url, repo_name and (if set)
repo_description from the environment variables of the same name, single-
quoted YAML scalars so an arbitrary value never needs the caller to think
about YAML escaping.
Pinning nixpkgs
fdroidserver's output feeds a signed index, so an unpinned or
channel-drifting nixpkgs is a worse trade here than for a disposable test
job. This repo carries a nixpkgs-pin file -- one line, a 40-character
nixpkgs revision -- and with-nixpkgs.sh refuses to run at all if that file
is missing or malformed, rather than silently falling back to an unpinned
nix shell nixpkgs#foo, which re-resolves the registry and fetches whatever
the channel currently points at.
This is deliberately not the same mechanism as weblib-ci's
with-nixpkgs.sh, which reads the consuming repo's flake.lock. This repo
has no flake (see weblib-ci#9 for the same tension in that repo: a
flake.lock existing only to hold one revision string), and per this repo's
own originating issue, the repo that consumes it holds signing keys and is
not guaranteed to be a flake either -- reading a caller's flake.lock the
way weblib-ci's script does would not even find one here. So the pin travels
with this repo instead, in a file named nixpkgs-pin on purpose: it is the
exact fallback weblib-ci#9 proposes adding to the shared script (flake.lock
present -> today's behaviour there; else nixpkgs-pin -> a bare revision;
else fail loudly). If that lands, this repo already speaks its format and
nothing here needs to change.
A nixpkgs-pin file has no nix flake update -- it is bumped by hand.
Accepted here for the same reason weblib-ci#9 accepted it for a
stdlib-only test suite, generalised: the alternative (a flake that builds
nothing, just to hold a lockfile) is a lot of structure for one string, and a
genuinely broken pinned nixpkgs would be caught immediately by this repo's
own test suite.
fdroidserver is in nixpkgs as fdroidserver, verified at 2.4.3.
publish.sh uses awscli2 for the same S3-compatible upload every consumer
here needs, resolved through the same pin.
Testing
bash run_tests.sh
123 assertions across eight suites offline (~20s on a warm nix store, 4 deliberately skipped -- see below), plus 13 more that only run against a real bucket:
test_check_zip_alignment.py-- pure standard library, no nix, no network at all; builds an aligned and a misaligned fixture zip by hand and checks both directions, including the 4096-byte boundary specific to.soentries.test_next_version_code.py-- pure standard library too, testingnext_version_code.pyas a subprocess against fixture index-v2.json files written to a temp dir: a normal commit yields its own minute, a commit older than the published maximum still yields published+1 (backwards -- rebase/amend/cherry-pick), a future-dated commit within tolerance passes through untouched, one beyond tolerance is clamped to roughly now and warns (forwards -- a bad clock or a malicious/typo'd date), a commit dated near the int32 ceiling (year 2999) does not produce a near-int32 result, several published versions at once, a version entry with no versionCode, an application id missing from the index entirely (not an error), and an unreadable/malformed index (fails loudly, nothing on stdout). A dedicated property test also checks, across every one of those scenarios in one place, that the result always strictly exceeds every versionCode already in the fixture -- the property that actually matters, which a future refactor could break while each individual case still looks right. Both the commit's committer date and "now" (used only for the forwards clamp) are pinned independently, the latter via an undocumented env var seam, so every expected number is fixed rather than depending on whatever minute the suite happens to run in.test_preserve_added.py-- standard library too, runningpreserve_added.pyas a subprocess against synthetic index-v2.json fixtures: staging puts the file at exactlyrepo/index-v2.json(the only pathcommon.KnownApkslooks at), a missing/truncated/not-an-index previous index is refused with the path named and nothing staged, and verification passes when every shared build kept its date while failing when one drifted -- the negative control, since a checker that only ever passes is worth nothing. Also: a build that aged out of the retention window is not drift, a leading/onfile.name(present in the index, stripped byKnownApks) must not make every build look new, a version entry with noaddedat all is skipped rather than defaulted, and the sub-second tolerance is exactly one second wide -- fdroidserver only ever emits whole seconds, but a whole second of drift still fails.test_fdroid_update.shcovers the same feature's shell-level contract (FDROID_PREVIOUS_INDEXset but missing, or set to garbage, aborts beforefdroid updateruns); the preservation itself, end to end against real APKs, was measured against a copy of the live dev index rather than in the suite, because this repo's fixture APK is deliberately not parseable by androguard and so never reaches the index at all.test_with_nixpkgs.sh-- offline; every way the pinning wrapper is supposed to refuse rather than silently degrade (missing pin, malformed pin, too few arguments). Does not exercise the happy path of actually resolving agithub:nixos/nixpkgs/...reference automatically, since that needs network (or an already-warm local cache) the same way weblib-ci's ownwith-nixpkgs.shdoes, and that repo does not automate a test for it either. It was checked by hand instead.test_sign_apk.shandtest_fdroid_update.sh-- full end-to-end runs against a throwaway keystore generated fresh per run, realapksignerandfdroid update. Needs the pinned toolchain to be resolvable (locally cached, or fetched once over the network); nothing beyond that.test_next_version_code.sh-- the realnext-version-code.shCLI, throughwith-nixpkgs.sh: argument count (now three: index, application id, committer timestamp), a local index file end to end, a first-ever publish for an application id, a far-future committer date clamped with a warning on stderr, a non-numeric committer timestamp refused rather than coerced, a missing/malformed local file, and an unreachable URL (http://127.0.0.1:1, the same closed-port tricktest_publish.shuses) -- proving the wrapper's argv/exit-code/stdout contract, not just the Python module's arithmetic.test_publish.sh-- argument/credential validation,--dry-runcommand construction (both the default two-command form, files then index, and each of--files-only/--index-onlyprinting exactly one command, plus their mutual exclusivity), and a real invocation against a closed local port (127.0.0.1:1), fully offline. Also runstest_s3_stub.py, a same-repo, HEAD-only local HTTP stub, to prove the credential check is a real authenticated call (a wrong bucket is rejected), that an already-published APK is refused before any upload, that--index-onlyalone never runs that check at all (it cannot upload an APK), and a negative control proving a non-colliding APK does not trip the check. The actual upload path -- and the two-pass ordering of real uploads, which nothing offline can see -- is exercised only by the opt-in live round trip below, matching the constraint that onlypublish.shneeds network in the first place.
test_publish.sh's live section (skipped by default; needs
FDROID_CI_LIVE_BUCKET, FDROID_CI_LIVE_ENDPOINT and real credentials) adds
13 more assertions against a real bucket:
- the round trip itself, both objects reading back byte-identical;
- upload ordering proven by construction:
--files-onlyrun alone, then the bucket inspected for real (file present, index absent), then--index-onlyrun alone, then the bucket inspected again (index now present) -- two genuinely separate invocations, nothing racing, no synchronization hook, noeval(see "--files-onlyand--index-only" above for why that matters in a public repo); - a fault-injection contrast proving the flags have real teeth: invoking
--index-onlyalone, against a prefix--files-onlynever touched, really does publish an index naming a file that is not there -- the exact state the whole ordering guarantee exists to prevent -- while the default path (neither flag, one call) never leaves that state, which holds by construction (the default's first phase is the identical command array--files-onlyruns) and is checked here by its final state; - a second publish refused when it would overwrite a real, already-published APK, and that APK's bytes unchanged afterwards.
No .gitea/workflows runs this, on the same reasoning weblib-ci#9 recorded:
run it by hand before pushing. If that changes for weblib-ci it should
probably change here too, at the same time.
Not hard-coded
No bucket name, repo URL, app id, keystore path, or channel name or count -- a consumer may have one publish target or several, under whatever names it likes; nothing here assumes "release" and "dev", or that there are exactly two. No consumer is named anywhere either: the repository that runs the publish has already changed twice during design, so naming one would be wrong again shortly.
The one exception is deliberate and unavoidable: this repository's own
address appears in the uses: examples above, because that is what a
consumer has to type. Those lines are what will need editing if this repo
moves organisation -- nothing else should.