fdroid-repo-ci

16 Commits 1 Branch 0 Tags

3 Commits

Include renames

Author	SHA1	Message	Date
claudeandbit	2906f00564	fdroid-update.sh: carry each build's "added" date across a rebuild weblib/weblib-viewer#95: every version in the published index claimed it was added at the moment of the last publish, so an F-Droid client's version list showed five separate builds all added within seconds of each other. Measured on the live dev repo: index-v2.json gave versionCodes 365139/365516/365666/365710/366099 the added dates 05:50:06/08/10/12/13 on 2026-09-12, while the bucket's own Last-Modified for those objects says they were actually published on 2026-09-11 at 13:53, 20:09, 22:37, 23:23 and 2026-09-12 05:50. It is not an mtime problem, and --use-date-from-apk does not fix it. fdroidserver takes the date from common.KnownApks, whose __init__ loads it out of an already existing repo/index-v2.json (and archive/index-v2.json), and whose recordapk falls back to datetime.now() for any filename it has not seen; --use-date-from-apk only changes that fallback to the local file's mtime, which for a re-downloaded APK is when it was downloaded. KnownApks' own docstring says there is nowhere else to look: the index is "currently the only way to permanently store the 'updated' date of APKs". A CI job that rebuilds repo/ on a fresh runner -- which is exactly what weblib-viewer#32's retention scheme does when it re-downloads the previous APKs -- therefore starts with an empty KnownApks, and every build in it counts as new. So hand fdroid-update.sh the previously published index, optionally, as FDROID_PREVIOUS_INDEX -- a local path, because fetching it needs to know the bucket and per fdroid-repo-ci#1 that stays on the consumer's side; this script still needs no network. preserve_added.py stages it where KnownApks looks, and afterwards checks that every build present in both indexes really did keep its date, failing the run if not. That is not belt-and-braces: fdroidserver's own read_added_date_from_all_apks carries a TODO about reading these dates differently, so a future version could drop the behaviour this depends on, and a silent return to flattened history is the failure mode that took a human noticing to catch the first time. Set-but-missing is a hard failure rather than a shrug, and so is a file that is not a parseable index-v2.json (a truncated download, an S3 NoSuchKey error document): KnownApks has no error handling of its own and would abort mid-library with a traceback that never names the file, and a consumer whose fetch silently produced nothing would otherwise flatten its whole history on every publish while looking healthy. A first-ever publish expresses "there is no previous index" by leaving the variable unset. Verified end to end without publishing anything, against the five real APKs and a copy of the live dev index-v2.json pulled over HTTPS: - no previous index (today's behaviour), all five re-stamped: 366099 06:23:19, 365710 06:23:18, 365666 06:23:17, 365516 06:23:15, 365139 06:23:14 -- a five-second spread, all of it "now". - previous index minus 366099, i.e. the real "four retained plus one genuinely new" case: 365139/365516/365666/365710 keep 05:50:06/08/10/12 exactly, 366099 gets 06:24:01, and lastUpdated follows the new build. - a second run on a fresh repo/ with the APK mtimes deliberately touched to now: all five unchanged, "5 already-published build(s) kept their original 'added' timestamp, 0 new". test_preserve_added.py (12 assertions, stdlib only) covers the drift detection in both directions, including the negative control that a drifted date fails; test_fdroid_update.sh covers the shell contract. run_tests.sh: 123 passed, 4 skipped, 0 failed. A weblib-viewer change is needed too -- its workflows have to fetch last publish's index and set FDROID_PREVIOUS_INDEX -- and is a separate PR in that repo. Co-authored-by: bit <bit@das-labor.org>	2026-09-12 06:29:20 +00:00
claudeandbit	4a25e3681a	Require FDROID_REPO_URL/FDROID_REPO_NAME so fdroid-update.sh cannot publish fdroidserver's placeholder The published repo advertised fdroidserver's own demo values (repo.address "https://MyFirstFDroidRepo.org/fdroid/repo", repo.name "My First F-Droid Repo Demo") because fdroid-update.sh wrote config.yml itself but never set repo_url/repo_name/repo_description, so fdroidserver fell back to its template. The "Add to F-Droid" button on the generated index.html is built from repo_url, which is how bit noticed. A publish that succeeds while advertising the wrong URL is silently wrong -- worse than a failed job. Add FDROID_REPO_URL/FDROID_REPO_NAME/FDROID_REPO_DESCRIPTION environment variables (not new positionals -- the script already has five plus pass-through extra args). The first two are now a hard requirement, naming the missing one; FDROID_REPO_DESCRIPTION stays optional. This is a breaking change to a public composite action's interface, acceptable only because weblib-viewer is the sole consumer and is updated in the same change (weblib-viewer#26). Also discovered while wiring this up and worth recording since it contradicts the "no /fdroid/repo suffix" assumption in weblib's working notes: fdroidserver's own common.py unconditionally refuses to run once config.yml sets repo_url at all, unless it ends in the literal suffix "/repo" -- there is no flag to disable that check, and the value is embedded verbatim as repo.address. So FDROID_REPO_URL has to end in /repo, and publish.sh's existing <prefix> argument has to put the published tree under a URL path that actually ends in /repo to match, or fdroid update refuses to run at all (checked positive) / the address points at a path the bucket never serves (checked against the live fdroid-weblib-dev bucket layout). Extends test_fdroid_update.sh: config.yml carries the values, the resulting index-v2.json has the right repo.address/name/description (not fdroidserver's placeholder), a name with an embedded single quote survives the YAML round-trip, and the script fails when FDROID_REPO_URL or FDROID_REPO_NAME is absent or FDROID_REPO_URL lacks the /repo suffix. run_tests.sh: 104 passed / 4 skipped (was 98/4). Filed as fdroid-repo-ci#11. Co-authored-by: bit <bit@das-labor.org>	2026-09-09 20:23:16 +00:00
claudeandbit	3f14c01b00	Build the shared F-Droid publishing tooling: sign, index, publish Adds sign-apk.sh (align-check + apksigner), fdroid-update.sh (fdroid update over a binary repo, signing the index with its own separate key), publish.sh (S3-compatible upload) and the composite action tying them together, matching weblib-ci's shape. zipalign is not a standalone nixpkgs attribute; resolved by measurement rather than assumption that Gradle/AGP already aligns its output (Android's own docs, corroborated by fdroidserver never shelling out to zipalign) and added check_zip_alignment.py as the safety net, since apksigner itself does not check alignment at all. nixpkgs is pinned via a committed nixpkgs-pin revision, since this repo has no flake.lock of its own for with-nixpkgs.sh to read (fdroid-repo-ci#1, weblib-ci#9) -- a missing or malformed pin fails loudly rather than degrading to an unpinned nix shell. No consumer repo, bucket, hostname or channel name is hard-coded: which repo holds the keys and runs the publish has already changed once during this issue and may again, so every example uses a placeholder. 42 offline/local-only test assertions across five suites (run_tests.sh), including fault injection on the alignment check, the password/argument validation, and the with-nixpkgs.sh fail-loud paths. Co-authored-by: bit <bit@das-labor.org>	2026-09-09 06:34:36 +00:00

Author

SHA1

Message

Date

claudeandbit

2906f00564

fdroid-update.sh: carry each build's "added" date across a rebuild

weblib/weblib-viewer#95: every version in the published index claimed it
was added at the moment of the *last* publish, so an F-Droid client's
version list showed five separate builds all added within seconds of each
other. Measured on the live dev repo: index-v2.json gave versionCodes
365139/365516/365666/365710/366099 the added dates 05:50:06/08/10/12/13 on
2026-09-12, while the bucket's own Last-Modified for those objects says
they were actually published on 2026-09-11 at 13:53, 20:09, 22:37, 23:23
and 2026-09-12 05:50.

It is not an mtime problem, and --use-date-from-apk does not fix it.
fdroidserver takes the date from common.KnownApks, whose __init__ loads it
out of an already existing repo/index-v2.json (and archive/index-v2.json),
and whose recordapk falls back to datetime.now() for any filename it has
not seen; --use-date-from-apk only changes that fallback to the local
file's mtime, which for a re-downloaded APK is when it was downloaded.
KnownApks' own docstring says there is nowhere else to look: the index is
"currently the only way to permanently store the 'updated' date of APKs".
A CI job that rebuilds repo/ on a fresh runner -- which is exactly what
weblib-viewer#32's retention scheme does when it re-downloads the previous
APKs -- therefore starts with an empty KnownApks, and every build in it
counts as new.

So hand fdroid-update.sh the previously published index, optionally, as
FDROID_PREVIOUS_INDEX -- a local path, because fetching it needs to know
the bucket and per fdroid-repo-ci#1 that stays on the consumer's side;
this script still needs no network. preserve_added.py stages it where
KnownApks looks, and afterwards *checks* that every build present in both
indexes really did keep its date, failing the run if not. That is not
belt-and-braces: fdroidserver's own read_added_date_from_all_apks carries
a TODO about reading these dates differently, so a future version could
drop the behaviour this depends on, and a silent return to flattened
history is the failure mode that took a human noticing to catch the first
time.

Set-but-missing is a hard failure rather than a shrug, and so is a file
that is not a parseable index-v2.json (a truncated download, an S3
NoSuchKey error document): KnownApks has no error handling of its own and
would abort mid-library with a traceback that never names the file, and a
consumer whose fetch silently produced nothing would otherwise flatten its
whole history on every publish while looking healthy. A first-ever publish
expresses "there is no previous index" by leaving the variable unset.

Verified end to end without publishing anything, against the five real
APKs and a copy of the live dev index-v2.json pulled over HTTPS:

  - no previous index (today's behaviour), all five re-stamped:
    366099 06:23:19, 365710 06:23:18, 365666 06:23:17, 365516 06:23:15,
    365139 06:23:14 -- a five-second spread, all of it "now".
  - previous index minus 366099, i.e. the real "four retained plus one
    genuinely new" case: 365139/365516/365666/365710 keep 05:50:06/08/10/12
    exactly, 366099 gets 06:24:01, and lastUpdated follows the new build.
  - a second run on a fresh repo/ with the APK mtimes deliberately touched
    to now: all five unchanged, "5 already-published build(s) kept their
    original 'added' timestamp, 0 new".

test_preserve_added.py (12 assertions, stdlib only) covers the drift
detection in both directions, including the negative control that a
drifted date fails; test_fdroid_update.sh covers the shell contract.
run_tests.sh: 123 passed, 4 skipped, 0 failed.

A weblib-viewer change is needed too -- its workflows have to fetch last
publish's index and set FDROID_PREVIOUS_INDEX -- and is a separate PR in
that repo.

Co-authored-by: bit <bit@das-labor.org>

2026-09-12 06:29:20 +00:00

claudeandbit

4a25e3681a

Require FDROID_REPO_URL/FDROID_REPO_NAME so fdroid-update.sh cannot publish fdroidserver's placeholder

The published repo advertised fdroidserver's own demo values
(repo.address "https://MyFirstFDroidRepo.org/fdroid/repo", repo.name "My
First F-Droid Repo Demo") because fdroid-update.sh wrote config.yml itself
but never set repo_url/repo_name/repo_description, so fdroidserver fell
back to its template. The "Add to F-Droid" button on the generated
index.html is built from repo_url, which is how bit noticed. A publish
that succeeds while advertising the wrong URL is silently wrong -- worse
than a failed job.

Add FDROID_REPO_URL/FDROID_REPO_NAME/FDROID_REPO_DESCRIPTION environment
variables (not new positionals -- the script already has five plus
pass-through extra args). The first two are now a hard requirement, naming
the missing one; FDROID_REPO_DESCRIPTION stays optional. This is a breaking
change to a public composite action's interface, acceptable only because
weblib-viewer is the sole consumer and is updated in the same change
(weblib-viewer#26).

Also discovered while wiring this up and worth recording since it
contradicts the "no /fdroid/repo suffix" assumption in weblib's working
notes: fdroidserver's own common.py unconditionally refuses to run once
config.yml sets repo_url at all, unless it ends in the literal suffix
"/repo" -- there is no flag to disable that check, and the value is
embedded verbatim as repo.address. So FDROID_REPO_URL has to end in /repo,
and publish.sh's existing <prefix> argument has to put the published tree
under a URL path that actually ends in /repo to match, or fdroid update
refuses to run at all (checked positive) / the address points at a path
the bucket never serves (checked against the live fdroid-weblib-dev bucket
layout).

Extends test_fdroid_update.sh: config.yml carries the values, the resulting
index-v2.json has the right repo.address/name/description (not
fdroidserver's placeholder), a name with an embedded single quote survives
the YAML round-trip, and the script fails when FDROID_REPO_URL or
FDROID_REPO_NAME is absent or FDROID_REPO_URL lacks the /repo suffix.
run_tests.sh: 104 passed / 4 skipped (was 98/4).

Filed as fdroid-repo-ci#11.

Co-authored-by: bit <bit@das-labor.org>

2026-09-09 20:23:16 +00:00

claudeandbit

3f14c01b00

Build the shared F-Droid publishing tooling: sign, index, publish

Adds sign-apk.sh (align-check + apksigner), fdroid-update.sh (fdroid
update over a binary repo, signing the index with its own separate
key), publish.sh (S3-compatible upload) and the composite action
tying them together, matching weblib-ci's shape.

zipalign is not a standalone nixpkgs attribute; resolved by measurement
rather than assumption that Gradle/AGP already aligns its output (Android's
own docs, corroborated by fdroidserver never shelling out to zipalign) and
added check_zip_alignment.py as the safety net, since apksigner itself does
not check alignment at all. nixpkgs is pinned via a committed nixpkgs-pin
revision, since this repo has no flake.lock of its own for with-nixpkgs.sh
to read (fdroid-repo-ci#1, weblib-ci#9) -- a missing or malformed pin fails
loudly rather than degrading to an unpinned nix shell.

No consumer repo, bucket, hostname or channel name is hard-coded: which
repo holds the keys and runs the publish has already changed once during
this issue and may again, so every example uses a placeholder.

42 offline/local-only test assertions across five suites (run_tests.sh),
including fault injection on the alignment check, the password/argument
validation, and the with-nixpkgs.sh fail-loud paths.

Co-authored-by: bit <bit@das-labor.org>

2026-09-09 06:34:36 +00:00