fdroid-update.sh: carry each build's "added" date across a rebuild

weblib/weblib-viewer#95: every version in the published index claimed it
was added at the moment of the *last* publish, so an F-Droid client's
version list showed five separate builds all added within seconds of each
other. Measured on the live dev repo: index-v2.json gave versionCodes
365139/365516/365666/365710/366099 the added dates 05:50:06/08/10/12/13 on
2026-09-12, while the bucket's own Last-Modified for those objects says
they were actually published on 2026-09-11 at 13:53, 20:09, 22:37, 23:23
and 2026-09-12 05:50.

It is not an mtime problem, and --use-date-from-apk does not fix it.
fdroidserver takes the date from common.KnownApks, whose __init__ loads it
out of an already existing repo/index-v2.json (and archive/index-v2.json),
and whose recordapk falls back to datetime.now() for any filename it has
not seen; --use-date-from-apk only changes that fallback to the local
file's mtime, which for a re-downloaded APK is when it was downloaded.
KnownApks' own docstring says there is nowhere else to look: the index is
"currently the only way to permanently store the 'updated' date of APKs".
A CI job that rebuilds repo/ on a fresh runner -- which is exactly what
weblib-viewer#32's retention scheme does when it re-downloads the previous
APKs -- therefore starts with an empty KnownApks, and every build in it
counts as new.

So hand fdroid-update.sh the previously published index, optionally, as
FDROID_PREVIOUS_INDEX -- a local path, because fetching it needs to know
the bucket and per fdroid-repo-ci#1 that stays on the consumer's side;
this script still needs no network. preserve_added.py stages it where
KnownApks looks, and afterwards *checks* that every build present in both
indexes really did keep its date, failing the run if not. That is not
belt-and-braces: fdroidserver's own read_added_date_from_all_apks carries
a TODO about reading these dates differently, so a future version could
drop the behaviour this depends on, and a silent return to flattened
history is the failure mode that took a human noticing to catch the first
time.

Set-but-missing is a hard failure rather than a shrug, and so is a file
that is not a parseable index-v2.json (a truncated download, an S3
NoSuchKey error document): KnownApks has no error handling of its own and
would abort mid-library with a traceback that never names the file, and a
consumer whose fetch silently produced nothing would otherwise flatten its
whole history on every publish while looking healthy. A first-ever publish
expresses "there is no previous index" by leaving the variable unset.

Verified end to end without publishing anything, against the five real
APKs and a copy of the live dev index-v2.json pulled over HTTPS:

  - no previous index (today's behaviour), all five re-stamped:
    366099 06:23:19, 365710 06:23:18, 365666 06:23:17, 365516 06:23:15,
    365139 06:23:14 -- a five-second spread, all of it "now".
  - previous index minus 366099, i.e. the real "four retained plus one
    genuinely new" case: 365139/365516/365666/365710 keep 05:50:06/08/10/12
    exactly, 366099 gets 06:24:01, and lastUpdated follows the new build.
  - a second run on a fresh repo/ with the APK mtimes deliberately touched
    to now: all five unchanged, "5 already-published build(s) kept their
    original 'added' timestamp, 0 new".

test_preserve_added.py (12 assertions, stdlib only) covers the drift
detection in both directions, including the negative control that a
drifted date fails; test_fdroid_update.sh covers the shell contract.
run_tests.sh: 123 passed, 4 skipped, 0 failed.

A weblib-viewer change is needed too -- its workflows have to fetch last
publish's index and set FDROID_PREVIOUS_INDEX -- and is a separate PR in
that repo.

Co-authored-by: bit <bit@das-labor.org>
This commit is contained in:
2026-09-12 06:29:20 +00:00
co-authored by bit
parent 760583b7e3
commit 2906f00564
7 changed files with 670 additions and 12 deletions
+4 -3
View File
@@ -37,8 +37,8 @@ outputs:
path:
description: >-
Directory holding with-nixpkgs.sh, nixpkgs-pin, sign-apk.sh,
fdroid-update.sh, next-version-code.sh, next_version_code.py and
publish.sh.
fdroid-update.sh, next-version-code.sh, next_version_code.py,
preserve_added.py and publish.sh.
value: ${{ github.action_path }}
runs:
@@ -48,6 +48,7 @@ runs:
run: |
echo "fdroid-repo-ci tools at ${{ github.action_path }}"
for f in with-nixpkgs.sh nixpkgs-pin sign-apk.sh fdroid-update.sh \
next-version-code.sh next_version_code.py publish.sh; do
next-version-code.sh next_version_code.py preserve_added.py \
publish.sh; do
test -f "${{ github.action_path }}/$f"
done