fdroid-update.sh: carry each build's "added" date across a rebuild

weblib/weblib-viewer#95: every version in the published index claimed it
was added at the moment of the *last* publish, so an F-Droid client's
version list showed five separate builds all added within seconds of each
other. Measured on the live dev repo: index-v2.json gave versionCodes
365139/365516/365666/365710/366099 the added dates 05:50:06/08/10/12/13 on
2026-09-12, while the bucket's own Last-Modified for those objects says
they were actually published on 2026-09-11 at 13:53, 20:09, 22:37, 23:23
and 2026-09-12 05:50.

It is not an mtime problem, and --use-date-from-apk does not fix it.
fdroidserver takes the date from common.KnownApks, whose __init__ loads it
out of an already existing repo/index-v2.json (and archive/index-v2.json),
and whose recordapk falls back to datetime.now() for any filename it has
not seen; --use-date-from-apk only changes that fallback to the local
file's mtime, which for a re-downloaded APK is when it was downloaded.
KnownApks' own docstring says there is nowhere else to look: the index is
"currently the only way to permanently store the 'updated' date of APKs".
A CI job that rebuilds repo/ on a fresh runner -- which is exactly what
weblib-viewer#32's retention scheme does when it re-downloads the previous
APKs -- therefore starts with an empty KnownApks, and every build in it
counts as new.

So hand fdroid-update.sh the previously published index, optionally, as
FDROID_PREVIOUS_INDEX -- a local path, because fetching it needs to know
the bucket and per fdroid-repo-ci#1 that stays on the consumer's side;
this script still needs no network. preserve_added.py stages it where
KnownApks looks, and afterwards *checks* that every build present in both
indexes really did keep its date, failing the run if not. That is not
belt-and-braces: fdroidserver's own read_added_date_from_all_apks carries
a TODO about reading these dates differently, so a future version could
drop the behaviour this depends on, and a silent return to flattened
history is the failure mode that took a human noticing to catch the first
time.

Set-but-missing is a hard failure rather than a shrug, and so is a file
that is not a parseable index-v2.json (a truncated download, an S3
NoSuchKey error document): KnownApks has no error handling of its own and
would abort mid-library with a traceback that never names the file, and a
consumer whose fetch silently produced nothing would otherwise flatten its
whole history on every publish while looking healthy. A first-ever publish
expresses "there is no previous index" by leaving the variable unset.

Verified end to end without publishing anything, against the five real
APKs and a copy of the live dev index-v2.json pulled over HTTPS:

  - no previous index (today's behaviour), all five re-stamped:
    366099 06:23:19, 365710 06:23:18, 365666 06:23:17, 365516 06:23:15,
    365139 06:23:14 -- a five-second spread, all of it "now".
  - previous index minus 366099, i.e. the real "four retained plus one
    genuinely new" case: 365139/365516/365666/365710 keep 05:50:06/08/10/12
    exactly, 366099 gets 06:24:01, and lastUpdated follows the new build.
  - a second run on a fresh repo/ with the APK mtimes deliberately touched
    to now: all five unchanged, "5 already-published build(s) kept their
    original 'added' timestamp, 0 new".

test_preserve_added.py (12 assertions, stdlib only) covers the drift
detection in both directions, including the negative control that a
drifted date fails; test_fdroid_update.sh covers the shell contract.
run_tests.sh: 123 passed, 4 skipped, 0 failed.

A weblib-viewer change is needed too -- its workflows have to fetch last
publish's index and set FDROID_PREVIOUS_INDEX -- and is a separate PR in
that repo.

Co-authored-by: bit <bit@das-labor.org>

test_fdroid_update.sh

@@ -131,4 +131,59 @@ assert repo['name']['en-US'] == \"bit's test repo\", repo['name']
 "
 export FDROID_REPO_NAME="fdroid-repo-ci test repo"
 
+# --- FDROID_PREVIOUS_INDEX (weblib/weblib-viewer#95) ---
+#
+# The preservation itself is measured by test_preserve_added.py (unit) and,
+# end to end, against a copy of the live dev index -- not here: this suite's
+# fixture APK is deliberately not parseable by androguard (see the top of
+# this file), so `fdroid update` indexes zero packages and there is no
+# `added` date to preserve. What IS testable here is the part that runs
+# before fdroidserver is ever reached: the contract that a set-but-wrong
+# FDROID_PREVIOUS_INDEX aborts loudly rather than silently flattening every
+# already-published build's history on the next publish.
+
+echo "--- fault: FDROID_PREVIOUS_INDEX pointing at a file that does not exist is refused ---"
+assert_failure "fdroid-update.sh refuses a FDROID_PREVIOUS_INDEX that does not exist" \
+  env FDROID_PREVIOUS_INDEX="$work/never-fetched.json" \
+    bash "$here/fdroid-update.sh" "$work" index.jks indexkey IDX_KS_PASS
+
+echo "--- fault: a truncated/garbage previous index is refused before fdroid update runs ---"
+printf '%s' '{"packages": {"de.example' > "$work/truncated.json"
+assert_failure "fdroid-update.sh refuses a previous index that is not valid JSON" \
+  env FDROID_PREVIOUS_INDEX="$work/truncated.json" \
+    bash "$here/fdroid-update.sh" "$work" index.jks indexkey IDX_KS_PASS
+
+echo "--- fault: valid JSON that is not an index (e.g. an S3 error document) is refused ---"
+printf '%s' '{"error": "NoSuchKey"}' > "$work/notanindex.json"
+assert_failure "fdroid-update.sh refuses a previous index with no top-level packages" \
+  env FDROID_PREVIOUS_INDEX="$work/notanindex.json" \
+    bash "$here/fdroid-update.sh" "$work" index.jks indexkey IDX_KS_PASS
+
+echo "--- a valid previous index is staged, survives the run, and is verified afterwards ---"
+# An empty-but-well-formed index is exactly the shape a channel's
+# first-ever publish would leave behind, and it proves the staged file does
+# not confuse `fdroid update` -- which overwrites it -- while still being
+# read back for the after-the-fact check.
+printf '%s' '{"repo": {}, "packages": {}}' > "$work/prev-empty.json"
+out="$work/prev-empty.log"
+if env FDROID_PREVIOUS_INDEX="$work/prev-empty.json" \
+     bash "$here/fdroid-update.sh" "$work" index.jks indexkey IDX_KS_PASS >"$out" 2>&1; then
+  pass "fdroid-update.sh runs with FDROID_PREVIOUS_INDEX set"
+else
+  fail "fdroid-update.sh runs with FDROID_PREVIOUS_INDEX set"
+  cat "$out" >&2
+fi
+assert_contains "the previous index was staged for fdroidserver to read" "$out" \
+  "fdroid update will carry the 'added' date of 0 known build(s)"
+assert_contains "the result was checked, not merely hoped for" "$out" \
+  "already-published build(s) kept their original 'added' timestamp"
+
+echo "--- and the staged copy never replaces the freshly generated index ---"
+assert_success "index-v2.json after the run is the new one, not the staged previous one" \
+  bash "$here/with-nixpkgs.sh" python3 python3 -c "
+import json
+index = json.load(open('repo/index-v2.json'))
+assert index['repo']['address'] == 'https://fdroid-repo-ci-test.example.invalid/repo', index['repo']
+"
+
 report_and_exit