claudeandbit
2906f00564
weblib/weblib-viewer#95: every version in the published index claimed it was added at the moment of the *last* publish, so an F-Droid client's version list showed five separate builds all added within seconds of each other. Measured on the live dev repo: index-v2.json gave versionCodes 365139/365516/365666/365710/366099 the added dates 05:50:06/08/10/12/13 on 2026-09-12, while the bucket's own Last-Modified for those objects says they were actually published on 2026-09-11 at 13:53, 20:09, 22:37, 23:23 and 2026-09-12 05:50. It is not an mtime problem, and --use-date-from-apk does not fix it. fdroidserver takes the date from common.KnownApks, whose __init__ loads it out of an already existing repo/index-v2.json (and archive/index-v2.json), and whose recordapk falls back to datetime.now() for any filename it has not seen; --use-date-from-apk only changes that fallback to the local file's mtime, which for a re-downloaded APK is when it was downloaded. KnownApks' own docstring says there is nowhere else to look: the index is "currently the only way to permanently store the 'updated' date of APKs". A CI job that rebuilds repo/ on a fresh runner -- which is exactly what weblib-viewer#32's retention scheme does when it re-downloads the previous APKs -- therefore starts with an empty KnownApks, and every build in it counts as new. So hand fdroid-update.sh the previously published index, optionally, as FDROID_PREVIOUS_INDEX -- a local path, because fetching it needs to know the bucket and per fdroid-repo-ci#1 that stays on the consumer's side; this script still needs no network. preserve_added.py stages it where KnownApks looks, and afterwards *checks* that every build present in both indexes really did keep its date, failing the run if not. That is not belt-and-braces: fdroidserver's own read_added_date_from_all_apks carries a TODO about reading these dates differently, so a future version could drop the behaviour this depends on, and a silent return to flattened history is the failure mode that took a human noticing to catch the first time. Set-but-missing is a hard failure rather than a shrug, and so is a file that is not a parseable index-v2.json (a truncated download, an S3 NoSuchKey error document): KnownApks has no error handling of its own and would abort mid-library with a traceback that never names the file, and a consumer whose fetch silently produced nothing would otherwise flatten its whole history on every publish while looking healthy. A first-ever publish expresses "there is no previous index" by leaving the variable unset. Verified end to end without publishing anything, against the five real APKs and a copy of the live dev index-v2.json pulled over HTTPS: - no previous index (today's behaviour), all five re-stamped: 366099 06:23:19, 365710 06:23:18, 365666 06:23:17, 365516 06:23:15, 365139 06:23:14 -- a five-second spread, all of it "now". - previous index minus 366099, i.e. the real "four retained plus one genuinely new" case: 365139/365516/365666/365710 keep 05:50:06/08/10/12 exactly, 366099 gets 06:24:01, and lastUpdated follows the new build. - a second run on a fresh repo/ with the APK mtimes deliberately touched to now: all five unchanged, "5 already-published build(s) kept their original 'added' timestamp, 0 new". test_preserve_added.py (12 assertions, stdlib only) covers the drift detection in both directions, including the negative control that a drifted date fails; test_fdroid_update.sh covers the shell contract. run_tests.sh: 123 passed, 4 skipped, 0 failed. A weblib-viewer change is needed too -- its workflows have to fetch last publish's index and set FDROID_PREVIOUS_INDEX -- and is a separate PR in that repo. Co-authored-by: bit <bit@das-labor.org>
77 lines
2.5 KiB
Bash
Executable File
77 lines
2.5 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
#
|
|
# Runs every test in this repo and prints one passed/skipped/failed total.
|
|
# Not wired to any `.gitea/workflows` -- see the README for why, the same
|
|
# reasoning weblib-ci#9 recorded for that repo: run it by hand before
|
|
# pushing.
|
|
#
|
|
# bash run_tests.sh
|
|
#
|
|
# check_zip_alignment.py's, next_version_code.py's and preserve_added.py's
|
|
# own suites need only python3 (via with-nixpkgs.sh, since none is assumed
|
|
# to be on PATH); everything else additionally needs
|
|
# apksigner/fdroidserver/jdk21/awscli2, resolved through the same pin. All of it stays on loopback/local-store --
|
|
# nothing here reaches a real host, bucket or key, except the opt-in live
|
|
# section inside test_publish.sh, which stays skipped unless
|
|
# FDROID_CI_LIVE_BUCKET/FDROID_CI_LIVE_ENDPOINT and real credentials are set.
|
|
set -uo pipefail
|
|
here=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)
|
|
|
|
total_pass=0
|
|
total_skip=0
|
|
total_fail=0
|
|
any_hard_fail=0
|
|
|
|
run_python_suite() {
|
|
local script=$1
|
|
echo "=== $script ==="
|
|
local out
|
|
if out=$(bash "$here/with-nixpkgs.sh" python3 python3 "$here/$script" -v 2>&1); then
|
|
local n
|
|
n=$(echo "$out" | grep -oE '^Ran [0-9]+ test' | grep -oE '[0-9]+')
|
|
total_pass=$((total_pass + ${n:-0}))
|
|
echo "$out" | tail -3
|
|
else
|
|
total_fail=$((total_fail + 1))
|
|
any_hard_fail=1
|
|
echo "$out"
|
|
fi
|
|
echo
|
|
}
|
|
|
|
run_shell_suite() {
|
|
local script=$1
|
|
echo "=== $script ==="
|
|
local out rc=0
|
|
out=$(bash "$here/$script" 2>&1) || rc=$?
|
|
echo "$out"
|
|
local p s f
|
|
p=$(echo "$out" | grep -oE '^passed=[0-9]+' | grep -oE '[0-9]+' | tail -1)
|
|
s=$(echo "$out" | grep -oE 'skipped=[0-9]+' | grep -oE '[0-9]+' | tail -1)
|
|
f=$(echo "$out" | grep -oE 'failed=[0-9]+' | grep -oE '[0-9]+' | tail -1)
|
|
if [ -z "$p" ] && [ -z "$f" ]; then
|
|
# the script itself crashed before reaching report_and_exit
|
|
total_fail=$((total_fail + 1))
|
|
any_hard_fail=1
|
|
else
|
|
total_pass=$((total_pass + ${p:-0}))
|
|
total_skip=$((total_skip + ${s:-0}))
|
|
total_fail=$((total_fail + ${f:-0}))
|
|
[ "${f:-0}" -ne 0 ] && any_hard_fail=1
|
|
fi
|
|
echo
|
|
}
|
|
|
|
run_python_suite test_check_zip_alignment.py
|
|
run_python_suite test_next_version_code.py
|
|
run_python_suite test_preserve_added.py
|
|
run_shell_suite test_with_nixpkgs.sh
|
|
run_shell_suite test_sign_apk.sh
|
|
run_shell_suite test_fdroid_update.sh
|
|
run_shell_suite test_next_version_code.sh
|
|
run_shell_suite test_publish.sh
|
|
|
|
echo "===================================="
|
|
echo "TOTAL: passed=$total_pass skipped=$total_skip failed=$total_fail"
|
|
exit "$any_hard_fail"
|