publish.sh: verify credentials and upload the index last; add next-version-code.sh

Closes #5
Closes #6

An F-Droid repo is a signed index plus the files it names, and a partial
upload could previously leave them disagreeing: aws s3 sync walks the tree
in its own order, so an interrupted publish could land the index before the
APKs it names, breaking the repo for every client until the upload finished
or was rolled back. publish.sh now uploads in two passes, unconditionally --
every non-index file first, the index (index-v1.json(.asc), index-v2.json,
entry.json, entry.jar) last -- so an interruption anywhere in pass 1 simply
leaves the repo describing the old state a little longer, never a mixture.
Credentials are also verified with one real authenticated call
(head-bucket) before either pass, rather than merely checking the
environment variables are non-empty, since an unusable-but-present
credential failing halfway through is exactly how a partial upload happens.

next-version-code.sh is the one implementation of the versionCode rule
(candidate = minutes since 2026-01-01Z, clamped to strictly exceed whatever
is already published for that applicationId), shared by CI and manual
builds so there is exactly one place this can drift. bit pointed out
(appended to #6) that the clamp is monotonic against *observed* state, not
monotonic by construction: two jobs starting in the same minute can both
read the same published value and race to publish the same versionCode on
different APKs. Nothing can prevent that wasted build, so publish.sh now
also refuses to overwrite an already-existing APK object before pass 1 --
turning the collision from a silent, permanent wrong state into a loud
failure at the last moment the information is reliable.

Tested offline (83 assertions: pure-stdlib suites for both scripts, the
real CLIs through with-nixpkgs.sh, and a small local HEAD-only S3 stub
proving the credential check is a real authenticated call and the
APK-collision refusal fires only when it should) and live against
fdroid-weblib-dev (7 more: a real publish observed mid-flight via a
same-process synchronization hook to prove the index is genuinely absent
while the file is already present, both objects reading back
byte-identical, and a second publish refused against a real
already-published APK with its bytes left unchanged).

Co-authored-by: bit <bit@das-labor.org>
This commit is contained in:
2026-09-09 08:00:45 +00:00
co-authored by bit
parent fcc6a741ef
commit 19e55c4a53
10 changed files with 1045 additions and 64 deletions
+13 -8
View File
@@ -7,11 +7,13 @@
#
# bash run_tests.sh
#
# check_zip_alignment.py's own suite needs only python3 (via with-nixpkgs.sh,
# since none is assumed to be on PATH); everything else additionally needs
# apksigner/fdroidserver/jdk21/awscli2, resolved through the same pin. All of
# it stays on loopback/local-store -- nothing here reaches a real host,
# bucket or key.
# check_zip_alignment.py's and next_version_code.py's own suites need only
# python3 (via with-nixpkgs.sh, since none is assumed to be on PATH);
# everything else additionally needs apksigner/fdroidserver/jdk21/awscli2,
# resolved through the same pin. All of it stays on loopback/local-store --
# nothing here reaches a real host, bucket or key, except the opt-in live
# section inside test_publish.sh, which stays skipped unless
# FDROID_CI_LIVE_BUCKET/FDROID_CI_LIVE_ENDPOINT and real credentials are set.
set -uo pipefail
here=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)
@@ -21,9 +23,10 @@ total_fail=0
any_hard_fail=0
run_python_suite() {
echo "=== test_check_zip_alignment.py ==="
local script=$1
echo "=== $script ==="
local out
if out=$(bash "$here/with-nixpkgs.sh" python3 python3 "$here/test_check_zip_alignment.py" -v 2>&1); then
if out=$(bash "$here/with-nixpkgs.sh" python3 python3 "$here/$script" -v 2>&1); then
local n
n=$(echo "$out" | grep -oE '^Ran [0-9]+ test' | grep -oE '[0-9]+')
total_pass=$((total_pass + ${n:-0}))
@@ -59,10 +62,12 @@ run_shell_suite() {
echo
}
run_python_suite
run_python_suite test_check_zip_alignment.py
run_python_suite test_next_version_code.py
run_shell_suite test_with_nixpkgs.sh
run_shell_suite test_sign_apk.sh
run_shell_suite test_fdroid_update.sh
run_shell_suite test_next_version_code.sh
run_shell_suite test_publish.sh
echo "===================================="