publish.sh: verify credentials and upload the index last; add next-version-code.sh
Closes #5 Closes #6 An F-Droid repo is a signed index plus the files it names, and a partial upload could previously leave them disagreeing: aws s3 sync walks the tree in its own order, so an interrupted publish could land the index before the APKs it names, breaking the repo for every client until the upload finished or was rolled back. publish.sh now uploads in two passes, unconditionally -- every non-index file first, the index (index-v1.json(.asc), index-v2.json, entry.json, entry.jar) last -- so an interruption anywhere in pass 1 simply leaves the repo describing the old state a little longer, never a mixture. Credentials are also verified with one real authenticated call (head-bucket) before either pass, rather than merely checking the environment variables are non-empty, since an unusable-but-present credential failing halfway through is exactly how a partial upload happens. next-version-code.sh is the one implementation of the versionCode rule (candidate = minutes since 2026-01-01Z, clamped to strictly exceed whatever is already published for that applicationId), shared by CI and manual builds so there is exactly one place this can drift. bit pointed out (appended to #6) that the clamp is monotonic against *observed* state, not monotonic by construction: two jobs starting in the same minute can both read the same published value and race to publish the same versionCode on different APKs. Nothing can prevent that wasted build, so publish.sh now also refuses to overwrite an already-existing APK object before pass 1 -- turning the collision from a silent, permanent wrong state into a loud failure at the last moment the information is reliable. Tested offline (83 assertions: pure-stdlib suites for both scripts, the real CLIs through with-nixpkgs.sh, and a small local HEAD-only S3 stub proving the credential check is a real authenticated call and the APK-collision refusal fires only when it should) and live against fdroid-weblib-dev (7 more: a real publish observed mid-flight via a same-process synchronization hook to prove the index is genuinely absent while the file is already present, both objects reading back byte-identical, and a second publish refused against a real already-published APK with its bytes left unchanged). Co-authored-by: bit <bit@das-labor.org>
This commit is contained in:
+251
-17
@@ -2,13 +2,24 @@
|
||||
#
|
||||
# Tests for publish.sh -- the one script in this repo that needs real
|
||||
# network to do its actual job (fdroid-repo-ci#1), so its happy path against
|
||||
# a genuine bucket is out of scope here. What this asserts instead:
|
||||
# a genuine bucket is out of scope for most of this file. What this asserts:
|
||||
#
|
||||
# * argument/credential validation, offline;
|
||||
# * `--dry-run` builds the exact command a real run would use, offline;
|
||||
# * `--dry-run` builds the exact two commands (files, then index) a real run
|
||||
# would use, in that order, offline;
|
||||
# * a real invocation against a closed local port fails on the network call
|
||||
# itself, not on argument parsing -- proving the command it built is
|
||||
# well-formed, without needing a real bucket or reaching beyond loopback.
|
||||
# itself -- during the credential check specifically, before any upload is
|
||||
# attempted -- proving both that the constructed command is well-formed
|
||||
# and that verification genuinely comes first (fdroid-repo-ci#5);
|
||||
# * a real invocation against a tiny local HEAD-only S3 stub (test_s3_stub.py)
|
||||
# proves the credential check is a genuine authenticated call (a wrong
|
||||
# bucket is rejected, not waved through because a variable was merely
|
||||
# non-empty) and that an already-published APK is refused before any
|
||||
# upload -- and, as a negative control, that a *non*-colliding APK does
|
||||
# NOT trip the same check (fdroid-repo-ci#6);
|
||||
# * the opt-in live round trip against a real bucket, extended to prove the
|
||||
# two-pass ordering with real uploads (not just command construction) and
|
||||
# the APK-collision refusal against a real object.
|
||||
set -euo pipefail
|
||||
here=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)
|
||||
|
||||
@@ -23,6 +34,14 @@ LIVE_SECRET="${AWS_SECRET_ACCESS_KEY:-}"
|
||||
LIVE_REGION="${AWS_DEFAULT_REGION:-}"
|
||||
source "$here/test_lib.sh"
|
||||
|
||||
PYTHON="python3"
|
||||
have_python=false
|
||||
if command -v python3 >/dev/null 2>&1; then
|
||||
have_python=true
|
||||
elif py=$(bash "$here/with-nixpkgs.sh" python3 sh -c 'command -v python3' 2>/dev/null); then
|
||||
have_python=true
|
||||
fi
|
||||
|
||||
work=$(mktemp -d)
|
||||
trap 'rm -rf "$work"' EXIT
|
||||
mkdir -p "$work/repo"
|
||||
@@ -40,7 +59,7 @@ echo "--- missing local directory is refused ---"
|
||||
assert_failure "publish.sh refuses a non-existent local directory" \
|
||||
bash "$here/publish.sh" "$work/does-not-exist" mybucket https://s3.example.invalid
|
||||
|
||||
echo "--- --dry-run builds the right command, touches no network ---"
|
||||
echo "--- --dry-run builds two commands, files then index, touches no network ---"
|
||||
out=$(bash "$here/publish.sh" --dry-run "$work/repo" mybucket https://s3.example.invalid myprefix)
|
||||
assert_contains "dry-run names the destination bucket and prefix" "$out" "s3://mybucket/myprefix"
|
||||
assert_contains "dry-run names the endpoint" "$out" "--endpoint-url https://s3.example.invalid"
|
||||
@@ -51,11 +70,39 @@ else
|
||||
pass "dry-run does not silently add --delete"
|
||||
fi
|
||||
|
||||
nlines=$(echo "$out" | grep -c 's3 sync' || true)
|
||||
if [ "$nlines" -eq 2 ]; then
|
||||
pass "dry-run prints exactly two sync commands (files pass, index pass)"
|
||||
else
|
||||
fail "dry-run prints exactly two sync commands (files pass, index pass) (got $nlines)"
|
||||
fi
|
||||
|
||||
files_line=$(echo "$out" | grep -n 's3 sync' | sed -n '1p' | cut -d: -f1)
|
||||
index_line=$(echo "$out" | grep -n 's3 sync' | sed -n '2p' | cut -d: -f1)
|
||||
files_cmd=$(echo "$out" | sed -n "${files_line}p")
|
||||
index_cmd=$(echo "$out" | sed -n "${index_line}p")
|
||||
|
||||
assert_contains "the first (files) pass excludes index-shaped names" "$files_cmd" "--exclude index"
|
||||
assert_contains "the first (files) pass excludes entry-shaped names" "$files_cmd" "--exclude entry."
|
||||
if echo "$files_cmd" | grep -q -- "--include"; then
|
||||
fail "the first (files) pass has no --include (it did!)"
|
||||
else
|
||||
pass "the first (files) pass has no --include"
|
||||
fi
|
||||
assert_contains "the second (index) pass includes index-shaped names" "$index_cmd" "--include index"
|
||||
assert_contains "the second (index) pass includes entry-shaped names" "$index_cmd" "--include entry."
|
||||
assert_contains "the second (index) pass excludes everything else first" "$index_cmd" "--exclude"
|
||||
if [ "$files_line" -lt "$index_line" ]; then
|
||||
pass "the files pass is printed before the index pass -- index really does go last"
|
||||
else
|
||||
fail "the files pass is printed before the index pass (files at line $files_line, index at $index_line)"
|
||||
fi
|
||||
|
||||
echo "--- --dry-run with no prefix omits the trailing slash entirely ---"
|
||||
out_noprefix=$(bash "$here/publish.sh" --dry-run "$work/repo" mybucket https://s3.example.invalid)
|
||||
assert_contains "no-prefix dry-run targets the bucket root" "$out_noprefix" "s3://mybucket "
|
||||
|
||||
echo "--- a real invocation fails on the network call, not on arguments ---"
|
||||
echo "--- a real invocation fails on the network call, not on arguments -- and fails during credential verification, before any upload ---"
|
||||
# port 1 is a reserved, unassigned TCP port -- nothing listens there, so this
|
||||
# never leaves the loopback interface and needs no real network access.
|
||||
real_out=$(bash "$here/publish.sh" "$work/repo" mybucket "http://127.0.0.1:1" 2>&1) && rc=0 || rc=$?
|
||||
@@ -63,43 +110,230 @@ if [ "${rc:-0}" -ne 0 ]; then pass "publish.sh against a closed port exits non-z
|
||||
fail "publish.sh against a closed port exits non-zero (got 0)"
|
||||
fi
|
||||
assert_contains "the failure is a connection error, not an argument error" "$real_out" "Could not connect"
|
||||
assert_contains "publish.sh announces credential verification" "$real_out" "verifying credentials"
|
||||
if echo "$real_out" | grep -q "pass 1/2"; then
|
||||
fail "publish.sh must not start pass 1 when credential verification never completed (it did!)"
|
||||
else
|
||||
pass "publish.sh never starts pass 1 when credential verification fails"
|
||||
fi
|
||||
|
||||
# --- offline, against a local HEAD-only S3 stub -----------------------------
|
||||
# Real S3 semantics (200/404 over HTTP, path-style addressing since the
|
||||
# endpoint host here is a literal IP) without a real bucket or real
|
||||
# credentials -- see test_s3_stub.py's own docstring for exactly what it
|
||||
# does and does not implement, and why that is enough for these two checks
|
||||
# without ever letting publish.sh reach `aws s3 sync`.
|
||||
if [ "$have_python" = true ]; then
|
||||
start_stub() {
|
||||
# start_stub <bucket> [<existing-key> ...] -- prints the chosen port on
|
||||
# stdout, leaves the server running in the background; caller kills $stub_pid.
|
||||
local logf
|
||||
logf=$(mktemp)
|
||||
bash "$here/with-nixpkgs.sh" python3 python3 "$here/test_s3_stub.py" "$@" \
|
||||
>"$logf" 2>&1 &
|
||||
stub_pid=$!
|
||||
for _ in $(seq 1 50); do
|
||||
[ -s "$logf" ] && break
|
||||
sleep 0.1
|
||||
done
|
||||
stub_port=$(head -1 "$logf")
|
||||
stub_log=$logf
|
||||
}
|
||||
|
||||
echo "--- credential verification is a real authenticated call, not just 'is it set' ---"
|
||||
start_stub realbucket
|
||||
out=$(bash "$here/publish.sh" "$work/repo" wrongbucket "http://127.0.0.1:$stub_port" 2>&1) && rc=0 || rc=$?
|
||||
kill "$stub_pid" 2>/dev/null || true
|
||||
wait "$stub_pid" 2>/dev/null || true
|
||||
if [ "${rc:-0}" -ne 0 ]; then
|
||||
pass "publish.sh refuses a bucket the stub does not recognise (bad/unusable credentials)"
|
||||
else
|
||||
fail "publish.sh refuses a bucket the stub does not recognise (bad/unusable credentials) (got exit 0)"
|
||||
fi
|
||||
assert_contains "the rejection is reported before any upload" "$out" "credentials rejected"
|
||||
if echo "$out" | grep -q "pass 1/2"; then
|
||||
fail "publish.sh must not start pass 1 after a rejected credential check (it did!)"
|
||||
else
|
||||
pass "publish.sh never starts pass 1 after a rejected credential check"
|
||||
fi
|
||||
|
||||
echo "--- an already-published APK is refused, before any upload (fdroid-repo-ci#6) ---"
|
||||
collide_dir="$work/collide"
|
||||
mkdir -p "$collide_dir"
|
||||
echo '{}' > "$collide_dir/index-v2.json"
|
||||
echo 'unsigned bytes racing the real one' > "$collide_dir/de.example.app_1.apk"
|
||||
start_stub mybucket "myprefix/de.example.app_1.apk"
|
||||
out=$(bash "$here/publish.sh" "$collide_dir" mybucket "http://127.0.0.1:$stub_port" myprefix 2>&1) && rc=0 || rc=$?
|
||||
kill "$stub_pid" 2>/dev/null || true
|
||||
wait "$stub_pid" 2>/dev/null || true
|
||||
if [ "${rc:-0}" -ne 0 ]; then
|
||||
pass "publish.sh refuses to publish over an already-existing APK"
|
||||
else
|
||||
fail "publish.sh refuses to publish over an already-existing APK (got exit 0)"
|
||||
fi
|
||||
assert_contains "the refusal names the colliding key" "$out" "myprefix/de.example.app_1.apk"
|
||||
assert_contains "the refusal explains why" "$out" "already exists"
|
||||
if echo "$out" | grep -q "pass 1/2"; then
|
||||
fail "publish.sh must not start pass 1 after refusing a colliding APK (it did!)"
|
||||
else
|
||||
pass "publish.sh never starts pass 1 after refusing a colliding APK"
|
||||
fi
|
||||
|
||||
echo "--- negative control: a non-colliding APK does not trip the same check ---"
|
||||
start_stub mybucket # nothing pre-exists
|
||||
out=$(bash "$here/publish.sh" "$collide_dir" mybucket "http://127.0.0.1:$stub_port" myprefix 2>&1) && rc=0 || rc=$?
|
||||
kill "$stub_pid" 2>/dev/null || true
|
||||
wait "$stub_pid" 2>/dev/null || true
|
||||
# The stub has no sync/PUT support at all, so pass 1 is expected to fail
|
||||
# once it gets there -- the only thing this proves is that the collision
|
||||
# check itself did not false-positive and let the script proceed past it.
|
||||
assert_contains "with nothing pre-existing, publish.sh proceeds to pass 1" "$out" "pass 1/2"
|
||||
else
|
||||
skip "credential-check-is-real / APK-collision-refusal (no python3 available, even via with-nixpkgs.sh)"
|
||||
skip "negative control: non-colliding APK does not trip the check"
|
||||
fi
|
||||
|
||||
# --- live round trip, opt-in ------------------------------------------------
|
||||
# Skipped unless a scratch bucket and credentials are present, so the default
|
||||
# offline run is unchanged. This is the only test here that talks to a real
|
||||
# server, and the only one that could have caught the Garage checksum
|
||||
# rejection (#3): everything above proves the command is well formed, which is
|
||||
# a different claim from a server accepting it.
|
||||
# offline run is unchanged. This is the only part of the suite that talks to
|
||||
# a real server, and the only part that can prove the two-pass *ordering*
|
||||
# with real uploads rather than command construction, and the APK-collision
|
||||
# refusal against a real, already-published object.
|
||||
if [ -n "${FDROID_CI_LIVE_BUCKET:-}" ] && [ -n "${FDROID_CI_LIVE_ENDPOINT:-}" ] \
|
||||
&& [ -n "$LIVE_KEY_ID" ] && [ -n "$LIVE_SECRET" ]; then
|
||||
echo "--- a real round trip against a real endpoint ---"
|
||||
export AWS_ACCESS_KEY_ID="$LIVE_KEY_ID"
|
||||
export AWS_SECRET_ACCESS_KEY="$LIVE_SECRET"
|
||||
[ -n "$LIVE_REGION" ] && export AWS_DEFAULT_REGION="$LIVE_REGION"
|
||||
# publish.sh sets these itself, but only inside its own (child) process --
|
||||
# it does not propagate back here, and every direct `aws s3 cp -` *write*
|
||||
# this file does below needs it too, or it fails exactly like the Garage
|
||||
# checksum bug documented in publish.sh (InvalidRequest: invalid checksum
|
||||
# algorithm), which reads like a permissions problem.
|
||||
export AWS_REQUEST_CHECKSUM_CALCULATION="${AWS_REQUEST_CHECKSUM_CALCULATION:-when_required}"
|
||||
export AWS_RESPONSE_CHECKSUM_VALIDATION="${AWS_RESPONSE_CHECKSUM_VALIDATION:-when_required}"
|
||||
live_dir=$(mktemp -d)
|
||||
stamp="live-$(date -u +%s)-$$"
|
||||
printf 'publish.sh live probe %s\n' "$stamp" > "$live_dir/probe.txt"
|
||||
printf '{"probe":"%s"}\n' "$stamp" > "$live_dir/index-v2.json"
|
||||
|
||||
if bash "$here/publish.sh" "$live_dir" "$FDROID_CI_LIVE_BUCKET" \
|
||||
"$FDROID_CI_LIVE_ENDPOINT" "$stamp" >/dev/null 2>&1; then
|
||||
cleanup_live() {
|
||||
bash "$here/with-nixpkgs.sh" awscli2 aws --endpoint-url "$FDROID_CI_LIVE_ENDPOINT" \
|
||||
s3 rm --recursive "s3://$FDROID_CI_LIVE_BUCKET/$stamp/" >/dev/null 2>&1 || true
|
||||
rm -rf "$live_dir"
|
||||
}
|
||||
|
||||
echo "--- upload ordering: index must not exist while the file already does (the hard part -- see below) ---"
|
||||
# This is the one property the offline suite structurally cannot see: it
|
||||
# is about the order two *real* uploads land in, which only a real bucket
|
||||
# can answer. Racing the check in from outside against a real network call
|
||||
# is exactly the flakiness fdroid-repo-ci#5 warned off ("worth building the
|
||||
# check into the script's own output rather than trying to catch a race
|
||||
# from outside"), so instead this uses the synchronization hook publish.sh
|
||||
# exposes for tests (PUBLISH_SH_TEST_HOOK_BETWEEN_PASSES): a real HEAD
|
||||
# against the real bucket runs from *inside* publish.sh's own process,
|
||||
# at the one moment guaranteed to be after pass 1 (files) finished and
|
||||
# before pass 2 (index) started -- not a poll racing a background upload.
|
||||
order_result=$(mktemp)
|
||||
hook_script=$(mktemp)
|
||||
cat > "$hook_script" <<'HOOK'
|
||||
#!/usr/bin/env bash
|
||||
set -uo pipefail
|
||||
aws=(bash "$FDROID_HOOK_HERE/with-nixpkgs.sh" awscli2 aws \
|
||||
--endpoint-url "$FDROID_HOOK_ENDPOINT" --region "${FDROID_HOOK_REGION:-us-east-1}")
|
||||
file_present=0
|
||||
index_absent=0
|
||||
if "${aws[@]}" s3api head-object --bucket "$FDROID_HOOK_BUCKET" \
|
||||
--key "$FDROID_HOOK_STAMP/probe.txt" >/dev/null 2>&1; then
|
||||
file_present=1
|
||||
fi
|
||||
if ! "${aws[@]}" s3api head-object --bucket "$FDROID_HOOK_BUCKET" \
|
||||
--key "$FDROID_HOOK_STAMP/index-v2.json" >/dev/null 2>&1; then
|
||||
index_absent=1
|
||||
fi
|
||||
echo "file_present=$file_present index_absent=$index_absent" > "$FDROID_HOOK_RESULT_FILE"
|
||||
HOOK
|
||||
|
||||
FDROID_HOOK_HERE="$here" FDROID_HOOK_ENDPOINT="$FDROID_CI_LIVE_ENDPOINT" \
|
||||
FDROID_HOOK_REGION="${LIVE_REGION:-us-east-1}" FDROID_HOOK_BUCKET="$FDROID_CI_LIVE_BUCKET" \
|
||||
FDROID_HOOK_STAMP="$stamp" FDROID_HOOK_RESULT_FILE="$order_result" \
|
||||
PUBLISH_SH_TEST_HOOK_BETWEEN_PASSES="bash '$hook_script'" \
|
||||
bash "$here/publish.sh" "$live_dir" "$FDROID_CI_LIVE_BUCKET" \
|
||||
"$FDROID_CI_LIVE_ENDPOINT" "$stamp" >/tmp/publish_live_out.$$ 2>&1
|
||||
publish_rc=$?
|
||||
if [ "$publish_rc" -eq 0 ]; then
|
||||
pass "publish.sh uploaded to a real bucket"
|
||||
else
|
||||
fail "publish.sh uploaded to a real bucket"
|
||||
fail "publish.sh uploaded to a real bucket (exit $publish_rc)"
|
||||
cat /tmp/publish_live_out.$$ >&2
|
||||
fi
|
||||
rm -f /tmp/publish_live_out.$$
|
||||
|
||||
if [ -s "$order_result" ]; then
|
||||
order_line=$(cat "$order_result")
|
||||
if echo "$order_line" | grep -q "file_present=1 index_absent=1"; then
|
||||
pass "mid-publish (real bucket): the file was already there and the index was not ($order_line)"
|
||||
else
|
||||
fail "mid-publish (real bucket): the file was already there and the index was not (got: $order_line)"
|
||||
fi
|
||||
else
|
||||
fail "the between-passes hook never ran against the real bucket -- ordering was not actually observed"
|
||||
fi
|
||||
rm -f "$order_result" "$hook_script"
|
||||
|
||||
got=$(bash "$here/with-nixpkgs.sh" awscli2 aws --endpoint-url "$FDROID_CI_LIVE_ENDPOINT" \
|
||||
s3 cp "s3://$FDROID_CI_LIVE_BUCKET/$stamp/probe.txt" - 2>/dev/null || true)
|
||||
if [ "$got" = "publish.sh live probe $stamp" ]; then
|
||||
pass "the uploaded object reads back byte-identical"
|
||||
pass "the uploaded file object reads back byte-identical"
|
||||
else
|
||||
fail "the uploaded object reads back byte-identical"
|
||||
fail "the uploaded file object reads back byte-identical"
|
||||
fi
|
||||
got_index=$(bash "$here/with-nixpkgs.sh" awscli2 aws --endpoint-url "$FDROID_CI_LIVE_ENDPOINT" \
|
||||
s3 cp "s3://$FDROID_CI_LIVE_BUCKET/$stamp/index-v2.json" - 2>/dev/null || true)
|
||||
if [ "$got_index" = "{\"probe\":\"$stamp\"}" ]; then
|
||||
pass "the uploaded index object (pass 2) reads back byte-identical"
|
||||
else
|
||||
fail "the uploaded index object (pass 2) reads back byte-identical"
|
||||
fi
|
||||
|
||||
echo "--- APK-collision refusal against a real, already-published object ---"
|
||||
apk_stamp="live-apk-$(date -u +%s)-$$"
|
||||
apk_key="$apk_stamp/de.example.app_1.apk"
|
||||
original_bytes="original apk bytes $apk_stamp"
|
||||
printf '%s' "$original_bytes" | bash "$here/with-nixpkgs.sh" awscli2 aws \
|
||||
--endpoint-url "$FDROID_CI_LIVE_ENDPOINT" --region "${LIVE_REGION:-us-east-1}" \
|
||||
s3 cp - "s3://$FDROID_CI_LIVE_BUCKET/$apk_key" >/dev/null 2>&1
|
||||
|
||||
race_dir=$(mktemp -d)
|
||||
echo '{}' > "$race_dir/index-v2.json"
|
||||
printf 'a different build racing the same versionCode' > "$race_dir/de.example.app_1.apk"
|
||||
|
||||
race_out=$(bash "$here/publish.sh" "$race_dir" "$FDROID_CI_LIVE_BUCKET" \
|
||||
"$FDROID_CI_LIVE_ENDPOINT" "$apk_stamp" 2>&1) && race_rc=0 || race_rc=$?
|
||||
if [ "${race_rc:-0}" -ne 0 ]; then
|
||||
pass "publish.sh refuses a second publish that would overwrite a real, already-published APK"
|
||||
else
|
||||
fail "publish.sh refuses a second publish that would overwrite a real, already-published APK (got exit 0)"
|
||||
fi
|
||||
assert_contains "the real-bucket refusal names the colliding key" "$race_out" "$apk_key"
|
||||
|
||||
after_bytes=$(bash "$here/with-nixpkgs.sh" awscli2 aws --endpoint-url "$FDROID_CI_LIVE_ENDPOINT" \
|
||||
s3 cp "s3://$FDROID_CI_LIVE_BUCKET/$apk_key" - 2>/dev/null || true)
|
||||
if [ "$after_bytes" = "$original_bytes" ]; then
|
||||
pass "the original APK's bytes are unchanged after the refused publish"
|
||||
else
|
||||
fail "the original APK's bytes are unchanged after the refused publish"
|
||||
fi
|
||||
|
||||
bash "$here/with-nixpkgs.sh" awscli2 aws --endpoint-url "$FDROID_CI_LIVE_ENDPOINT" \
|
||||
s3 rm --recursive "s3://$FDROID_CI_LIVE_BUCKET/$stamp/" >/dev/null 2>&1 || true
|
||||
rm -rf "$live_dir"
|
||||
s3 rm --recursive "s3://$FDROID_CI_LIVE_BUCKET/$apk_stamp/" >/dev/null 2>&1 || true
|
||||
rm -rf "$race_dir"
|
||||
|
||||
cleanup_live
|
||||
else
|
||||
skip "live round trip (set FDROID_CI_LIVE_BUCKET, FDROID_CI_LIVE_ENDPOINT and AWS credentials)"
|
||||
skip "live upload ordering (file present, index absent mid-publish)"
|
||||
skip "live APK-collision refusal against a real object"
|
||||
fi
|
||||
|
||||
report_and_exit
|
||||
|
||||
Reference in New Issue
Block a user