weblib/fdroid-repo-ci
2
0
Fork 0
Code Issues Pull Requests Actions Activity

publish.sh: stop awscli2 sending a checksum Garage rejects

Browse Source 
Measured against the real endpoint, not inferred: every PutObject failed
with

    An error occurred (InvalidRequest) when calling the PutObject
    operation: Bad request: invalid checksum algorithm

on aws-cli/2.34.24. Recent awscli2 attaches a checksum algorithm by
default and Garage refuses it, so publish.sh could not upload anything --
after passing its entire test suite.

It also reads as a permissions problem, which is the expensive part: reads
succeed, writes fail, and it fails identically against a bucket the same
credentials demonstrably can write. Set here rather than documented,
because a caller should not need to know a client-side default to publish
to the object store this script exists for, and leaving it to the
environment makes it work on one machine and not another.
`when_required` keeps checksums where the protocol needs them, so this is
narrower than switching them off.

Adds the test that would have caught it: an opt-in live round trip that
uploads, reads back and deletes, skipped unless a scratch bucket and
credentials are present so the offline run is unchanged. Everything else in
this suite proves the command is well formed, which is a different claim
from a server accepting it, and this failure lived exactly in that gap.

The live block captures the real credentials at the top of the file,
before the deliberate fakes (AKIAFAKETESTKEY) are exported for the offline
tests -- otherwise it authenticates as the fake and fails every time, and a
live test that always fails gets disabled, which is worse than not having
one.

Verified both directions against the real bucket: with the fix, 11 passed /
0 skipped; with it removed, the two live assertions fail. Offline, 9 passed
/ 1 skipped.

Closes #3.

Co-authored-by: bit <bit@das-labor.org>
This commit is contained in:
claude
2026-09-09 06:58:46 +00:00
co-authored by bit
parent c42c8a66a6
commit 5e4b2fabc6
2 changed files with 66 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
test_publish.sh
+48
View File
@@ -11,6 +11,16 @@
# well-formed, without needing a real bucket or reaching beyond loopback.
set -euo pipefail
here=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)
# Captured BEFORE the fake credentials below are exported. Those fakes are
# deliberate -- they keep the offline tests from ever reaching a real service
# -- but they also overwrite the real ones for the rest of this file, so the
# live round trip at the bottom has to hold on to the originals here or it
# would authenticate as AKIAFAKETESTKEY and fail every time. A live test that
# always fails gets disabled, which is worse than not having one.
LIVE_KEY_ID="${AWS_ACCESS_KEY_ID:-}"
LIVE_SECRET="${AWS_SECRET_ACCESS_KEY:-}"
LIVE_REGION="${AWS_DEFAULT_REGION:-}"
source "$here/test_lib.sh"
work=$(mktemp -d)
@@ -54,4 +64,42 @@ if [ "${rc:-0}" -ne 0 ]; then pass "publish.sh against a closed port exits non-z
fi
assert_contains "the failure is a connection error, not an argument error" "$real_out" "Could not connect"
# --- live round trip, opt-in ------------------------------------------------
# Skipped unless a scratch bucket and credentials are present, so the default
# offline run is unchanged. This is the only test here that talks to a real
# server, and the only one that could have caught the Garage checksum
# rejection (#3): everything above proves the command is well formed, which is
# a different claim from a server accepting it.
if [ -n "${FDROID_CI_LIVE_BUCKET:-}" ] && [ -n "${FDROID_CI_LIVE_ENDPOINT:-}" ] \
&& [ -n "$LIVE_KEY_ID" ] && [ -n "$LIVE_SECRET" ]; then
echo "--- a real round trip against a real endpoint ---"
export AWS_ACCESS_KEY_ID="$LIVE_KEY_ID"
export AWS_SECRET_ACCESS_KEY="$LIVE_SECRET"
[ -n "$LIVE_REGION" ] && export AWS_DEFAULT_REGION="$LIVE_REGION"
live_dir=$(mktemp -d)
stamp="live-$(date -u +%s)-$$"
printf 'publish.sh live probe %s\n' "$stamp" > "$live_dir/probe.txt"
if bash "$here/publish.sh" "$live_dir" "$FDROID_CI_LIVE_BUCKET" \
"$FDROID_CI_LIVE_ENDPOINT" "$stamp" >/dev/null 2>&1; then
pass "publish.sh uploaded to a real bucket"
else
fail "publish.sh uploaded to a real bucket"
fi
got=$(bash "$here/with-nixpkgs.sh" awscli2 aws --endpoint-url "$FDROID_CI_LIVE_ENDPOINT" \
s3 cp "s3://$FDROID_CI_LIVE_BUCKET/$stamp/probe.txt" - 2>/dev/null || true)
if [ "$got" = "publish.sh live probe $stamp" ]; then
pass "the uploaded object reads back byte-identical"
else
fail "the uploaded object reads back byte-identical"
fi
bash "$here/with-nixpkgs.sh" awscli2 aws --endpoint-url "$FDROID_CI_LIVE_ENDPOINT" \
s3 rm --recursive "s3://$FDROID_CI_LIVE_BUCKET/$stamp/" >/dev/null 2>&1 || true
rm -rf "$live_dir"
else
skip "live round trip (set FDROID_CI_LIVE_BUCKET, FDROID_CI_LIVE_ENDPOINT and AWS credentials)"
fi
report_and_exit