claudeandbit
5e4b2fabc6
Measured against the real endpoint, not inferred: every PutObject failed
with
An error occurred (InvalidRequest) when calling the PutObject
operation: Bad request: invalid checksum algorithm
on aws-cli/2.34.24. Recent awscli2 attaches a checksum algorithm by
default and Garage refuses it, so publish.sh could not upload anything --
after passing its entire test suite.
It also reads as a permissions problem, which is the expensive part: reads
succeed, writes fail, and it fails identically against a bucket the same
credentials demonstrably can write. Set here rather than documented,
because a caller should not need to know a client-side default to publish
to the object store this script exists for, and leaving it to the
environment makes it work on one machine and not another.
`when_required` keeps checksums where the protocol needs them, so this is
narrower than switching them off.
Adds the test that would have caught it: an opt-in live round trip that
uploads, reads back and deletes, skipped unless a scratch bucket and
credentials are present so the offline run is unchanged. Everything else in
this suite proves the command is well formed, which is a different claim
from a server accepting it, and this failure lived exactly in that gap.
The live block captures the real credentials at the top of the file,
before the deliberate fakes (AKIAFAKETESTKEY) are exported for the offline
tests -- otherwise it authenticates as the fake and fails every time, and a
live test that always fails gets disabled, which is worse than not having
one.
Verified both directions against the real bucket: with the fix, 11 passed /
0 skipped; with it removed, the two live assertions fail. Offline, 9 passed
/ 1 skipped.
Closes #3.
Co-authored-by: bit <bit@das-labor.org>
106 lines
5.1 KiB
Bash
Executable File
106 lines
5.1 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
#
|
|
# Tests for publish.sh -- the one script in this repo that needs real
|
|
# network to do its actual job (fdroid-repo-ci#1), so its happy path against
|
|
# a genuine bucket is out of scope here. What this asserts instead:
|
|
#
|
|
# * argument/credential validation, offline;
|
|
# * `--dry-run` builds the exact command a real run would use, offline;
|
|
# * a real invocation against a closed local port fails on the network call
|
|
# itself, not on argument parsing -- proving the command it built is
|
|
# well-formed, without needing a real bucket or reaching beyond loopback.
|
|
set -euo pipefail
|
|
here=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)
|
|
|
|
# Captured BEFORE the fake credentials below are exported. Those fakes are
|
|
# deliberate -- they keep the offline tests from ever reaching a real service
|
|
# -- but they also overwrite the real ones for the rest of this file, so the
|
|
# live round trip at the bottom has to hold on to the originals here or it
|
|
# would authenticate as AKIAFAKETESTKEY and fail every time. A live test that
|
|
# always fails gets disabled, which is worse than not having one.
|
|
LIVE_KEY_ID="${AWS_ACCESS_KEY_ID:-}"
|
|
LIVE_SECRET="${AWS_SECRET_ACCESS_KEY:-}"
|
|
LIVE_REGION="${AWS_DEFAULT_REGION:-}"
|
|
source "$here/test_lib.sh"
|
|
|
|
work=$(mktemp -d)
|
|
trap 'rm -rf "$work"' EXIT
|
|
mkdir -p "$work/repo"
|
|
echo '{}' > "$work/repo/index-v2.json"
|
|
|
|
echo "--- missing credentials are refused, not sent as empty ---"
|
|
assert_failure "publish.sh refuses with no AWS_ACCESS_KEY_ID" \
|
|
env -u AWS_ACCESS_KEY_ID -u AWS_SECRET_ACCESS_KEY \
|
|
bash "$here/publish.sh" "$work/repo" mybucket https://s3.example.invalid
|
|
|
|
export AWS_ACCESS_KEY_ID=AKIAFAKETESTKEY
|
|
export AWS_SECRET_ACCESS_KEY=fake-secret-not-real
|
|
|
|
echo "--- missing local directory is refused ---"
|
|
assert_failure "publish.sh refuses a non-existent local directory" \
|
|
bash "$here/publish.sh" "$work/does-not-exist" mybucket https://s3.example.invalid
|
|
|
|
echo "--- --dry-run builds the right command, touches no network ---"
|
|
out=$(bash "$here/publish.sh" --dry-run "$work/repo" mybucket https://s3.example.invalid myprefix)
|
|
assert_contains "dry-run names the destination bucket and prefix" "$out" "s3://mybucket/myprefix"
|
|
assert_contains "dry-run names the endpoint" "$out" "--endpoint-url https://s3.example.invalid"
|
|
assert_contains "dry-run goes through with-nixpkgs.sh, same as every other script" "$out" "with-nixpkgs.sh"
|
|
if echo "$out" | grep -q -- --delete; then
|
|
fail "dry-run does not silently add --delete (it did!)"
|
|
else
|
|
pass "dry-run does not silently add --delete"
|
|
fi
|
|
|
|
echo "--- --dry-run with no prefix omits the trailing slash entirely ---"
|
|
out_noprefix=$(bash "$here/publish.sh" --dry-run "$work/repo" mybucket https://s3.example.invalid)
|
|
assert_contains "no-prefix dry-run targets the bucket root" "$out_noprefix" "s3://mybucket "
|
|
|
|
echo "--- a real invocation fails on the network call, not on arguments ---"
|
|
# port 1 is a reserved, unassigned TCP port -- nothing listens there, so this
|
|
# never leaves the loopback interface and needs no real network access.
|
|
real_out=$(bash "$here/publish.sh" "$work/repo" mybucket "http://127.0.0.1:1" 2>&1) && rc=0 || rc=$?
|
|
if [ "${rc:-0}" -ne 0 ]; then pass "publish.sh against a closed port exits non-zero"; else
|
|
fail "publish.sh against a closed port exits non-zero (got 0)"
|
|
fi
|
|
assert_contains "the failure is a connection error, not an argument error" "$real_out" "Could not connect"
|
|
|
|
# --- live round trip, opt-in ------------------------------------------------
|
|
# Skipped unless a scratch bucket and credentials are present, so the default
|
|
# offline run is unchanged. This is the only test here that talks to a real
|
|
# server, and the only one that could have caught the Garage checksum
|
|
# rejection (#3): everything above proves the command is well formed, which is
|
|
# a different claim from a server accepting it.
|
|
if [ -n "${FDROID_CI_LIVE_BUCKET:-}" ] && [ -n "${FDROID_CI_LIVE_ENDPOINT:-}" ] \
|
|
&& [ -n "$LIVE_KEY_ID" ] && [ -n "$LIVE_SECRET" ]; then
|
|
echo "--- a real round trip against a real endpoint ---"
|
|
export AWS_ACCESS_KEY_ID="$LIVE_KEY_ID"
|
|
export AWS_SECRET_ACCESS_KEY="$LIVE_SECRET"
|
|
[ -n "$LIVE_REGION" ] && export AWS_DEFAULT_REGION="$LIVE_REGION"
|
|
live_dir=$(mktemp -d)
|
|
stamp="live-$(date -u +%s)-$$"
|
|
printf 'publish.sh live probe %s\n' "$stamp" > "$live_dir/probe.txt"
|
|
|
|
if bash "$here/publish.sh" "$live_dir" "$FDROID_CI_LIVE_BUCKET" \
|
|
"$FDROID_CI_LIVE_ENDPOINT" "$stamp" >/dev/null 2>&1; then
|
|
pass "publish.sh uploaded to a real bucket"
|
|
else
|
|
fail "publish.sh uploaded to a real bucket"
|
|
fi
|
|
|
|
got=$(bash "$here/with-nixpkgs.sh" awscli2 aws --endpoint-url "$FDROID_CI_LIVE_ENDPOINT" \
|
|
s3 cp "s3://$FDROID_CI_LIVE_BUCKET/$stamp/probe.txt" - 2>/dev/null || true)
|
|
if [ "$got" = "publish.sh live probe $stamp" ]; then
|
|
pass "the uploaded object reads back byte-identical"
|
|
else
|
|
fail "the uploaded object reads back byte-identical"
|
|
fi
|
|
|
|
bash "$here/with-nixpkgs.sh" awscli2 aws --endpoint-url "$FDROID_CI_LIVE_ENDPOINT" \
|
|
s3 rm --recursive "s3://$FDROID_CI_LIVE_BUCKET/$stamp/" >/dev/null 2>&1 || true
|
|
rm -rf "$live_dir"
|
|
else
|
|
skip "live round trip (set FDROID_CI_LIVE_BUCKET, FDROID_CI_LIVE_ENDPOINT and AWS credentials)"
|
|
fi
|
|
|
|
report_and_exit
|