From 4a25e3681a822fbdba465d706de297ec6f19c3c6 Mon Sep 17 00:00:00 2001 From: claude Date: Wed, 9 Sep 2026 20:23:16 +0000 Subject: [PATCH] Require FDROID_REPO_URL/FDROID_REPO_NAME so fdroid-update.sh cannot publish fdroidserver's placeholder The published repo advertised fdroidserver's own demo values (repo.address "https://MyFirstFDroidRepo.org/fdroid/repo", repo.name "My First F-Droid Repo Demo") because fdroid-update.sh wrote config.yml itself but never set repo_url/repo_name/repo_description, so fdroidserver fell back to its template. The "Add to F-Droid" button on the generated index.html is built from repo_url, which is how bit noticed. A publish that succeeds while advertising the wrong URL is silently wrong -- worse than a failed job. Add FDROID_REPO_URL/FDROID_REPO_NAME/FDROID_REPO_DESCRIPTION environment variables (not new positionals -- the script already has five plus pass-through extra args). The first two are now a hard requirement, naming the missing one; FDROID_REPO_DESCRIPTION stays optional. This is a breaking change to a public composite action's interface, acceptable only because weblib-viewer is the sole consumer and is updated in the same change (weblib-viewer#26). Also discovered while wiring this up and worth recording since it contradicts the "no /fdroid/repo suffix" assumption in weblib's working notes: fdroidserver's own common.py unconditionally refuses to run once config.yml sets repo_url at all, unless it ends in the literal suffix "/repo" -- there is no flag to disable that check, and the value is embedded verbatim as repo.address. So FDROID_REPO_URL has to end in /repo, and publish.sh's existing argument has to put the published tree under a URL path that actually ends in /repo to match, or fdroid update refuses to run at all (checked positive) / the address points at a path the bucket never serves (checked against the live fdroid-weblib-dev bucket layout). Extends test_fdroid_update.sh: config.yml carries the values, the resulting index-v2.json has the right repo.address/name/description (not fdroidserver's placeholder), a name with an embedded single quote survives the YAML round-trip, and the script fails when FDROID_REPO_URL or FDROID_REPO_NAME is absent or FDROID_REPO_URL lacks the /repo suffix. run_tests.sh: 104 passed / 4 skipped (was 98/4). Filed as fdroid-repo-ci#11. Co-authored-by: bit --- README.md | 47 +++++++++++++++++++++++++++++++++++++-- fdroid-update.sh | 51 ++++++++++++++++++++++++++++++++++++++++++- test_fdroid_update.sh | 45 +++++++++++++++++++++++++++++++++++++- 3 files changed, 139 insertions(+), 4 deletions(-) diff --git a/README.md b/README.md index c760cfd..a7a820b 100644 --- a/README.md +++ b/README.md @@ -79,6 +79,9 @@ only the repo index key, in the `fdroid-update.sh` step. - env: INDEX_KEYSTORE_PASSWORD: ${{ secrets.INDEX_KEYSTORE_PASSWORD }} + FDROID_REPO_URL: https://fdroid-weblib.s3w.dev.chaosbit.de/repo + FDROID_REPO_NAME: weblib + FDROID_REPO_DESCRIPTION: "weblib's F-Droid repository" run: | bash "${{ steps.ci.outputs.path }}/fdroid-update.sh" \ "$FDROID_REPO_ROOT" "$INDEX_KEYSTORE" repo-key INDEX_KEYSTORE_PASSWORD @@ -88,9 +91,13 @@ only the repo index key, in the `fdroid-update.sh` step. AWS_SECRET_ACCESS_KEY: ${{ secrets.PUBLISH_SECRET_ACCESS_KEY }} run: | bash "${{ steps.ci.outputs.path }}/publish.sh" \ - "$FDROID_REPO_ROOT/repo" "$BUCKET" "$S3_ENDPOINT_URL" + "$FDROID_REPO_ROOT/repo" "$BUCKET" "$S3_ENDPOINT_URL" repo ``` +The trailing `repo` is `publish.sh`'s `` argument -- it has to put +the tree under a URL path ending in `/repo` to match `FDROID_REPO_URL` above +(see "`FDROID_REPO_URL` must end in the literal suffix `/repo`" below). + **Use the full URL, `https://git.chaosbit.de/weblib/fdroid-repo-ci@main`, not `weblib/fdroid-repo-ci@main`.** Measured on weblib-ci (weblib-archive#44): the bare `owner/repo` form resolves against the instance's default actions @@ -111,6 +118,39 @@ index key is not the APK signing key" below) -- that way separating the two keys later is a value change on the consumer's side, not a rename of this convention. +**`FDROID_REPO_URL` and `FDROID_REPO_NAME` are required environment +variables, not arguments** -- unlike the password env var *names* above, +these are the actual values (not secrets), so they are passed directly +rather than through an indirection. `fdroid-update.sh` refuses to run +without them, naming the missing one (fdroid-repo-ci#11): a repo that +`fdroid update` builds without `repo_url`/`repo_name` set publishes +*successfully* while advertising fdroidserver's own placeholder template +(`MyFirstFDroidRepo.org`, "My First F-Droid Repo Demo") -- silently wrong, +which is worse than a failed job. `FDROID_REPO_DESCRIPTION` is optional and +has no fallback if omitted (fdroidserver's own default, which is the same +placeholder problem, one field narrower). + +**`FDROID_REPO_URL` must end in the literal suffix `/repo`.** This is not a +convention of this repo -- fdroidserver's own `common.py` refuses to run +*any* command once `config.yml` sets `repo_url` at all, unless it ends in +`/repo` (`repo_url needs to end with /repo`), and there is no flag to turn +that check off. Whatever value is set here is also embedded verbatim as +`repo.address` in `index-v2.json`, so it has to be **the actual URL path +`index-v2.json` is served under**, not merely a plausible-looking one: +confirm with `/index-v2.json` resolving, don't guess. + +Concretely, that means `publish.sh`'s own `` argument has to be used +to put the tree under a URL path ending in `/repo` -- e.g. publish with +prefix `repo` so the bucket serves `.../repo/index-v2.json`, and set +`FDROID_REPO_URL` to `https://./repo` to match. Publishing +to the bucket root with no prefix (as this repo's consumer did before this +was noticed) cannot satisfy fdroidserver's own requirement at all: any +`repo_url` set there either lacks the suffix (and `fdroid update` refuses to +run) or has the suffix and points at a path the bucket never actually +serves (and a client's "Add to F-Droid" 404s on `index-v2.json`) -- the +`/repo` prefix is not optional decoration, it is what makes `repo_url` +and the published tree agree at all. + ### The exception: a consumer whose build does not already sign `sign-apk.sh` is **not part of the flow above** and `action.yml` does not @@ -353,7 +393,10 @@ using fdroidserver's own `{env: VARNAME}` indirection for both passwords -- the exact mechanism its `common.py` implements *specifically so config.yml can be safely git-managed*: "it is a good idea to manage config.yml (WITHOUT PASSWORDS!) in git." A caller never has to know or hand-write fdroidserver's -config format. +config format. It also writes `repo_url`, `repo_name` and (if set) +`repo_description` from the environment variables of the same name, single- +quoted YAML scalars so an arbitrary value never needs the caller to think +about YAML escaping. ## Pinning nixpkgs diff --git a/fdroid-update.sh b/fdroid-update.sh index 646716f..4809191 100755 --- a/fdroid-update.sh +++ b/fdroid-update.sh @@ -23,6 +23,28 @@ # is omitted, the keystore password is reused as the key # password. # +# FDROID_REPO_URL and FDROID_REPO_NAME (environment variables, not +# arguments -- they are not secrets, but they are metadata that belongs +# beside the other config.yml fields this script already writes, not a sixth +# and seventh positional) are REQUIRED and this script refuses to run +# without them. Omitting them used to mean fdroidserver silently fell back to +# its own placeholder template ("MyFirstFDroidRepo.org", "My First F-Droid +# Repo Demo") -- a repo that publishes successfully while advertising +# someone else's URL and name, which is worse than failing loudly. +# FDROID_REPO_DESCRIPTION is optional. This is a breaking change to this +# script's interface (fdroid-repo-ci#11); acceptable only because +# weblib-viewer is this action's sole consumer and is updated in the same +# change. +# +# FDROID_REPO_URL MUST end in the literal suffix "/repo" -- fdroidserver +# enforces this itself (common.py: "repo_url needs to end with /repo"), +# unconditionally, whenever config.yml sets repo_url at all, and refuses to +# run otherwise. It is not this script's own rule and there is no way to +# turn it off, so the caller's publish step has to serve the tree under a +# URL path actually ending in /repo -- e.g. `publish.sh`'s own +# argument -- or the value embedded here (verbatim, as repo.address in +# index-v2.json) will not be where the index actually is. +# # Needs no network (fdroid-repo-ci#1) -- it only reads repo/ and signs. # Pins fdroidserver, the JDK (fdroidserver shells out to jarsigner/keytool # for the index signature) and apksigner (for the per-entry v1 signature in @@ -35,6 +57,9 @@ if [ "$#" -lt 4 ]; then cat >&2 <<'USAGE' usage: fdroid-update.sh \ [] [extra fdroid update args...] + +required environment variables: FDROID_REPO_URL, FDROID_REPO_NAME +optional environment variable: FDROID_REPO_DESCRIPTION USAGE exit 2 fi @@ -58,6 +83,23 @@ extra_args=("$@") : "${!ks_pass_env:?fdroid-update.sh: environment variable $ks_pass_env (keystore password) is not set}" : "${!key_pass_env:?fdroid-update.sh: environment variable $key_pass_env (key password) is not set}" +# Hard-required: an unset repo_url/repo_name used to mean fdroidserver fell +# back to its own placeholder template, which is a *successful* publish that +# advertises the wrong repo -- silently wrong, the worst outcome. Failing +# loudly, naming the variable, is strictly better than defaulting. +: "${FDROID_REPO_URL:?fdroid-update.sh: environment variable FDROID_REPO_URL (canonical repo URL, no trailing slash) is not set}" +: "${FDROID_REPO_NAME:?fdroid-update.sh: environment variable FDROID_REPO_NAME (repo display name) is not set}" + +# YAML single-quoted scalar: doubling an embedded single quote is the only +# escape it needs, and (unlike double-quoted) it does not interpret +# backslashes -- so this is safe for an arbitrary URL/name/description +# without having to know what characters fdroidserver's own parser accepts. +yaml_squote() { + local s=$1 + s=${s//\'/\'\'} + printf "'%s'" "$s" +} + # fdroidserver's config.yml supports "{env: VARNAME}" in place of any string # value specifically so a config that names a keystore and its passwords can # still be free of secrets -- fdroidserver's own common.py: "it is a good @@ -65,14 +107,21 @@ extra_args=("$@") # means a caller never has to hand-write fdroidserver's config format, and # this repo -- not every consumer -- is the one place that has to track it if # it changes. -cat > "$root/config.yml" < "$root/config.yml" chmod 600 "$root/config.yml" echo "fdroid-update.sh: running fdroid update in $root" diff --git a/test_fdroid_update.sh b/test_fdroid_update.sh index 0475540..b715ce5 100755 --- a/test_fdroid_update.sh +++ b/test_fdroid_update.sh @@ -67,8 +67,28 @@ echo "--- fault: missing password env var is refused ---" assert_failure "fdroid-update.sh refuses when the password env var is unset" \ env -u IDX_KS_PASS bash "$here/fdroid-update.sh" "$work" index.jks indexkey IDX_KS_PASS -echo "--- revert: a real run, needs no network ---" export IDX_KS_PASS=idxpass123 + +echo "--- fault: missing FDROID_REPO_URL is refused (fdroid-repo-ci#11) ---" +assert_failure "fdroid-update.sh refuses when FDROID_REPO_URL is unset" \ + env -u FDROID_REPO_URL FDROID_REPO_NAME="weblib" \ + bash "$here/fdroid-update.sh" "$work" index.jks indexkey IDX_KS_PASS + +echo "--- fault: missing FDROID_REPO_NAME is refused (fdroid-repo-ci#11) ---" +assert_failure "fdroid-update.sh refuses when FDROID_REPO_NAME is unset" \ + env -u FDROID_REPO_NAME FDROID_REPO_URL="https://example.invalid" \ + bash "$here/fdroid-update.sh" "$work" index.jks indexkey IDX_KS_PASS + +echo "--- fault: FDROID_REPO_URL not ending in /repo is refused by fdroidserver itself ---" +assert_failure "fdroid-update.sh (fdroidserver) refuses a repo_url without the /repo suffix" \ + env FDROID_REPO_URL="https://fdroid-repo-ci-test.example.invalid" \ + FDROID_REPO_NAME="fdroid-repo-ci test repo" \ + bash "$here/fdroid-update.sh" "$work" index.jks indexkey IDX_KS_PASS + +echo "--- revert: a real run, needs no network ---" +export FDROID_REPO_URL="https://fdroid-repo-ci-test.example.invalid/repo" +export FDROID_REPO_NAME="fdroid-repo-ci test repo" +export FDROID_REPO_DESCRIPTION="a throwaway repo built by test_fdroid_update.sh" assert_success "fdroid-update.sh runs fdroid update successfully" \ bash "$here/fdroid-update.sh" "$work" index.jks indexkey IDX_KS_PASS @@ -80,6 +100,17 @@ echo "--- index-v2.json is valid JSON, not just a non-empty file ---" assert_success "index-v2.json parses as JSON" \ bash "$here/with-nixpkgs.sh" python3 python3 -c "import json; json.load(open('repo/index-v2.json'))" +echo "--- index-v2.json carries the repo metadata, not fdroidserver's placeholder ---" +assert_success "repo.address is FDROID_REPO_URL, not MyFirstFDroidRepo.org" \ + bash "$here/with-nixpkgs.sh" python3 python3 -c " +import json +repo = json.load(open('repo/index-v2.json'))['repo'] +assert repo['address'] == 'https://fdroid-repo-ci-test.example.invalid/repo', repo['address'] +assert repo['name']['en-US'] == 'fdroid-repo-ci test repo', repo['name'] +assert repo['description']['en-US'] == 'a throwaway repo built by test_fdroid_update.sh', repo['description'] +assert 'MyFirstFDroidRepo' not in json.dumps(repo) +" + echo "--- index-v1.jar is a validly signed jar (its own signature, not apksigner's) ---" assert_success "jarsigner -verify accepts the signed index" \ bash "$here/with-nixpkgs.sh" jdk21 jarsigner -verify repo/index-v1.jar @@ -88,4 +119,16 @@ echo "--- config.yml never holds a literal password ---" assert_failure "config.yml has no literal password (uses {env: ...} instead)" \ grep -q "idxpass123" "$work/config.yml" +echo "--- a value with an embedded single quote survives config.yml round-trip ---" +export FDROID_REPO_NAME="bit's test repo" +assert_success "fdroid-update.sh accepts a name with an embedded quote" \ + bash "$here/fdroid-update.sh" "$work" index.jks indexkey IDX_KS_PASS +assert_success "the quoted name round-trips through config.yml into index-v2.json" \ + bash "$here/with-nixpkgs.sh" python3 python3 -c " +import json +repo = json.load(open('repo/index-v2.json'))['repo'] +assert repo['name']['en-US'] == \"bit's test repo\", repo['name'] +" +export FDROID_REPO_NAME="fdroid-repo-ci test repo" + report_and_exit