Require FDROID_REPO_URL/FDROID_REPO_NAME in fdroid-update.sh #12

Merged
bit merged 1 commits from fdroid-repo-metadata into main 2026-09-09 20:31:49 +00:00
3 changed files with 139 additions and 4 deletions
Showing only changes of commit 4a25e3681a - Show all commits
+45 -2
View File
@@ -79,6 +79,9 @@ only the repo index key, in the `fdroid-update.sh` step.
- env: - env:
INDEX_KEYSTORE_PASSWORD: ${{ secrets.INDEX_KEYSTORE_PASSWORD }} INDEX_KEYSTORE_PASSWORD: ${{ secrets.INDEX_KEYSTORE_PASSWORD }}
FDROID_REPO_URL: https://fdroid-weblib.s3w.dev.chaosbit.de/repo
FDROID_REPO_NAME: weblib
FDROID_REPO_DESCRIPTION: "weblib's F-Droid repository"
run: | run: |
bash "${{ steps.ci.outputs.path }}/fdroid-update.sh" \ bash "${{ steps.ci.outputs.path }}/fdroid-update.sh" \
"$FDROID_REPO_ROOT" "$INDEX_KEYSTORE" repo-key INDEX_KEYSTORE_PASSWORD "$FDROID_REPO_ROOT" "$INDEX_KEYSTORE" repo-key INDEX_KEYSTORE_PASSWORD
@@ -88,9 +91,13 @@ only the repo index key, in the `fdroid-update.sh` step.
AWS_SECRET_ACCESS_KEY: ${{ secrets.PUBLISH_SECRET_ACCESS_KEY }} AWS_SECRET_ACCESS_KEY: ${{ secrets.PUBLISH_SECRET_ACCESS_KEY }}
run: | run: |
bash "${{ steps.ci.outputs.path }}/publish.sh" \ bash "${{ steps.ci.outputs.path }}/publish.sh" \
"$FDROID_REPO_ROOT/repo" "$BUCKET" "$S3_ENDPOINT_URL" "$FDROID_REPO_ROOT/repo" "$BUCKET" "$S3_ENDPOINT_URL" repo
``` ```
The trailing `repo` is `publish.sh`'s `<prefix>` argument -- it has to put
the tree under a URL path ending in `/repo` to match `FDROID_REPO_URL` above
(see "`FDROID_REPO_URL` must end in the literal suffix `/repo`" below).
**Use the full URL, `https://git.chaosbit.de/weblib/fdroid-repo-ci@main`, not **Use the full URL, `https://git.chaosbit.de/weblib/fdroid-repo-ci@main`, not
`weblib/fdroid-repo-ci@main`.** Measured on weblib-ci (weblib-archive#44): `weblib/fdroid-repo-ci@main`.** Measured on weblib-ci (weblib-archive#44):
the bare `owner/repo` form resolves against the instance's default actions the bare `owner/repo` form resolves against the instance's default actions
@@ -111,6 +118,39 @@ index key is not the APK signing key" below) -- that way separating the two
keys later is a value change on the consumer's side, not a rename of this keys later is a value change on the consumer's side, not a rename of this
convention. convention.
**`FDROID_REPO_URL` and `FDROID_REPO_NAME` are required environment
variables, not arguments** -- unlike the password env var *names* above,
these are the actual values (not secrets), so they are passed directly
rather than through an indirection. `fdroid-update.sh` refuses to run
without them, naming the missing one (fdroid-repo-ci#11): a repo that
`fdroid update` builds without `repo_url`/`repo_name` set publishes
*successfully* while advertising fdroidserver's own placeholder template
(`MyFirstFDroidRepo.org`, "My First F-Droid Repo Demo") -- silently wrong,
which is worse than a failed job. `FDROID_REPO_DESCRIPTION` is optional and
has no fallback if omitted (fdroidserver's own default, which is the same
placeholder problem, one field narrower).
**`FDROID_REPO_URL` must end in the literal suffix `/repo`.** This is not a
convention of this repo -- fdroidserver's own `common.py` refuses to run
*any* command once `config.yml` sets `repo_url` at all, unless it ends in
`/repo` (`repo_url needs to end with /repo`), and there is no flag to turn
that check off. Whatever value is set here is also embedded verbatim as
`repo.address` in `index-v2.json`, so it has to be **the actual URL path
`index-v2.json` is served under**, not merely a plausible-looking one:
confirm with `<repo_url>/index-v2.json` resolving, don't guess.
Concretely, that means `publish.sh`'s own `<prefix>` argument has to be used
to put the tree under a URL path ending in `/repo` -- e.g. publish with
prefix `repo` so the bucket serves `.../repo/index-v2.json`, and set
`FDROID_REPO_URL` to `https://<bucket>.<domain>/repo` to match. Publishing
to the bucket root with no prefix (as this repo's consumer did before this
was noticed) cannot satisfy fdroidserver's own requirement at all: any
`repo_url` set there either lacks the suffix (and `fdroid update` refuses to
run) or has the suffix and points at a path the bucket never actually
serves (and a client's "Add to F-Droid" 404s on `index-v2.json`) -- the
`/repo` prefix is not optional decoration, it is what makes `repo_url`
and the published tree agree at all.
### The exception: a consumer whose build does not already sign ### The exception: a consumer whose build does not already sign
`sign-apk.sh` is **not part of the flow above** and `action.yml` does not `sign-apk.sh` is **not part of the flow above** and `action.yml` does not
@@ -353,7 +393,10 @@ using fdroidserver's own `{env: VARNAME}` indirection for both passwords --
the exact mechanism its `common.py` implements *specifically so config.yml the exact mechanism its `common.py` implements *specifically so config.yml
can be safely git-managed*: "it is a good idea to manage config.yml (WITHOUT can be safely git-managed*: "it is a good idea to manage config.yml (WITHOUT
PASSWORDS!) in git." A caller never has to know or hand-write fdroidserver's PASSWORDS!) in git." A caller never has to know or hand-write fdroidserver's
config format. config format. It also writes `repo_url`, `repo_name` and (if set)
`repo_description` from the environment variables of the same name, single-
quoted YAML scalars so an arbitrary value never needs the caller to think
about YAML escaping.
## Pinning nixpkgs ## Pinning nixpkgs
+50 -1
View File
@@ -23,6 +23,28 @@
# <key-pass-env> is omitted, the keystore password is reused as the key # <key-pass-env> is omitted, the keystore password is reused as the key
# password. # password.
# #
# FDROID_REPO_URL and FDROID_REPO_NAME (environment variables, not
# arguments -- they are not secrets, but they are metadata that belongs
# beside the other config.yml fields this script already writes, not a sixth
# and seventh positional) are REQUIRED and this script refuses to run
# without them. Omitting them used to mean fdroidserver silently fell back to
# its own placeholder template ("MyFirstFDroidRepo.org", "My First F-Droid
# Repo Demo") -- a repo that publishes successfully while advertising
# someone else's URL and name, which is worse than failing loudly.
# FDROID_REPO_DESCRIPTION is optional. This is a breaking change to this
# script's interface (fdroid-repo-ci#11); acceptable only because
# weblib-viewer is this action's sole consumer and is updated in the same
# change.
#
# FDROID_REPO_URL MUST end in the literal suffix "/repo" -- fdroidserver
# enforces this itself (common.py: "repo_url needs to end with /repo"),
# unconditionally, whenever config.yml sets repo_url at all, and refuses to
# run otherwise. It is not this script's own rule and there is no way to
# turn it off, so the caller's publish step has to serve the tree under a
# URL path actually ending in /repo -- e.g. `publish.sh`'s own <prefix>
# argument -- or the value embedded here (verbatim, as repo.address in
# index-v2.json) will not be where the index actually is.
#
# Needs no network (fdroid-repo-ci#1) -- it only reads repo/ and signs. # Needs no network (fdroid-repo-ci#1) -- it only reads repo/ and signs.
# Pins fdroidserver, the JDK (fdroidserver shells out to jarsigner/keytool # Pins fdroidserver, the JDK (fdroidserver shells out to jarsigner/keytool
# for the index signature) and apksigner (for the per-entry v1 signature in # for the index signature) and apksigner (for the per-entry v1 signature in
@@ -35,6 +57,9 @@ if [ "$#" -lt 4 ]; then
cat >&2 <<'USAGE' cat >&2 <<'USAGE'
usage: fdroid-update.sh <repo-root> <index-keystore> <key-alias> \ usage: fdroid-update.sh <repo-root> <index-keystore> <key-alias> \
<keystore-pass-env> [<key-pass-env>] [extra fdroid update args...] <keystore-pass-env> [<key-pass-env>] [extra fdroid update args...]
required environment variables: FDROID_REPO_URL, FDROID_REPO_NAME
optional environment variable: FDROID_REPO_DESCRIPTION
USAGE USAGE
exit 2 exit 2
fi fi
@@ -58,6 +83,23 @@ extra_args=("$@")
: "${!ks_pass_env:?fdroid-update.sh: environment variable $ks_pass_env (keystore password) is not set}" : "${!ks_pass_env:?fdroid-update.sh: environment variable $ks_pass_env (keystore password) is not set}"
: "${!key_pass_env:?fdroid-update.sh: environment variable $key_pass_env (key password) is not set}" : "${!key_pass_env:?fdroid-update.sh: environment variable $key_pass_env (key password) is not set}"
# Hard-required: an unset repo_url/repo_name used to mean fdroidserver fell
# back to its own placeholder template, which is a *successful* publish that
# advertises the wrong repo -- silently wrong, the worst outcome. Failing
# loudly, naming the variable, is strictly better than defaulting.
: "${FDROID_REPO_URL:?fdroid-update.sh: environment variable FDROID_REPO_URL (canonical repo URL, no trailing slash) is not set}"
: "${FDROID_REPO_NAME:?fdroid-update.sh: environment variable FDROID_REPO_NAME (repo display name) is not set}"
# YAML single-quoted scalar: doubling an embedded single quote is the only
# escape it needs, and (unlike double-quoted) it does not interpret
# backslashes -- so this is safe for an arbitrary URL/name/description
# without having to know what characters fdroidserver's own parser accepts.
yaml_squote() {
local s=$1
s=${s//\'/\'\'}
printf "'%s'" "$s"
}
# fdroidserver's config.yml supports "{env: VARNAME}" in place of any string # fdroidserver's config.yml supports "{env: VARNAME}" in place of any string
# value specifically so a config that names a keystore and its passwords can # value specifically so a config that names a keystore and its passwords can
# still be free of secrets -- fdroidserver's own common.py: "it is a good # still be free of secrets -- fdroidserver's own common.py: "it is a good
@@ -65,14 +107,21 @@ extra_args=("$@")
# means a caller never has to hand-write fdroidserver's config format, and # means a caller never has to hand-write fdroidserver's config format, and
# this repo -- not every consumer -- is the one place that has to track it if # this repo -- not every consumer -- is the one place that has to track it if
# it changes. # it changes.
cat > "$root/config.yml" <<EOF {
cat <<EOF
keystore: $keystore keystore: $keystore
keystorepass: keystorepass:
env: $ks_pass_env env: $ks_pass_env
keypass: keypass:
env: $key_pass_env env: $key_pass_env
repo_keyalias: $alias_name repo_keyalias: $alias_name
repo_url: $(yaml_squote "$FDROID_REPO_URL")
repo_name: $(yaml_squote "$FDROID_REPO_NAME")
EOF EOF
if [ -n "${FDROID_REPO_DESCRIPTION:-}" ]; then
echo "repo_description: $(yaml_squote "$FDROID_REPO_DESCRIPTION")"
fi
} > "$root/config.yml"
chmod 600 "$root/config.yml" chmod 600 "$root/config.yml"
echo "fdroid-update.sh: running fdroid update in $root" echo "fdroid-update.sh: running fdroid update in $root"
+44 -1
View File
@@ -67,8 +67,28 @@ echo "--- fault: missing password env var is refused ---"
assert_failure "fdroid-update.sh refuses when the password env var is unset" \ assert_failure "fdroid-update.sh refuses when the password env var is unset" \
env -u IDX_KS_PASS bash "$here/fdroid-update.sh" "$work" index.jks indexkey IDX_KS_PASS env -u IDX_KS_PASS bash "$here/fdroid-update.sh" "$work" index.jks indexkey IDX_KS_PASS
echo "--- revert: a real run, needs no network ---"
export IDX_KS_PASS=idxpass123 export IDX_KS_PASS=idxpass123
echo "--- fault: missing FDROID_REPO_URL is refused (fdroid-repo-ci#11) ---"
assert_failure "fdroid-update.sh refuses when FDROID_REPO_URL is unset" \
env -u FDROID_REPO_URL FDROID_REPO_NAME="weblib" \
bash "$here/fdroid-update.sh" "$work" index.jks indexkey IDX_KS_PASS
echo "--- fault: missing FDROID_REPO_NAME is refused (fdroid-repo-ci#11) ---"
assert_failure "fdroid-update.sh refuses when FDROID_REPO_NAME is unset" \
env -u FDROID_REPO_NAME FDROID_REPO_URL="https://example.invalid" \
bash "$here/fdroid-update.sh" "$work" index.jks indexkey IDX_KS_PASS
echo "--- fault: FDROID_REPO_URL not ending in /repo is refused by fdroidserver itself ---"
assert_failure "fdroid-update.sh (fdroidserver) refuses a repo_url without the /repo suffix" \
env FDROID_REPO_URL="https://fdroid-repo-ci-test.example.invalid" \
FDROID_REPO_NAME="fdroid-repo-ci test repo" \
bash "$here/fdroid-update.sh" "$work" index.jks indexkey IDX_KS_PASS
echo "--- revert: a real run, needs no network ---"
export FDROID_REPO_URL="https://fdroid-repo-ci-test.example.invalid/repo"
export FDROID_REPO_NAME="fdroid-repo-ci test repo"
export FDROID_REPO_DESCRIPTION="a throwaway repo built by test_fdroid_update.sh"
assert_success "fdroid-update.sh runs fdroid update successfully" \ assert_success "fdroid-update.sh runs fdroid update successfully" \
bash "$here/fdroid-update.sh" "$work" index.jks indexkey IDX_KS_PASS bash "$here/fdroid-update.sh" "$work" index.jks indexkey IDX_KS_PASS
@@ -80,6 +100,17 @@ echo "--- index-v2.json is valid JSON, not just a non-empty file ---"
assert_success "index-v2.json parses as JSON" \ assert_success "index-v2.json parses as JSON" \
bash "$here/with-nixpkgs.sh" python3 python3 -c "import json; json.load(open('repo/index-v2.json'))" bash "$here/with-nixpkgs.sh" python3 python3 -c "import json; json.load(open('repo/index-v2.json'))"
echo "--- index-v2.json carries the repo metadata, not fdroidserver's placeholder ---"
assert_success "repo.address is FDROID_REPO_URL, not MyFirstFDroidRepo.org" \
bash "$here/with-nixpkgs.sh" python3 python3 -c "
import json
repo = json.load(open('repo/index-v2.json'))['repo']
assert repo['address'] == 'https://fdroid-repo-ci-test.example.invalid/repo', repo['address']
assert repo['name']['en-US'] == 'fdroid-repo-ci test repo', repo['name']
assert repo['description']['en-US'] == 'a throwaway repo built by test_fdroid_update.sh', repo['description']
assert 'MyFirstFDroidRepo' not in json.dumps(repo)
"
echo "--- index-v1.jar is a validly signed jar (its own signature, not apksigner's) ---" echo "--- index-v1.jar is a validly signed jar (its own signature, not apksigner's) ---"
assert_success "jarsigner -verify accepts the signed index" \ assert_success "jarsigner -verify accepts the signed index" \
bash "$here/with-nixpkgs.sh" jdk21 jarsigner -verify repo/index-v1.jar bash "$here/with-nixpkgs.sh" jdk21 jarsigner -verify repo/index-v1.jar
@@ -88,4 +119,16 @@ echo "--- config.yml never holds a literal password ---"
assert_failure "config.yml has no literal password (uses {env: ...} instead)" \ assert_failure "config.yml has no literal password (uses {env: ...} instead)" \
grep -q "idxpass123" "$work/config.yml" grep -q "idxpass123" "$work/config.yml"
echo "--- a value with an embedded single quote survives config.yml round-trip ---"
export FDROID_REPO_NAME="bit's test repo"
assert_success "fdroid-update.sh accepts a name with an embedded quote" \
bash "$here/fdroid-update.sh" "$work" index.jks indexkey IDX_KS_PASS
assert_success "the quoted name round-trips through config.yml into index-v2.json" \
bash "$here/with-nixpkgs.sh" python3 python3 -c "
import json
repo = json.load(open('repo/index-v2.json'))['repo']
assert repo['name']['en-US'] == \"bit's test repo\", repo['name']
"
export FDROID_REPO_NAME="fdroid-repo-ci test repo"
report_and_exit report_and_exit