Require FDROID_REPO_URL/FDROID_REPO_NAME in fdroid-update.sh #12
@@ -79,6 +79,9 @@ only the repo index key, in the `fdroid-update.sh` step.
|
|||||||
|
|
||||||
- env:
|
- env:
|
||||||
INDEX_KEYSTORE_PASSWORD: ${{ secrets.INDEX_KEYSTORE_PASSWORD }}
|
INDEX_KEYSTORE_PASSWORD: ${{ secrets.INDEX_KEYSTORE_PASSWORD }}
|
||||||
|
FDROID_REPO_URL: https://fdroid-weblib.s3w.dev.chaosbit.de/repo
|
||||||
|
FDROID_REPO_NAME: weblib
|
||||||
|
FDROID_REPO_DESCRIPTION: "weblib's F-Droid repository"
|
||||||
run: |
|
run: |
|
||||||
bash "${{ steps.ci.outputs.path }}/fdroid-update.sh" \
|
bash "${{ steps.ci.outputs.path }}/fdroid-update.sh" \
|
||||||
"$FDROID_REPO_ROOT" "$INDEX_KEYSTORE" repo-key INDEX_KEYSTORE_PASSWORD
|
"$FDROID_REPO_ROOT" "$INDEX_KEYSTORE" repo-key INDEX_KEYSTORE_PASSWORD
|
||||||
@@ -88,9 +91,13 @@ only the repo index key, in the `fdroid-update.sh` step.
|
|||||||
AWS_SECRET_ACCESS_KEY: ${{ secrets.PUBLISH_SECRET_ACCESS_KEY }}
|
AWS_SECRET_ACCESS_KEY: ${{ secrets.PUBLISH_SECRET_ACCESS_KEY }}
|
||||||
run: |
|
run: |
|
||||||
bash "${{ steps.ci.outputs.path }}/publish.sh" \
|
bash "${{ steps.ci.outputs.path }}/publish.sh" \
|
||||||
"$FDROID_REPO_ROOT/repo" "$BUCKET" "$S3_ENDPOINT_URL"
|
"$FDROID_REPO_ROOT/repo" "$BUCKET" "$S3_ENDPOINT_URL" repo
|
||||||
```
|
```
|
||||||
|
|
||||||
|
The trailing `repo` is `publish.sh`'s `<prefix>` argument -- it has to put
|
||||||
|
the tree under a URL path ending in `/repo` to match `FDROID_REPO_URL` above
|
||||||
|
(see "`FDROID_REPO_URL` must end in the literal suffix `/repo`" below).
|
||||||
|
|
||||||
**Use the full URL, `https://git.chaosbit.de/weblib/fdroid-repo-ci@main`, not
|
**Use the full URL, `https://git.chaosbit.de/weblib/fdroid-repo-ci@main`, not
|
||||||
`weblib/fdroid-repo-ci@main`.** Measured on weblib-ci (weblib-archive#44):
|
`weblib/fdroid-repo-ci@main`.** Measured on weblib-ci (weblib-archive#44):
|
||||||
the bare `owner/repo` form resolves against the instance's default actions
|
the bare `owner/repo` form resolves against the instance's default actions
|
||||||
@@ -111,6 +118,39 @@ index key is not the APK signing key" below) -- that way separating the two
|
|||||||
keys later is a value change on the consumer's side, not a rename of this
|
keys later is a value change on the consumer's side, not a rename of this
|
||||||
convention.
|
convention.
|
||||||
|
|
||||||
|
**`FDROID_REPO_URL` and `FDROID_REPO_NAME` are required environment
|
||||||
|
variables, not arguments** -- unlike the password env var *names* above,
|
||||||
|
these are the actual values (not secrets), so they are passed directly
|
||||||
|
rather than through an indirection. `fdroid-update.sh` refuses to run
|
||||||
|
without them, naming the missing one (fdroid-repo-ci#11): a repo that
|
||||||
|
`fdroid update` builds without `repo_url`/`repo_name` set publishes
|
||||||
|
*successfully* while advertising fdroidserver's own placeholder template
|
||||||
|
(`MyFirstFDroidRepo.org`, "My First F-Droid Repo Demo") -- silently wrong,
|
||||||
|
which is worse than a failed job. `FDROID_REPO_DESCRIPTION` is optional and
|
||||||
|
has no fallback if omitted (fdroidserver's own default, which is the same
|
||||||
|
placeholder problem, one field narrower).
|
||||||
|
|
||||||
|
**`FDROID_REPO_URL` must end in the literal suffix `/repo`.** This is not a
|
||||||
|
convention of this repo -- fdroidserver's own `common.py` refuses to run
|
||||||
|
*any* command once `config.yml` sets `repo_url` at all, unless it ends in
|
||||||
|
`/repo` (`repo_url needs to end with /repo`), and there is no flag to turn
|
||||||
|
that check off. Whatever value is set here is also embedded verbatim as
|
||||||
|
`repo.address` in `index-v2.json`, so it has to be **the actual URL path
|
||||||
|
`index-v2.json` is served under**, not merely a plausible-looking one:
|
||||||
|
confirm with `<repo_url>/index-v2.json` resolving, don't guess.
|
||||||
|
|
||||||
|
Concretely, that means `publish.sh`'s own `<prefix>` argument has to be used
|
||||||
|
to put the tree under a URL path ending in `/repo` -- e.g. publish with
|
||||||
|
prefix `repo` so the bucket serves `.../repo/index-v2.json`, and set
|
||||||
|
`FDROID_REPO_URL` to `https://<bucket>.<domain>/repo` to match. Publishing
|
||||||
|
to the bucket root with no prefix (as this repo's consumer did before this
|
||||||
|
was noticed) cannot satisfy fdroidserver's own requirement at all: any
|
||||||
|
`repo_url` set there either lacks the suffix (and `fdroid update` refuses to
|
||||||
|
run) or has the suffix and points at a path the bucket never actually
|
||||||
|
serves (and a client's "Add to F-Droid" 404s on `index-v2.json`) -- the
|
||||||
|
`/repo` prefix is not optional decoration, it is what makes `repo_url`
|
||||||
|
and the published tree agree at all.
|
||||||
|
|
||||||
### The exception: a consumer whose build does not already sign
|
### The exception: a consumer whose build does not already sign
|
||||||
|
|
||||||
`sign-apk.sh` is **not part of the flow above** and `action.yml` does not
|
`sign-apk.sh` is **not part of the flow above** and `action.yml` does not
|
||||||
@@ -353,7 +393,10 @@ using fdroidserver's own `{env: VARNAME}` indirection for both passwords --
|
|||||||
the exact mechanism its `common.py` implements *specifically so config.yml
|
the exact mechanism its `common.py` implements *specifically so config.yml
|
||||||
can be safely git-managed*: "it is a good idea to manage config.yml (WITHOUT
|
can be safely git-managed*: "it is a good idea to manage config.yml (WITHOUT
|
||||||
PASSWORDS!) in git." A caller never has to know or hand-write fdroidserver's
|
PASSWORDS!) in git." A caller never has to know or hand-write fdroidserver's
|
||||||
config format.
|
config format. It also writes `repo_url`, `repo_name` and (if set)
|
||||||
|
`repo_description` from the environment variables of the same name, single-
|
||||||
|
quoted YAML scalars so an arbitrary value never needs the caller to think
|
||||||
|
about YAML escaping.
|
||||||
|
|
||||||
## Pinning nixpkgs
|
## Pinning nixpkgs
|
||||||
|
|
||||||
|
|||||||
+50
-1
@@ -23,6 +23,28 @@
|
|||||||
# <key-pass-env> is omitted, the keystore password is reused as the key
|
# <key-pass-env> is omitted, the keystore password is reused as the key
|
||||||
# password.
|
# password.
|
||||||
#
|
#
|
||||||
|
# FDROID_REPO_URL and FDROID_REPO_NAME (environment variables, not
|
||||||
|
# arguments -- they are not secrets, but they are metadata that belongs
|
||||||
|
# beside the other config.yml fields this script already writes, not a sixth
|
||||||
|
# and seventh positional) are REQUIRED and this script refuses to run
|
||||||
|
# without them. Omitting them used to mean fdroidserver silently fell back to
|
||||||
|
# its own placeholder template ("MyFirstFDroidRepo.org", "My First F-Droid
|
||||||
|
# Repo Demo") -- a repo that publishes successfully while advertising
|
||||||
|
# someone else's URL and name, which is worse than failing loudly.
|
||||||
|
# FDROID_REPO_DESCRIPTION is optional. This is a breaking change to this
|
||||||
|
# script's interface (fdroid-repo-ci#11); acceptable only because
|
||||||
|
# weblib-viewer is this action's sole consumer and is updated in the same
|
||||||
|
# change.
|
||||||
|
#
|
||||||
|
# FDROID_REPO_URL MUST end in the literal suffix "/repo" -- fdroidserver
|
||||||
|
# enforces this itself (common.py: "repo_url needs to end with /repo"),
|
||||||
|
# unconditionally, whenever config.yml sets repo_url at all, and refuses to
|
||||||
|
# run otherwise. It is not this script's own rule and there is no way to
|
||||||
|
# turn it off, so the caller's publish step has to serve the tree under a
|
||||||
|
# URL path actually ending in /repo -- e.g. `publish.sh`'s own <prefix>
|
||||||
|
# argument -- or the value embedded here (verbatim, as repo.address in
|
||||||
|
# index-v2.json) will not be where the index actually is.
|
||||||
|
#
|
||||||
# Needs no network (fdroid-repo-ci#1) -- it only reads repo/ and signs.
|
# Needs no network (fdroid-repo-ci#1) -- it only reads repo/ and signs.
|
||||||
# Pins fdroidserver, the JDK (fdroidserver shells out to jarsigner/keytool
|
# Pins fdroidserver, the JDK (fdroidserver shells out to jarsigner/keytool
|
||||||
# for the index signature) and apksigner (for the per-entry v1 signature in
|
# for the index signature) and apksigner (for the per-entry v1 signature in
|
||||||
@@ -35,6 +57,9 @@ if [ "$#" -lt 4 ]; then
|
|||||||
cat >&2 <<'USAGE'
|
cat >&2 <<'USAGE'
|
||||||
usage: fdroid-update.sh <repo-root> <index-keystore> <key-alias> \
|
usage: fdroid-update.sh <repo-root> <index-keystore> <key-alias> \
|
||||||
<keystore-pass-env> [<key-pass-env>] [extra fdroid update args...]
|
<keystore-pass-env> [<key-pass-env>] [extra fdroid update args...]
|
||||||
|
|
||||||
|
required environment variables: FDROID_REPO_URL, FDROID_REPO_NAME
|
||||||
|
optional environment variable: FDROID_REPO_DESCRIPTION
|
||||||
USAGE
|
USAGE
|
||||||
exit 2
|
exit 2
|
||||||
fi
|
fi
|
||||||
@@ -58,6 +83,23 @@ extra_args=("$@")
|
|||||||
: "${!ks_pass_env:?fdroid-update.sh: environment variable $ks_pass_env (keystore password) is not set}"
|
: "${!ks_pass_env:?fdroid-update.sh: environment variable $ks_pass_env (keystore password) is not set}"
|
||||||
: "${!key_pass_env:?fdroid-update.sh: environment variable $key_pass_env (key password) is not set}"
|
: "${!key_pass_env:?fdroid-update.sh: environment variable $key_pass_env (key password) is not set}"
|
||||||
|
|
||||||
|
# Hard-required: an unset repo_url/repo_name used to mean fdroidserver fell
|
||||||
|
# back to its own placeholder template, which is a *successful* publish that
|
||||||
|
# advertises the wrong repo -- silently wrong, the worst outcome. Failing
|
||||||
|
# loudly, naming the variable, is strictly better than defaulting.
|
||||||
|
: "${FDROID_REPO_URL:?fdroid-update.sh: environment variable FDROID_REPO_URL (canonical repo URL, no trailing slash) is not set}"
|
||||||
|
: "${FDROID_REPO_NAME:?fdroid-update.sh: environment variable FDROID_REPO_NAME (repo display name) is not set}"
|
||||||
|
|
||||||
|
# YAML single-quoted scalar: doubling an embedded single quote is the only
|
||||||
|
# escape it needs, and (unlike double-quoted) it does not interpret
|
||||||
|
# backslashes -- so this is safe for an arbitrary URL/name/description
|
||||||
|
# without having to know what characters fdroidserver's own parser accepts.
|
||||||
|
yaml_squote() {
|
||||||
|
local s=$1
|
||||||
|
s=${s//\'/\'\'}
|
||||||
|
printf "'%s'" "$s"
|
||||||
|
}
|
||||||
|
|
||||||
# fdroidserver's config.yml supports "{env: VARNAME}" in place of any string
|
# fdroidserver's config.yml supports "{env: VARNAME}" in place of any string
|
||||||
# value specifically so a config that names a keystore and its passwords can
|
# value specifically so a config that names a keystore and its passwords can
|
||||||
# still be free of secrets -- fdroidserver's own common.py: "it is a good
|
# still be free of secrets -- fdroidserver's own common.py: "it is a good
|
||||||
@@ -65,14 +107,21 @@ extra_args=("$@")
|
|||||||
# means a caller never has to hand-write fdroidserver's config format, and
|
# means a caller never has to hand-write fdroidserver's config format, and
|
||||||
# this repo -- not every consumer -- is the one place that has to track it if
|
# this repo -- not every consumer -- is the one place that has to track it if
|
||||||
# it changes.
|
# it changes.
|
||||||
cat > "$root/config.yml" <<EOF
|
{
|
||||||
|
cat <<EOF
|
||||||
keystore: $keystore
|
keystore: $keystore
|
||||||
keystorepass:
|
keystorepass:
|
||||||
env: $ks_pass_env
|
env: $ks_pass_env
|
||||||
keypass:
|
keypass:
|
||||||
env: $key_pass_env
|
env: $key_pass_env
|
||||||
repo_keyalias: $alias_name
|
repo_keyalias: $alias_name
|
||||||
|
repo_url: $(yaml_squote "$FDROID_REPO_URL")
|
||||||
|
repo_name: $(yaml_squote "$FDROID_REPO_NAME")
|
||||||
EOF
|
EOF
|
||||||
|
if [ -n "${FDROID_REPO_DESCRIPTION:-}" ]; then
|
||||||
|
echo "repo_description: $(yaml_squote "$FDROID_REPO_DESCRIPTION")"
|
||||||
|
fi
|
||||||
|
} > "$root/config.yml"
|
||||||
chmod 600 "$root/config.yml"
|
chmod 600 "$root/config.yml"
|
||||||
|
|
||||||
echo "fdroid-update.sh: running fdroid update in $root"
|
echo "fdroid-update.sh: running fdroid update in $root"
|
||||||
|
|||||||
+44
-1
@@ -67,8 +67,28 @@ echo "--- fault: missing password env var is refused ---"
|
|||||||
assert_failure "fdroid-update.sh refuses when the password env var is unset" \
|
assert_failure "fdroid-update.sh refuses when the password env var is unset" \
|
||||||
env -u IDX_KS_PASS bash "$here/fdroid-update.sh" "$work" index.jks indexkey IDX_KS_PASS
|
env -u IDX_KS_PASS bash "$here/fdroid-update.sh" "$work" index.jks indexkey IDX_KS_PASS
|
||||||
|
|
||||||
echo "--- revert: a real run, needs no network ---"
|
|
||||||
export IDX_KS_PASS=idxpass123
|
export IDX_KS_PASS=idxpass123
|
||||||
|
|
||||||
|
echo "--- fault: missing FDROID_REPO_URL is refused (fdroid-repo-ci#11) ---"
|
||||||
|
assert_failure "fdroid-update.sh refuses when FDROID_REPO_URL is unset" \
|
||||||
|
env -u FDROID_REPO_URL FDROID_REPO_NAME="weblib" \
|
||||||
|
bash "$here/fdroid-update.sh" "$work" index.jks indexkey IDX_KS_PASS
|
||||||
|
|
||||||
|
echo "--- fault: missing FDROID_REPO_NAME is refused (fdroid-repo-ci#11) ---"
|
||||||
|
assert_failure "fdroid-update.sh refuses when FDROID_REPO_NAME is unset" \
|
||||||
|
env -u FDROID_REPO_NAME FDROID_REPO_URL="https://example.invalid" \
|
||||||
|
bash "$here/fdroid-update.sh" "$work" index.jks indexkey IDX_KS_PASS
|
||||||
|
|
||||||
|
echo "--- fault: FDROID_REPO_URL not ending in /repo is refused by fdroidserver itself ---"
|
||||||
|
assert_failure "fdroid-update.sh (fdroidserver) refuses a repo_url without the /repo suffix" \
|
||||||
|
env FDROID_REPO_URL="https://fdroid-repo-ci-test.example.invalid" \
|
||||||
|
FDROID_REPO_NAME="fdroid-repo-ci test repo" \
|
||||||
|
bash "$here/fdroid-update.sh" "$work" index.jks indexkey IDX_KS_PASS
|
||||||
|
|
||||||
|
echo "--- revert: a real run, needs no network ---"
|
||||||
|
export FDROID_REPO_URL="https://fdroid-repo-ci-test.example.invalid/repo"
|
||||||
|
export FDROID_REPO_NAME="fdroid-repo-ci test repo"
|
||||||
|
export FDROID_REPO_DESCRIPTION="a throwaway repo built by test_fdroid_update.sh"
|
||||||
assert_success "fdroid-update.sh runs fdroid update successfully" \
|
assert_success "fdroid-update.sh runs fdroid update successfully" \
|
||||||
bash "$here/fdroid-update.sh" "$work" index.jks indexkey IDX_KS_PASS
|
bash "$here/fdroid-update.sh" "$work" index.jks indexkey IDX_KS_PASS
|
||||||
|
|
||||||
@@ -80,6 +100,17 @@ echo "--- index-v2.json is valid JSON, not just a non-empty file ---"
|
|||||||
assert_success "index-v2.json parses as JSON" \
|
assert_success "index-v2.json parses as JSON" \
|
||||||
bash "$here/with-nixpkgs.sh" python3 python3 -c "import json; json.load(open('repo/index-v2.json'))"
|
bash "$here/with-nixpkgs.sh" python3 python3 -c "import json; json.load(open('repo/index-v2.json'))"
|
||||||
|
|
||||||
|
echo "--- index-v2.json carries the repo metadata, not fdroidserver's placeholder ---"
|
||||||
|
assert_success "repo.address is FDROID_REPO_URL, not MyFirstFDroidRepo.org" \
|
||||||
|
bash "$here/with-nixpkgs.sh" python3 python3 -c "
|
||||||
|
import json
|
||||||
|
repo = json.load(open('repo/index-v2.json'))['repo']
|
||||||
|
assert repo['address'] == 'https://fdroid-repo-ci-test.example.invalid/repo', repo['address']
|
||||||
|
assert repo['name']['en-US'] == 'fdroid-repo-ci test repo', repo['name']
|
||||||
|
assert repo['description']['en-US'] == 'a throwaway repo built by test_fdroid_update.sh', repo['description']
|
||||||
|
assert 'MyFirstFDroidRepo' not in json.dumps(repo)
|
||||||
|
"
|
||||||
|
|
||||||
echo "--- index-v1.jar is a validly signed jar (its own signature, not apksigner's) ---"
|
echo "--- index-v1.jar is a validly signed jar (its own signature, not apksigner's) ---"
|
||||||
assert_success "jarsigner -verify accepts the signed index" \
|
assert_success "jarsigner -verify accepts the signed index" \
|
||||||
bash "$here/with-nixpkgs.sh" jdk21 jarsigner -verify repo/index-v1.jar
|
bash "$here/with-nixpkgs.sh" jdk21 jarsigner -verify repo/index-v1.jar
|
||||||
@@ -88,4 +119,16 @@ echo "--- config.yml never holds a literal password ---"
|
|||||||
assert_failure "config.yml has no literal password (uses {env: ...} instead)" \
|
assert_failure "config.yml has no literal password (uses {env: ...} instead)" \
|
||||||
grep -q "idxpass123" "$work/config.yml"
|
grep -q "idxpass123" "$work/config.yml"
|
||||||
|
|
||||||
|
echo "--- a value with an embedded single quote survives config.yml round-trip ---"
|
||||||
|
export FDROID_REPO_NAME="bit's test repo"
|
||||||
|
assert_success "fdroid-update.sh accepts a name with an embedded quote" \
|
||||||
|
bash "$here/fdroid-update.sh" "$work" index.jks indexkey IDX_KS_PASS
|
||||||
|
assert_success "the quoted name round-trips through config.yml into index-v2.json" \
|
||||||
|
bash "$here/with-nixpkgs.sh" python3 python3 -c "
|
||||||
|
import json
|
||||||
|
repo = json.load(open('repo/index-v2.json'))['repo']
|
||||||
|
assert repo['name']['en-US'] == \"bit's test repo\", repo['name']
|
||||||
|
"
|
||||||
|
export FDROID_REPO_NAME="fdroid-repo-ci test repo"
|
||||||
|
|
||||||
report_and_exit
|
report_and_exit
|
||||||
|
|||||||
Reference in New Issue
Block a user