c529ca9564977f4ff05918509cd3fdf000b91a0e
10
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 bit
|
c529ca9564 |
Merge pull request 'Document the APK filename/versionCode naming contract' (#14) from apk-versioncode-naming-contract into main
Reviewed-on: #14 |
||
 claudeandbit
|
2906f00564 |
fdroid-update.sh: carry each build's "added" date across a rebuild
weblib/weblib-viewer#95: every version in the published index claimed it was added at the moment of the *last* publish, so an F-Droid client's version list showed five separate builds all added within seconds of each other. Measured on the live dev repo: index-v2.json gave versionCodes 365139/365516/365666/365710/366099 the added dates 05:50:06/08/10/12/13 on 2026-09-12, while the bucket's own Last-Modified for those objects says they were actually published on 2026-09-11 at 13:53, 20:09, 22:37, 23:23 and 2026-09-12 05:50. It is not an mtime problem, and --use-date-from-apk does not fix it. fdroidserver takes the date from common.KnownApks, whose __init__ loads it out of an already existing repo/index-v2.json (and archive/index-v2.json), and whose recordapk falls back to datetime.now() for any filename it has not seen; --use-date-from-apk only changes that fallback to the local file's mtime, which for a re-downloaded APK is when it was downloaded. KnownApks' own docstring says there is nowhere else to look: the index is "currently the only way to permanently store the 'updated' date of APKs". A CI job that rebuilds repo/ on a fresh runner -- which is exactly what weblib-viewer#32's retention scheme does when it re-downloads the previous APKs -- therefore starts with an empty KnownApks, and every build in it counts as new. So hand fdroid-update.sh the previously published index, optionally, as FDROID_PREVIOUS_INDEX -- a local path, because fetching it needs to know the bucket and per fdroid-repo-ci#1 that stays on the consumer's side; this script still needs no network. preserve_added.py stages it where KnownApks looks, and afterwards *checks* that every build present in both indexes really did keep its date, failing the run if not. That is not belt-and-braces: fdroidserver's own read_added_date_from_all_apks carries a TODO about reading these dates differently, so a future version could drop the behaviour this depends on, and a silent return to flattened history is the failure mode that took a human noticing to catch the first time. Set-but-missing is a hard failure rather than a shrug, and so is a file that is not a parseable index-v2.json (a truncated download, an S3 NoSuchKey error document): KnownApks has no error handling of its own and would abort mid-library with a traceback that never names the file, and a consumer whose fetch silently produced nothing would otherwise flatten its whole history on every publish while looking healthy. A first-ever publish expresses "there is no previous index" by leaving the variable unset. Verified end to end without publishing anything, against the five real APKs and a copy of the live dev index-v2.json pulled over HTTPS: - no previous index (today's behaviour), all five re-stamped: 366099 06:23:19, 365710 06:23:18, 365666 06:23:17, 365516 06:23:15, 365139 06:23:14 -- a five-second spread, all of it "now". - previous index minus 366099, i.e. the real "four retained plus one genuinely new" case: 365139/365516/365666/365710 keep 05:50:06/08/10/12 exactly, 366099 gets 06:24:01, and lastUpdated follows the new build. - a second run on a fresh repo/ with the APK mtimes deliberately touched to now: all five unchanged, "5 already-published build(s) kept their original 'added' timestamp, 0 new". test_preserve_added.py (12 assertions, stdlib only) covers the drift detection in both directions, including the negative control that a drifted date fails; test_fdroid_update.sh covers the shell contract. run_tests.sh: 123 passed, 4 skipped, 0 failed. A weblib-viewer change is needed too -- its workflows have to fetch last publish's index and set FDROID_PREVIOUS_INDEX -- and is a separate PR in that repo. Co-authored-by: bit <bit@das-labor.org> |
||
|
claudeandbit
|
af32b7ca12 |
Document and hint at the APK-filename/versionCode naming contract
publish.sh's overwrite guard (fdroid-repo-ci#6) is correct but implies a contract nothing states: a colliding filename with no versionCode in it means only the first publish into a prefix can ever succeed. Flutter's constant app-<flavor>-release.apk hit this in weblib-viewer (#29), and the failure only surfaces on the second publish with an error that reads like a bucket-state problem. Add the naming contract to the README, and append a hint to the refusal message when the colliding object's basename has no digits at all -- a pure text addition, never changing which publishes are refused. Closes fdroid-repo-ci#13. Co-authored-by: bit <bit@das-labor.org> |
||
|
claudeandbit
|
4a25e3681a |
Require FDROID_REPO_URL/FDROID_REPO_NAME so fdroid-update.sh cannot publish fdroidserver's placeholder
The published repo advertised fdroidserver's own demo values (repo.address "https://MyFirstFDroidRepo.org/fdroid/repo", repo.name "My First F-Droid Repo Demo") because fdroid-update.sh wrote config.yml itself but never set repo_url/repo_name/repo_description, so fdroidserver fell back to its template. The "Add to F-Droid" button on the generated index.html is built from repo_url, which is how bit noticed. A publish that succeeds while advertising the wrong URL is silently wrong -- worse than a failed job. Add FDROID_REPO_URL/FDROID_REPO_NAME/FDROID_REPO_DESCRIPTION environment variables (not new positionals -- the script already has five plus pass-through extra args). The first two are now a hard requirement, naming the missing one; FDROID_REPO_DESCRIPTION stays optional. This is a breaking change to a public composite action's interface, acceptable only because weblib-viewer is the sole consumer and is updated in the same change (weblib-viewer#26). Also discovered while wiring this up and worth recording since it contradicts the "no /fdroid/repo suffix" assumption in weblib's working notes: fdroidserver's own common.py unconditionally refuses to run once config.yml sets repo_url at all, unless it ends in the literal suffix "/repo" -- there is no flag to disable that check, and the value is embedded verbatim as repo.address. So FDROID_REPO_URL has to end in /repo, and publish.sh's existing <prefix> argument has to put the published tree under a URL path that actually ends in /repo to match, or fdroid update refuses to run at all (checked positive) / the address points at a path the bucket never serves (checked against the live fdroid-weblib-dev bucket layout). Extends test_fdroid_update.sh: config.yml carries the values, the resulting index-v2.json has the right repo.address/name/description (not fdroidserver's placeholder), a name with an embedded single quote survives the YAML round-trip, and the script fails when FDROID_REPO_URL or FDROID_REPO_NAME is absent or FDROID_REPO_URL lacks the /repo suffix. run_tests.sh: 104 passed / 4 skipped (was 98/4). Filed as fdroid-repo-ci#11. Co-authored-by: bit <bit@das-labor.org> |
||
|
claudeandbit
|
a5e4cb6a01 |
Take sign-apk.sh out of the default publish path; document the never-an-app-key invariant
bit settled the model on weblib-viewer#19: the builder signs an APK, and this repo only distributes it -- the same signature has to work whether the APK reached a device via a git release or via F-Droid, and fdroid update never creates a signature of its own. sign-apk.sh remains for a consumer whose build genuinely produces an unsigned artifact, but action.yml's header comment and the README's "Using it" example no longer show it as a default-flow step, and both now say explicitly that this repo is never handed an app signing key -- only the repo index key. Also records two decisions that had nowhere to live yet: the index key's config/secret names must stay distinct from the APK key's even while both resolve to the same keystore for now (bit intends to split them later), and the index key must be generated as RSA 4096, overriding F-Droid's own docs, which suggest 2048. Fixes #9. Co-authored-by: bit <bit@das-labor.org> |
||
|
claudeandbit
|
e5d1f9a5bf |
next-version-code.sh: derive the candidate from the commit, not the build clock
fdroid-repo-ci#6's "Superseded" sections change the rule: the candidate is now the committer date of the commit being built, clamped on both sides, rather than whatever minute the runner happened to pick the job up in. Backwards: rebase, amend and cherry-pick can produce a commit older than one already published, so the clamp keys off the committer date, not the author date -- author date survives a rebase unchanged and would replay a stale, already-superseded number. Forwards: a commit's timestamp is attacker- and typo-controlled and the number only ever rises, so a bad one can never be walked back. A value placed just below the int32 ceiling would still publish and then no later build could ever exceed it -- the app becomes permanently un-updatable. TOLERANCE absorbs honest clock skew and the clamp warns loudly rather than rejecting, since rejecting would block a legitimate build over a few minutes of skew while buying nothing the clamp doesn't already cover. next_version_code.py now takes the committer timestamp as a third argument (Unix seconds, what `git log -1 --format=%ct` prints) instead of reading the system clock, keeping it pure and testable without a git checkout. next-version-code.sh, README.md and action.yml updated to match. Granularity stays minutes: concurrency is handled separately (a workflow-level concurrency group, plus publish.sh's refuse-on-existing), so it is now a pure headroom choice. Tests (test_next_version_code.py, test_next_version_code.sh): a normal commit yields its own minute, a commit older than the published maximum still yields published+1, a future-dated commit within tolerance passes through untouched while one beyond it is clamped and warns, a commit dated near the int32 ceiling does not produce a near-int32 result, and a dedicated property test checks the result always strictly exceeds every versionCode in the fixture across all of the above. Each was verified by injecting the corresponding fault (removing the backwards clamp, removing the forwards clamp, and ignoring the commit's date entirely) and confirming the relevant test failed, then reverting. 98 assertions offline (0 failed, 4 skipped -- pre-existing, live-bucket only), up from 91. Co-authored-by: bit <bit@das-labor.org> |
||
|
claudeandbit
|
ccb8cb616e |
publish.sh: replace the test-only eval hook with --files-only/--index-only
bit's review on PR #7 (comment on the PR, appended to this reasoning): run_test_hook() eval'd an environment variable in publish.sh, which runs with the release signing key and S3 credentials in its environment, in a public repo. "Never set by a real caller" was true, but it was still a documented, discoverable path from any env-var influence to arbitrary code execution at the worst possible moment -- unnecessary, which is the stronger reason to remove it than any claim about likelihood. The replacement is a feature rather than a seam: --files-only and --index-only split the two passes into separately invokable steps. With neither flag the default is unchanged (both passes, files first). This makes the ordering guarantee testable by construction -- run --files-only, assert the bucket state, run --index-only, assert again -- with no hook, no eval, and no test-only branch in production code. It is also independently useful: an interrupted publish can resume at the index step without re-uploading every APK, and a caller can stage files now and publish the index later. The APK-existence check is skipped entirely for a plain --index-only invocation, since that pass can never upload a *.apk. Re-tested end to end: offline 91 passed / 4 skipped / 0 failed (up from 83/ 3/0, mostly new flag-construction and stub assertions); live against fdroid-weblib-dev 104 passed / 0 skipped / 0 failed, bucket verified empty before and after. Added a live fault-injection test that --index-only alone against a prefix --files-only never touched really does publish an index naming an absent file -- the exact state the ordering guarantee exists to prevent -- contrasted with the default path, which never reaches it. Offline, removing the run_files/run_index gating (simulating the flags being ignored) was confirmed to fail 5 of the new assertions before being reverted. Left the versionCode granularity as minutes, per instruction: that is being settled separately on weblib-viewer#15 and is a one-constant change for whenever it lands, not something to pre-empt here. Co-authored-by: bit <bit@das-labor.org> |
||
|
claudeandbit
|
19e55c4a53 |
publish.sh: verify credentials and upload the index last; add next-version-code.sh
Closes #5 Closes #6 An F-Droid repo is a signed index plus the files it names, and a partial upload could previously leave them disagreeing: aws s3 sync walks the tree in its own order, so an interrupted publish could land the index before the APKs it names, breaking the repo for every client until the upload finished or was rolled back. publish.sh now uploads in two passes, unconditionally -- every non-index file first, the index (index-v1.json(.asc), index-v2.json, entry.json, entry.jar) last -- so an interruption anywhere in pass 1 simply leaves the repo describing the old state a little longer, never a mixture. Credentials are also verified with one real authenticated call (head-bucket) before either pass, rather than merely checking the environment variables are non-empty, since an unusable-but-present credential failing halfway through is exactly how a partial upload happens. next-version-code.sh is the one implementation of the versionCode rule (candidate = minutes since 2026-01-01Z, clamped to strictly exceed whatever is already published for that applicationId), shared by CI and manual builds so there is exactly one place this can drift. bit pointed out (appended to #6) that the clamp is monotonic against *observed* state, not monotonic by construction: two jobs starting in the same minute can both read the same published value and race to publish the same versionCode on different APKs. Nothing can prevent that wasted build, so publish.sh now also refuses to overwrite an already-existing APK object before pass 1 -- turning the collision from a silent, permanent wrong state into a loud failure at the last moment the information is reliable. Tested offline (83 assertions: pure-stdlib suites for both scripts, the real CLIs through with-nixpkgs.sh, and a small local HEAD-only S3 stub proving the credential check is a real authenticated call and the APK-collision refusal fires only when it should) and live against fdroid-weblib-dev (7 more: a real publish observed mid-flight via a same-process synchronization hook to prove the index is genuinely absent while the file is already present, both objects reading back byte-identical, and a second publish refused against a real already-published APK with its bytes left unchanged). Co-authored-by: bit <bit@das-labor.org> |
||
|
claudeandbit
|
52f8de9993 |
README: narrow the "not hard-coded" claim to what is true
It said no organisation appears "in a path, URL or workflow reference", while the usage examples immediately above necessarily contain `https://git.chaosbit.de/weblib/fdroid-repo-ci@main` -- a consumer has to type this repository's address, so it cannot not appear. A rule a file visibly breaks is worse than no rule: the next person either "fixes" the examples or stops trusting the section. So the claim now covers what it should -- consumer-specific things, and any named consumer -- and calls out this repo's own address as the one deliberate exception, which is also a useful pointer to the exact lines to edit when the repo moves org. Found in the cold re-read of #2. Co-authored-by: bit <bit@das-labor.org> |
||
|
claudeandbit
|
3f14c01b00 |
Build the shared F-Droid publishing tooling: sign, index, publish
Adds sign-apk.sh (align-check + apksigner), fdroid-update.sh (fdroid update over a binary repo, signing the index with its own separate key), publish.sh (S3-compatible upload) and the composite action tying them together, matching weblib-ci's shape. zipalign is not a standalone nixpkgs attribute; resolved by measurement rather than assumption that Gradle/AGP already aligns its output (Android's own docs, corroborated by fdroidserver never shelling out to zipalign) and added check_zip_alignment.py as the safety net, since apksigner itself does not check alignment at all. nixpkgs is pinned via a committed nixpkgs-pin revision, since this repo has no flake.lock of its own for with-nixpkgs.sh to read (fdroid-repo-ci#1, weblib-ci#9) -- a missing or malformed pin fails loudly rather than degrading to an unpinned nix shell. No consumer repo, bucket, hostname or channel name is hard-coded: which repo holds the keys and runs the publish has already changed once during this issue and may again, so every example uses a placeholder. 42 offline/local-only test assertions across five suites (run_tests.sh), including fault injection on the alignment check, the password/argument validation, and the with-nixpkgs.sh fail-loud paths. Co-authored-by: bit <bit@das-labor.org> |