2906f00564ae236b5206a7328c2bee6a07417c37
5
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
  claudeandbit
|
2906f00564 |
fdroid-update.sh: carry each build's "added" date across a rebuild
weblib/weblib-viewer#95: every version in the published index claimed it was added at the moment of the *last* publish, so an F-Droid client's version list showed five separate builds all added within seconds of each other. Measured on the live dev repo: index-v2.json gave versionCodes 365139/365516/365666/365710/366099 the added dates 05:50:06/08/10/12/13 on 2026-09-12, while the bucket's own Last-Modified for those objects says they were actually published on 2026-09-11 at 13:53, 20:09, 22:37, 23:23 and 2026-09-12 05:50. It is not an mtime problem, and --use-date-from-apk does not fix it. fdroidserver takes the date from common.KnownApks, whose __init__ loads it out of an already existing repo/index-v2.json (and archive/index-v2.json), and whose recordapk falls back to datetime.now() for any filename it has not seen; --use-date-from-apk only changes that fallback to the local file's mtime, which for a re-downloaded APK is when it was downloaded. KnownApks' own docstring says there is nowhere else to look: the index is "currently the only way to permanently store the 'updated' date of APKs". A CI job that rebuilds repo/ on a fresh runner -- which is exactly what weblib-viewer#32's retention scheme does when it re-downloads the previous APKs -- therefore starts with an empty KnownApks, and every build in it counts as new. So hand fdroid-update.sh the previously published index, optionally, as FDROID_PREVIOUS_INDEX -- a local path, because fetching it needs to know the bucket and per fdroid-repo-ci#1 that stays on the consumer's side; this script still needs no network. preserve_added.py stages it where KnownApks looks, and afterwards *checks* that every build present in both indexes really did keep its date, failing the run if not. That is not belt-and-braces: fdroidserver's own read_added_date_from_all_apks carries a TODO about reading these dates differently, so a future version could drop the behaviour this depends on, and a silent return to flattened history is the failure mode that took a human noticing to catch the first time. Set-but-missing is a hard failure rather than a shrug, and so is a file that is not a parseable index-v2.json (a truncated download, an S3 NoSuchKey error document): KnownApks has no error handling of its own and would abort mid-library with a traceback that never names the file, and a consumer whose fetch silently produced nothing would otherwise flatten its whole history on every publish while looking healthy. A first-ever publish expresses "there is no previous index" by leaving the variable unset. Verified end to end without publishing anything, against the five real APKs and a copy of the live dev index-v2.json pulled over HTTPS: - no previous index (today's behaviour), all five re-stamped: 366099 06:23:19, 365710 06:23:18, 365666 06:23:17, 365516 06:23:15, 365139 06:23:14 -- a five-second spread, all of it "now". - previous index minus 366099, i.e. the real "four retained plus one genuinely new" case: 365139/365516/365666/365710 keep 05:50:06/08/10/12 exactly, 366099 gets 06:24:01, and lastUpdated follows the new build. - a second run on a fresh repo/ with the APK mtimes deliberately touched to now: all five unchanged, "5 already-published build(s) kept their original 'added' timestamp, 0 new". test_preserve_added.py (12 assertions, stdlib only) covers the drift detection in both directions, including the negative control that a drifted date fails; test_fdroid_update.sh covers the shell contract. run_tests.sh: 123 passed, 4 skipped, 0 failed. A weblib-viewer change is needed too -- its workflows have to fetch last publish's index and set FDROID_PREVIOUS_INDEX -- and is a separate PR in that repo. Co-authored-by: bit <bit@das-labor.org> |
||
|
claudeandbit
|
a5e4cb6a01 |
Take sign-apk.sh out of the default publish path; document the never-an-app-key invariant
bit settled the model on weblib-viewer#19: the builder signs an APK, and this repo only distributes it -- the same signature has to work whether the APK reached a device via a git release or via F-Droid, and fdroid update never creates a signature of its own. sign-apk.sh remains for a consumer whose build genuinely produces an unsigned artifact, but action.yml's header comment and the README's "Using it" example no longer show it as a default-flow step, and both now say explicitly that this repo is never handed an app signing key -- only the repo index key. Also records two decisions that had nowhere to live yet: the index key's config/secret names must stay distinct from the APK key's even while both resolve to the same keystore for now (bit intends to split them later), and the index key must be generated as RSA 4096, overriding F-Droid's own docs, which suggest 2048. Fixes #9. Co-authored-by: bit <bit@das-labor.org> |
||
|
claudeandbit
|
e5d1f9a5bf |
next-version-code.sh: derive the candidate from the commit, not the build clock
fdroid-repo-ci#6's "Superseded" sections change the rule: the candidate is now the committer date of the commit being built, clamped on both sides, rather than whatever minute the runner happened to pick the job up in. Backwards: rebase, amend and cherry-pick can produce a commit older than one already published, so the clamp keys off the committer date, not the author date -- author date survives a rebase unchanged and would replay a stale, already-superseded number. Forwards: a commit's timestamp is attacker- and typo-controlled and the number only ever rises, so a bad one can never be walked back. A value placed just below the int32 ceiling would still publish and then no later build could ever exceed it -- the app becomes permanently un-updatable. TOLERANCE absorbs honest clock skew and the clamp warns loudly rather than rejecting, since rejecting would block a legitimate build over a few minutes of skew while buying nothing the clamp doesn't already cover. next_version_code.py now takes the committer timestamp as a third argument (Unix seconds, what `git log -1 --format=%ct` prints) instead of reading the system clock, keeping it pure and testable without a git checkout. next-version-code.sh, README.md and action.yml updated to match. Granularity stays minutes: concurrency is handled separately (a workflow-level concurrency group, plus publish.sh's refuse-on-existing), so it is now a pure headroom choice. Tests (test_next_version_code.py, test_next_version_code.sh): a normal commit yields its own minute, a commit older than the published maximum still yields published+1, a future-dated commit within tolerance passes through untouched while one beyond it is clamped and warns, a commit dated near the int32 ceiling does not produce a near-int32 result, and a dedicated property test checks the result always strictly exceeds every versionCode in the fixture across all of the above. Each was verified by injecting the corresponding fault (removing the backwards clamp, removing the forwards clamp, and ignoring the commit's date entirely) and confirming the relevant test failed, then reverting. 98 assertions offline (0 failed, 4 skipped -- pre-existing, live-bucket only), up from 91. Co-authored-by: bit <bit@das-labor.org> |
||
|
claudeandbit
|
19e55c4a53 |
publish.sh: verify credentials and upload the index last; add next-version-code.sh
Closes #5 Closes #6 An F-Droid repo is a signed index plus the files it names, and a partial upload could previously leave them disagreeing: aws s3 sync walks the tree in its own order, so an interrupted publish could land the index before the APKs it names, breaking the repo for every client until the upload finished or was rolled back. publish.sh now uploads in two passes, unconditionally -- every non-index file first, the index (index-v1.json(.asc), index-v2.json, entry.json, entry.jar) last -- so an interruption anywhere in pass 1 simply leaves the repo describing the old state a little longer, never a mixture. Credentials are also verified with one real authenticated call (head-bucket) before either pass, rather than merely checking the environment variables are non-empty, since an unusable-but-present credential failing halfway through is exactly how a partial upload happens. next-version-code.sh is the one implementation of the versionCode rule (candidate = minutes since 2026-01-01Z, clamped to strictly exceed whatever is already published for that applicationId), shared by CI and manual builds so there is exactly one place this can drift. bit pointed out (appended to #6) that the clamp is monotonic against *observed* state, not monotonic by construction: two jobs starting in the same minute can both read the same published value and race to publish the same versionCode on different APKs. Nothing can prevent that wasted build, so publish.sh now also refuses to overwrite an already-existing APK object before pass 1 -- turning the collision from a silent, permanent wrong state into a loud failure at the last moment the information is reliable. Tested offline (83 assertions: pure-stdlib suites for both scripts, the real CLIs through with-nixpkgs.sh, and a small local HEAD-only S3 stub proving the credential check is a real authenticated call and the APK-collision refusal fires only when it should) and live against fdroid-weblib-dev (7 more: a real publish observed mid-flight via a same-process synchronization hook to prove the index is genuinely absent while the file is already present, both objects reading back byte-identical, and a second publish refused against a real already-published APK with its bytes left unchanged). Co-authored-by: bit <bit@das-labor.org> |
||
|
claudeandbit
|
3f14c01b00 |
Build the shared F-Droid publishing tooling: sign, index, publish
Adds sign-apk.sh (align-check + apksigner), fdroid-update.sh (fdroid update over a binary repo, signing the index with its own separate key), publish.sh (S3-compatible upload) and the composite action tying them together, matching weblib-ci's shape. zipalign is not a standalone nixpkgs attribute; resolved by measurement rather than assumption that Gradle/AGP already aligns its output (Android's own docs, corroborated by fdroidserver never shelling out to zipalign) and added check_zip_alignment.py as the safety net, since apksigner itself does not check alignment at all. nixpkgs is pinned via a committed nixpkgs-pin revision, since this repo has no flake.lock of its own for with-nixpkgs.sh to read (fdroid-repo-ci#1, weblib-ci#9) -- a missing or malformed pin fails loudly rather than degrading to an unpinned nix shell. No consumer repo, bucket, hostname or channel name is hard-coded: which repo holds the keys and runs the publish has already changed once during this issue and may again, so every example uses a placeholder. 42 offline/local-only test assertions across five suites (run_tests.sh), including fault injection on the alignment check, the password/argument validation, and the with-nixpkgs.sh fail-loud paths. Co-authored-by: bit <bit@das-labor.org> |