publish.sh uploads the whole tree with a single aws s3 sync, which gives no control over the order files land in. For an F-Droid repo that matters, and the failure is public rather than local.
The problem
An F-Droid repo is a signed index plus the files it names, and a partial upload can leave them disagreeing in two ways:
what happened
what a client sees
index landed first, APKs still uploading
an index naming files that 404
APKs landed, index not updated
the new build is invisible — merely useless
The second is harmless and self-correcting. The first breaks the repo for every client until the upload finishes or is rolled back, and an interrupted publish (network drop, cancelled job, expired credential mid-run) produces exactly it.
aws s3 sync walks the tree in its own order; nothing stops index-v2.json going up before the APK it names.
What to change
Upload in two passes: files first, index last. The index is the only thing that makes new APKs visible, so writing it last means the repo is always internally consistent — before the publish it describes the old state, after it describes the new one, and never a mixture.
Verify the credentials before uploading anything, with a real authenticated call rather than a check that the variables are non-empty. bit, weblib-viewer#16: the manual publish path must "abort if they are not set/not usable" — an unusable-but-present credential that fails halfway is precisely how a partial upload happens.
Both belong here rather than in a consumer: every consumer publishes the same shape of repo and would otherwise reimplement the ordering, probably differently.
Verification
The existing opt-in live test (#3) already uploads and reads back, so it can be extended: publish a tree containing both an index and a file, and assert the index is not present at the destination until the file is. Doing that reliably may need the two passes to be separately observable — worth building the check into the script's own output rather than trying to catch a race from outside.
Note the offline tests cannot see any of this: they assert command construction, and this is about the order two real uploads happen in.
`publish.sh` uploads the whole tree with a single `aws s3 sync`, which gives no control over the order files land in. For an F-Droid repo that matters, and the failure is public rather than local.
## The problem
An F-Droid repo is a **signed index plus the files it names**, and a partial upload can leave them disagreeing in two ways:
| what happened | what a client sees |
|---|---|
| index landed first, APKs still uploading | an index naming files that **404** |
| APKs landed, index not updated | the new build is invisible — merely useless |
The second is harmless and self-correcting. The first **breaks the repo for every client** until the upload finishes or is rolled back, and an interrupted publish (network drop, cancelled job, expired credential mid-run) produces exactly it.
`aws s3 sync` walks the tree in its own order; nothing stops `index-v2.json` going up before the APK it names.
## What to change
1. **Upload in two passes: files first, index last.** The index is the only thing that makes new APKs visible, so writing it last means the repo is always internally consistent — before the publish it describes the old state, after it describes the new one, and never a mixture.
2. **Verify the credentials before uploading anything**, with a real authenticated call rather than a check that the variables are non-empty. bit, weblib-viewer#16: the manual publish path must "abort if they are not set/**not usable**" — an unusable-but-present credential that fails halfway is precisely how a partial upload happens.
Both belong here rather than in a consumer: every consumer publishes the same shape of repo and would otherwise reimplement the ordering, probably differently.
## Verification
The existing opt-in live test (#3) already uploads and reads back, so it can be extended: publish a tree containing both an index and a file, and assert the index is not present at the destination until the file is. Doing that reliably may need the two passes to be separately observable — worth building the check into the script's own output rather than trying to catch a race from outside.
Note the offline tests cannot see any of this: they assert command construction, and this is about the order two real uploads happen in.
publish.sh now uploads in two passes with the index last, verifies credentials with a real head-bucket call before uploading anything, and refuses to overwrite an existing *.apk. The two passes are separately invokable (--files-only / --index-only), which is what makes the ordering testable by construction rather than by reaching into the middle of a run.
The assertion worth keeping: invoking --index-only alone genuinely produces an index naming a file that 404s — the exact state this issue is about — and the default path is checked never to reach it. A test that only proved the good path would not have shown the failure mode is real.
Delivered by #7 and closing.
`publish.sh` now uploads in two passes with the index last, verifies credentials with a real `head-bucket` call before uploading anything, and refuses to overwrite an existing `*.apk`. The two passes are separately invokable (`--files-only` / `--index-only`), which is what makes the ordering testable by construction rather than by reaching into the middle of a run.
The assertion worth keeping: invoking `--index-only` alone genuinely produces an index naming a file that 404s — the exact state this issue is about — and the default path is checked never to reach it. A test that only proved the good path would not have shown the failure mode is real.
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
publish.shuploads the whole tree with a singleaws s3 sync, which gives no control over the order files land in. For an F-Droid repo that matters, and the failure is public rather than local.The problem
An F-Droid repo is a signed index plus the files it names, and a partial upload can leave them disagreeing in two ways:
The second is harmless and self-correcting. The first breaks the repo for every client until the upload finishes or is rolled back, and an interrupted publish (network drop, cancelled job, expired credential mid-run) produces exactly it.
aws s3 syncwalks the tree in its own order; nothing stopsindex-v2.jsongoing up before the APK it names.What to change
Both belong here rather than in a consumer: every consumer publishes the same shape of repo and would otherwise reimplement the ordering, probably differently.
Verification
The existing opt-in live test (#3) already uploads and reads back, so it can be extended: publish a tree containing both an index and a file, and assert the index is not present at the destination until the file is. Doing that reliably may need the two passes to be separately observable — worth building the check into the script's own output rather than trying to catch a race from outside.
Note the offline tests cannot see any of this: they assert command construction, and this is about the order two real uploads happen in.
Delivered by #7 and closing.
publish.shnow uploads in two passes with the index last, verifies credentials with a realhead-bucketcall before uploading anything, and refuses to overwrite an existing*.apk. The two passes are separately invokable (--files-only/--index-only), which is what makes the ordering testable by construction rather than by reaching into the middle of a run.The assertion worth keeping: invoking
--index-onlyalone genuinely produces an index naming a file that 404s — the exact state this issue is about — and the default path is checked never to reach it. A test that only proved the good path would not have shown the failure mode is real.