publish.sh: verify credentials and upload the index last; add next-version-code.sh
Closes #5 Closes #6 An F-Droid repo is a signed index plus the files it names, and a partial upload could previously leave them disagreeing: aws s3 sync walks the tree in its own order, so an interrupted publish could land the index before the APKs it names, breaking the repo for every client until the upload finished or was rolled back. publish.sh now uploads in two passes, unconditionally -- every non-index file first, the index (index-v1.json(.asc), index-v2.json, entry.json, entry.jar) last -- so an interruption anywhere in pass 1 simply leaves the repo describing the old state a little longer, never a mixture. Credentials are also verified with one real authenticated call (head-bucket) before either pass, rather than merely checking the environment variables are non-empty, since an unusable-but-present credential failing halfway through is exactly how a partial upload happens. next-version-code.sh is the one implementation of the versionCode rule (candidate = minutes since 2026-01-01Z, clamped to strictly exceed whatever is already published for that applicationId), shared by CI and manual builds so there is exactly one place this can drift. bit pointed out (appended to #6) that the clamp is monotonic against *observed* state, not monotonic by construction: two jobs starting in the same minute can both read the same published value and race to publish the same versionCode on different APKs. Nothing can prevent that wasted build, so publish.sh now also refuses to overwrite an already-existing APK object before pass 1 -- turning the collision from a silent, permanent wrong state into a loud failure at the last moment the information is reliable. Tested offline (83 assertions: pure-stdlib suites for both scripts, the real CLIs through with-nixpkgs.sh, and a small local HEAD-only S3 stub proving the credential check is a real authenticated call and the APK-collision refusal fires only when it should) and live against fdroid-weblib-dev (7 more: a real publish observed mid-flight via a same-process synchronization hook to prove the index is genuinely absent while the file is already present, both objects reading back byte-identical, and a second publish refused against a real already-published APK with its bytes left unchanged). Co-authored-by: bit <bit@das-labor.org>
This commit is contained in:
Executable
+70
@@ -0,0 +1,70 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Minimal path-style S3 HEAD-only stub server, used only by test_publish.sh
|
||||
to test publish.sh's credential check and APK-existence check entirely
|
||||
offline -- no real bucket, no real credentials, nothing beyond loopback.
|
||||
|
||||
Only implements what those two checks actually call:
|
||||
|
||||
HEAD /<bucket> -- head-bucket, used to verify credentials
|
||||
HEAD /<bucket>/<key...> -- head-object, used to check for an existing APK
|
||||
|
||||
`aws`'s S3 client only uses path-style addressing (rather than
|
||||
`<bucket>.<host>` virtual-hosted style) when the endpoint host is a literal
|
||||
IP address, which `--endpoint-url http://127.0.0.1:<port>` is -- so no
|
||||
`--endpoint-url` or config trickery is needed to get requests shaped this
|
||||
way.
|
||||
|
||||
Existing objects are given on the command line; every other key 404s, as
|
||||
does any bucket name other than the one given -- so a test that passes the
|
||||
wrong bucket name notices, rather than getting a silent "yes" back.
|
||||
|
||||
Deliberately never handles PUT/GET/LIST: publish.sh must never reach the
|
||||
`aws s3 sync` calls in a test that uses this stub, since both a rejected
|
||||
credential and an existing APK are supposed to abort before any upload is
|
||||
attempted. If a future test needs sync to succeed against this stub, that is
|
||||
new scope for this file, not an oversight here.
|
||||
|
||||
test_s3_stub.py <bucket> [<existing-key> ...]
|
||||
|
||||
Prints the bound port as the one line on stdout, then serves until killed.
|
||||
"""
|
||||
import http.server
|
||||
import socketserver
|
||||
import sys
|
||||
|
||||
|
||||
def main():
|
||||
bucket = sys.argv[1]
|
||||
existing = set(sys.argv[2:])
|
||||
|
||||
class Handler(http.server.BaseHTTPRequestHandler):
|
||||
def do_HEAD(self):
|
||||
path = self.path.lstrip("/")
|
||||
parts = path.split("/", 1)
|
||||
if not parts or parts[0] != bucket:
|
||||
self.send_response(404)
|
||||
self.end_headers()
|
||||
return
|
||||
if len(parts) == 1 or parts[1] == "":
|
||||
# head-bucket: the one bucket named on the command line
|
||||
# always "exists" and is reachable with these "credentials".
|
||||
self.send_response(200)
|
||||
self.end_headers()
|
||||
return
|
||||
key = parts[1]
|
||||
self.send_response(200 if key in existing else 404)
|
||||
self.end_headers()
|
||||
|
||||
def log_message(self, fmt, *args):
|
||||
pass # keep test output quiet; nothing here is diagnostic
|
||||
|
||||
class Server(socketserver.TCPServer):
|
||||
allow_reuse_address = True
|
||||
|
||||
with Server(("127.0.0.1", 0), Handler) as httpd:
|
||||
print(httpd.server_address[1], flush=True)
|
||||
httpd.serve_forever()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
Reference in New Issue
Block a user