Compare commits
14
Commits
9c26afd8d2
...
main
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
41c8b81112 | ||
|
|
c2947db08a | ||
|
|
8c0088987b | ||
|
|
66ef3f78d1 | ||
|
|
05e8103eb0 | ||
|
|
337946e533 | ||
|
|
13e396a795 | ||
|
|
028c162874 | ||
|
|
be73235c01 | ||
|
|
67b133a115 | ||
|
|
259bdd93ca | ||
|
|
e6f18b5a3b | ||
|
|
3a0d8db656 | ||
|
|
f9777ccbd2 |
@@ -0,0 +1,4 @@
|
||||
# The test suite imports report_job_log, and a tracked .pyc of it went stale
|
||||
# the moment the source changed.
|
||||
__pycache__/
|
||||
*.pyc
|
||||
@@ -14,12 +14,27 @@ fetch it — which was measured to be the difference between one step and three.
|
||||
| file | what it does |
|
||||
|---|---|
|
||||
| `with-nixpkgs.sh` | Runs a command with one nixpkgs package on PATH, pinned to the *consuming* repo's `flake.lock`. Avoids `nix shell nixpkgs#x`, which re-resolves the registry and refetches a channel tarball whenever the branch moves. |
|
||||
| `report_job_log.py` | Posts the tail of a build log as a PR comment. Exists because `actions/jobs/{id}/logs` returns 500 for every id on Gitea 1.25.2, so a red job otherwise says only that it failed. |
|
||||
| `sync_blocked_label.py` | Keeps `Status/Blocked` in step with Gitea's dependency graph. |
|
||||
| `report_job_log.py` | Posts the tail of a build log as a PR comment, falling back to a commit status when a `push`-triggered job has no PR to comment on. Exists because `actions/jobs/{id}/logs` returns 500 for every id on Gitea 1.25.2, so a red job otherwise says only that it failed. Takes `--status`; see below. |
|
||||
| `sync_blocked_label.py` | Keeps `Status/Blocked` in step with Gitea's dependency graph. Resolves the label from the repo *or the organisation*, never touches an issue marked `Status/On Hold` or `Status/Abandoned`, and leaves the label alone (rather than guessing) on a blocker whose state a repo-scoped token cannot resolve — see weblib-ci#11. |
|
||||
|
||||
All three are standard library / plain bash only. They are *run*, not built, so
|
||||
this repo has no flake.
|
||||
|
||||
`test_report_job_log.py` covers the reporter and `test_sync_blocked_label.py`
|
||||
covers the reconciler. Both are standard library and offline — the forge each
|
||||
talks to is an `http.server` on localhost that keeps what it is sent, so a
|
||||
test reads state back rather than trusting an exit status of 0 (which the
|
||||
reporter in particular returns even when its POST failed).
|
||||
|
||||
```bash
|
||||
python3 test_report_job_log.py # 28 tests, ~3s, no network
|
||||
python3 test_sync_blocked_label.py # 12 tests, no network
|
||||
```
|
||||
|
||||
There is no workflow running it: this repo has no `.gitea/workflows` at all,
|
||||
and no `flake.lock` for `with-nixpkgs.sh` to read. Run it by hand before
|
||||
pushing.
|
||||
|
||||
## Using it
|
||||
|
||||
```yaml
|
||||
@@ -57,6 +72,78 @@ The `outputs.path` row is listed separately on purpose: the action *running* and
|
||||
its output *reaching the caller* are different claims, and a composite action
|
||||
returning an empty string is exactly the sort of thing that looks green.
|
||||
|
||||
### `report_job_log.py --status`
|
||||
|
||||
The header used to be hardcoded to *failed*. That is true of every caller here,
|
||||
because each guards the step with `if: failure()` — but a probe run under
|
||||
`if: always()` posted a failure report for a job that had passed
|
||||
(weblib-viewer#10, filed as #8).
|
||||
|
||||
**The default is still `failed`**, so a caller passing only the log path is
|
||||
unchanged. A step that can run on success has to say so:
|
||||
|
||||
```yaml
|
||||
- name: report the log
|
||||
if: always()
|
||||
env:
|
||||
GITEA_TOKEN: ${{ secrets.GITEA_TOKEN }}
|
||||
run: |
|
||||
bash "${{ steps.ci.outputs.path }}/with-nixpkgs.sh" python3 \
|
||||
python3 "${{ steps.ci.outputs.path }}/report_job_log.py" /tmp/build.log \
|
||||
--status "${{ job.status }}"
|
||||
```
|
||||
|
||||
`${{ job.status }}` yields `success`/`failure`/`cancelled`/`skipped`, so those
|
||||
spellings are accepted alongside `passed`/`failed`. `JOB_STATUS` in the
|
||||
environment does the same thing if a flag is awkward. An **unrecognised**
|
||||
status is put in the header verbatim rather than rejected: `argparse`'s
|
||||
`choices=` would exit 2 on a value this list has not heard of, and the log —
|
||||
the entire reason the script exists — would never be posted.
|
||||
|
||||
### `push`-triggered jobs: no open PR does not mean no channel
|
||||
|
||||
Found the hard way on weblib-ci#14: a `push` to `development`/`main` — the
|
||||
common shape of a workflow like a publish pipeline, which has no
|
||||
`pull_request` trigger at all — has no *open* PR for `report_job_log.py` to
|
||||
comment on. The branch that was pushed to is the merge's **target**, not the
|
||||
PR that produced it, and that PR is already closed. The old behaviour was to
|
||||
print "no pull request for this run" and exit 0, so a failure there was
|
||||
completely invisible: not the log (500 on this Gitea), not a comment
|
||||
(nowhere to put it), nothing. That is how weblib-viewer run 1769's failure
|
||||
(`publish.sh` refusing to overwrite an existing APK) went unnoticed until it
|
||||
was reproduced by hand.
|
||||
|
||||
Two things changed, tried in order, neither needing a new flag:
|
||||
|
||||
1. **Ask Gitea which PR produced this exact commit.**
|
||||
`GET repos/{repo}/commits/{sha}/pull` resolves the PR a commit came from —
|
||||
open or closed — keyed on the commit itself rather than a branch name.
|
||||
Verified against both merge styles used in this org: an explicit merge
|
||||
commit, and a fast-forward promotion where the "merge commit" is just the
|
||||
PR branch's own tip. This needs the **full** 40-character `GITHUB_SHA`; a
|
||||
short sha gets the same 404 shape as a commit with no PR at all, so it
|
||||
would silently look like "not found". This covers the ordinary case —
|
||||
almost every push to a protected branch here is a merge landing.
|
||||
2. **If that also finds nothing** — a commit that was never part of any
|
||||
PR — post a **commit status** instead
|
||||
(`POST repos/{repo}/statuses/{sha}`, context `report_job_log`). It is not
|
||||
a substitute for the full log: this Gitea did not truncate a 3000-character
|
||||
description in testing, but a status is UI chrome, not a page, so what
|
||||
goes there is deliberately a short snippet (`STATUS_TAIL`, 500 characters),
|
||||
not the whole tail. `state` follows the same outcome as the header phrase
|
||||
(`failed → failure`, `passed → success`, `cancelled → warning`,
|
||||
`skipped → skipped`; an unrecognised status maps to `error` rather than
|
||||
guessing pass or fail), and `target_url` points at the commit.
|
||||
|
||||
**A commit *comment* API (GitHub-style, `commits/{sha}/comments`) does not
|
||||
exist on this Gitea** — 404, and absent from `/swagger.v1.json` entirely, not
|
||||
merely unauthorised. That ruled out the most direct-looking option; the
|
||||
commit-status endpoint is the one that is actually there.
|
||||
|
||||
If a PR *is* found (by either the original lookups or the new sha lookup),
|
||||
behaviour is identical to before: a comment, same format. The status path is
|
||||
only reached when no PR exists anywhere for the commit.
|
||||
|
||||
### Why not a flake input
|
||||
|
||||
These are scripts a workflow runs, not derivations. A flake input would cost a
|
||||
|
||||
+205
-15
@@ -9,13 +9,34 @@ can be read, next to the change that caused it.
|
||||
Standard library only, like `sync_blocked_label.py`, so it needs nothing but an
|
||||
interpreter.
|
||||
|
||||
Lives in `weblib/weblib-ci` and is used by the other repos from there, so the
|
||||
comment it posts must not name a path inside the repo it is reporting on --
|
||||
there is no copy there to find.
|
||||
|
||||
Usage, from a workflow step guarded by `if: failure()`:
|
||||
|
||||
report_job_log.py /tmp/build.log
|
||||
|
||||
The header says *failed* by default, which is what that guard means. A step
|
||||
guarded by `if: always()` -- a probe, or a job whose output is the point rather
|
||||
than its exit code -- has to say so, or the comment reports a failure that did
|
||||
not happen:
|
||||
|
||||
report_job_log.py /tmp/build.log --status "${{ job.status }}"
|
||||
|
||||
Everything else comes from the Actions environment: `GITHUB_REPOSITORY`,
|
||||
`GITHUB_EVENT_PATH` (for the PR number), `GITEA_TOKEN`/`GITHUB_TOKEN`, and
|
||||
`GITHUB_SERVER_URL`.
|
||||
`GITHUB_EVENT_PATH` (for the PR number), `GITEA_TOKEN`/`GITHUB_TOKEN`,
|
||||
`GITHUB_SERVER_URL`, and `JOB_STATUS` if `--status` is not passed.
|
||||
|
||||
A `push` event has no open PR to comment on -- the common case is that a PR
|
||||
just merged into `development`/`main` and closed, so `GITHUB_HEAD_REF` is the
|
||||
target branch, not the one that was merged (weblib-ci#14). Before giving up,
|
||||
this asks Gitea which PR produced `GITHUB_SHA`
|
||||
(`GET .../commits/{sha}/pull`, which resolves a closed PR from its merge
|
||||
commit -- verified against both merge styles this org uses). If that also
|
||||
comes up empty -- a commit that was never part of any PR -- the log itself has
|
||||
nowhere to go, so a commit status is posted instead: shorter, but still
|
||||
readable next to the commit, unlike silence.
|
||||
|
||||
Remove this once the instance is updated and logs can be read directly.
|
||||
"""
|
||||
@@ -37,6 +58,75 @@ DEFAULT_TAIL = 12000
|
||||
# nix colours its output, and the raw escapes are noise in Markdown.
|
||||
ANSI = re.compile(r"\x1b\[[0-9;]*[A-Za-z]")
|
||||
|
||||
# A commit status is UI chrome (a line in a list), not a page -- measured
|
||||
# against this Gitea: it accepted a 3000-char description unchanged, so
|
||||
# nothing here truncates it server-side, but that does not make a wall of log
|
||||
# a good idea to put there. Keep it to a snippet.
|
||||
STATUS_TAIL = 500
|
||||
|
||||
# The header used to hardcode "failed". Every caller guards the step with
|
||||
# `if: failure()`, so that was true by construction -- until an `if: always()`
|
||||
# probe in weblib-viewer#10 posted a failure report for a job that had passed.
|
||||
# The default stays `failed` so those guarded callers are unchanged.
|
||||
DEFAULT_STATUS = "failed"
|
||||
|
||||
# `${{ job.status }}` yields success/failure/cancelled/skipped; a human writing
|
||||
# the flag by hand reaches for passed/failed. Accept both spellings so a call
|
||||
# site can pass the expression straight through.
|
||||
STATUS_PHRASES = {
|
||||
"failed": "failed",
|
||||
"failure": "failed",
|
||||
"passed": "passed",
|
||||
"success": "passed",
|
||||
"succeeded": "passed",
|
||||
"cancelled": "was cancelled",
|
||||
"canceled": "was cancelled",
|
||||
"skipped": "was skipped",
|
||||
}
|
||||
|
||||
# Gitea's status `state` enum is not the same vocabulary as the header
|
||||
# phrases above (`pending, success, error, failure, warning, skipped`).
|
||||
# A phrase this table has not heard of -- the "finished with status `X`"
|
||||
# fallback -- maps to `error`: the honest reading is "something unusual
|
||||
# happened", not a guess at pass or fail.
|
||||
STATUS_STATE = {
|
||||
"failed": "failure",
|
||||
"passed": "success",
|
||||
"was cancelled": "warning",
|
||||
"was skipped": "skipped",
|
||||
}
|
||||
|
||||
|
||||
def status_phrase(status):
|
||||
"""The verb for the comment header.
|
||||
|
||||
An unrecognised status is reported verbatim rather than rejected. This
|
||||
script exists because a red job says nothing, so it must not itself become
|
||||
the thing that says nothing: `argparse`'s `choices=` would exit 2 on a
|
||||
status this table has not heard of, and the log would never be posted.
|
||||
"""
|
||||
key = (status or "").strip().lower()
|
||||
if not key:
|
||||
key = DEFAULT_STATUS
|
||||
if key in STATUS_PHRASES:
|
||||
return STATUS_PHRASES[key]
|
||||
# Backticks removed, not escaped: the verbatim value goes inside a code
|
||||
# span in a **bold** header, and a backtick in it closes the span early --
|
||||
# the rest of the status then renders as markdown. Nothing hostile is
|
||||
# expected here (`${{ job.status }}` is written by whoever wrote the
|
||||
# workflow), but this repo is public and consumed by four others, and a
|
||||
# branch name or matrix value could reach this argument later.
|
||||
return "finished with status `" + status.strip().replace("`", "") + "`"
|
||||
|
||||
|
||||
def missing_log_note(path, phrase):
|
||||
"""What to say when the log file the step named is not there."""
|
||||
if phrase == "failed":
|
||||
return (f"The step produced no log file at `{path}` — it failed "
|
||||
"before the build started.")
|
||||
return (f"The step produced no log file at `{path}` — nothing was "
|
||||
"captured.")
|
||||
|
||||
|
||||
def pull_request_number(event_path):
|
||||
"""The PR this job is running for, from the event payload, or None."""
|
||||
@@ -57,8 +147,9 @@ def pull_request_for_branch(host, repo, token, branch):
|
||||
dispatched run that failed stayed exactly as silent as the one this script
|
||||
exists to fix. Looking the branch up covers those.
|
||||
|
||||
A push to a branch with no PR still has nowhere to comment, which is
|
||||
honest: there is no thread for it.
|
||||
Only finds an **open** PR. A push whose branch's PR has already merged and
|
||||
closed (the common shape for `development`/`main`) is `pull_request_for_sha`'s
|
||||
job, tried next.
|
||||
"""
|
||||
if not branch:
|
||||
return None
|
||||
@@ -79,6 +170,44 @@ def pull_request_for_branch(host, repo, token, branch):
|
||||
return None
|
||||
|
||||
|
||||
def pull_request_for_sha(host, repo, token, sha):
|
||||
"""The PR that produced this commit, open or closed, or None.
|
||||
|
||||
Covers what `pull_request_for_branch` cannot: a `push` to
|
||||
`development`/`main` is almost always a merge landing, and by the time
|
||||
the job runs, the PR that produced it is already **closed** -- so no
|
||||
*open* PR has the pushed branch as its head (weblib-ci#14, found via
|
||||
weblib-viewer#27). Gitea still remembers which PR a given commit came
|
||||
from, independent of the PR's state:
|
||||
`GET repos/{repo}/commits/{sha}/pull`. Verified against both merge
|
||||
styles used in this org -- an explicit merge commit (weblib-ci#13) and a
|
||||
fast-forward promotion, where the "merge commit" is just the PR branch's
|
||||
own tip (weblib-fs#27) -- both resolve correctly.
|
||||
|
||||
This needs the **full** 40-character sha. A short/abbreviated sha (the
|
||||
kind used for display, e.g. `GITHUB_SHA[:8]`) gets exactly the same
|
||||
`{"message": "pull request does not exist [id: 0, ...]"}` 404 shape as a
|
||||
commit that genuinely has no PR -- so passing one silently looks
|
||||
identical to "not found" rather than erroring. `GITHUB_SHA` itself is
|
||||
already the full sha; this just must not be truncated before it gets
|
||||
here.
|
||||
"""
|
||||
if not sha:
|
||||
return None
|
||||
url = f"{host.rstrip('/')}/api/v1/repos/{repo}/commits/{sha}/pull"
|
||||
req = urllib.request.Request(url)
|
||||
req.add_header("Authorization", f"token {token}")
|
||||
try:
|
||||
with urllib.request.urlopen(req, timeout=30) as res:
|
||||
pull = json.loads(res.read() or b"null")
|
||||
except (urllib.error.URLError, OSError, ValueError):
|
||||
# Same reasoning as pull_request_for_branch: a 404 (no PR for this
|
||||
# commit) is the expected outcome for most calls, not an error to
|
||||
# surface, and a reporter that raises reports nothing.
|
||||
return None
|
||||
return (pull or {}).get("number")
|
||||
|
||||
|
||||
def tail(path, limit):
|
||||
with open(path, "rb") as fh:
|
||||
try:
|
||||
@@ -107,6 +236,28 @@ def comment(host, repo, number, token, body):
|
||||
return json.loads(res.read() or b"null")
|
||||
|
||||
|
||||
def status_description(job, phrase, raw_text, limit=STATUS_TAIL):
|
||||
"""The one-line-ish body for a commit status, when there is no PR to post
|
||||
the full log to. `raw_text` is plain text (no code fence, no ANSI) --
|
||||
either a log tail or `missing_log_note`'s message."""
|
||||
snippet = (raw_text or "").strip().replace("\n", " ").strip()
|
||||
if len(snippet) > limit:
|
||||
snippet = "…" + snippet[-limit:]
|
||||
return f"{job} {phrase}: {snippet}" if snippet else f"{job} {phrase}"
|
||||
|
||||
|
||||
def report_status(host, repo, sha, token, state, description, target_url):
|
||||
url = f"{host.rstrip('/')}/api/v1/repos/{repo}/statuses/{sha}"
|
||||
payload = {"context": "report_job_log", "state": state,
|
||||
"description": description, "target_url": target_url}
|
||||
req = urllib.request.Request(url, data=json.dumps(payload).encode(),
|
||||
method="POST")
|
||||
req.add_header("Content-Type", "application/json")
|
||||
req.add_header("Authorization", f"token {token}")
|
||||
with urllib.request.urlopen(req, timeout=30) as res:
|
||||
return json.loads(res.read() or b"null")
|
||||
|
||||
|
||||
def main(argv=None):
|
||||
ap = argparse.ArgumentParser(description=__doc__,
|
||||
formatter_class=argparse.RawDescriptionHelpFormatter)
|
||||
@@ -119,35 +270,50 @@ def main(argv=None):
|
||||
ap.add_argument("--pr", type=int, default=None)
|
||||
ap.add_argument("--job", default=os.environ.get("GITHUB_JOB") or "job")
|
||||
ap.add_argument("--tail", type=int, default=DEFAULT_TAIL)
|
||||
ap.add_argument("--status",
|
||||
default=os.environ.get("JOB_STATUS") or DEFAULT_STATUS,
|
||||
help="outcome of the job being reported: failed (the "
|
||||
"default, and what `if: failure()` means), passed, "
|
||||
"cancelled, skipped, or ${{ job.status }} verbatim")
|
||||
args = ap.parse_args(argv)
|
||||
|
||||
if not args.repo or not args.token:
|
||||
print("no repo or no token; set GITHUB_REPOSITORY and GITEA_TOKEN")
|
||||
return 0
|
||||
|
||||
full_sha = os.environ.get("GITHUB_SHA") or ""
|
||||
sha = full_sha[:8]
|
||||
|
||||
number = args.pr or pull_request_number(os.environ.get("GITHUB_EVENT_PATH"))
|
||||
if number is None:
|
||||
branch = (os.environ.get("GITHUB_HEAD_REF")
|
||||
or os.environ.get("GITHUB_REF_NAME"))
|
||||
number = pull_request_for_branch(args.host, args.repo, args.token, branch)
|
||||
if number is None:
|
||||
print("no pull request for this run: nothing to comment on")
|
||||
return 0
|
||||
# A push to development/main has no *open* PR for the branch it
|
||||
# pushed to -- that branch is the target, not the one that was
|
||||
# merged and closed. Ask Gitea which PR this exact commit came from
|
||||
# instead (weblib-ci#14).
|
||||
number = pull_request_for_sha(args.host, args.repo, args.token, full_sha)
|
||||
|
||||
if not os.path.exists(args.log):
|
||||
text = ("The step produced no log file at "
|
||||
f"`{args.log}` — it failed before the build started.")
|
||||
phrase = status_phrase(args.status)
|
||||
|
||||
log_exists = os.path.exists(args.log)
|
||||
if not log_exists:
|
||||
raw = missing_log_note(args.log, phrase)
|
||||
else:
|
||||
text = "```\n" + tail(args.log, args.tail).rstrip() + "\n```"
|
||||
raw = tail(args.log, args.tail).rstrip()
|
||||
|
||||
sha = (os.environ.get("GITHUB_SHA") or "")[:8]
|
||||
body = (f"**`{args.job}` failed**{f' on `{sha}`' if sha else ''}.\n\n"
|
||||
if number is not None:
|
||||
text = ("```\n" + raw + "\n```") if log_exists else raw
|
||||
body = (f"**`{args.job}` {phrase}**{f' on `{sha}`' if sha else ''}.\n\n"
|
||||
"Job logs return 500 on this Gitea, so here is the tail of the "
|
||||
"build output, posted by `tools/report_job_log.py`.\n\n"
|
||||
"build output, posted by [`report_job_log.py`]"
|
||||
"(https://git.chaosbit.de/weblib/weblib-ci).\n\n"
|
||||
f"{text}\n")
|
||||
|
||||
# A failure to report a failure must not itself be silent, but it also must
|
||||
# not mask the real one: the step is already `if: failure()`.
|
||||
# A failure to report a failure must not itself be silent, but it also
|
||||
# must not mask the real one: the step is already `if: failure()`.
|
||||
try:
|
||||
result = comment(args.host, args.repo, number, args.token, body)
|
||||
except urllib.error.HTTPError as e:
|
||||
@@ -157,6 +323,30 @@ def main(argv=None):
|
||||
print(f"reported to {args.repo}#{number}: {(result or {}).get('html_url')}")
|
||||
return 0
|
||||
|
||||
# No PR anywhere -- a commit that was never part of one (a direct push,
|
||||
# or the lookups above genuinely found nothing). There is still no log
|
||||
# endpoint to point at, but a commit status is a channel that does not
|
||||
# need a thread to post into, and this repo's Actions already post
|
||||
# statuses that people read.
|
||||
if not full_sha:
|
||||
print("no pull request and no commit sha for this run: nothing to "
|
||||
"report")
|
||||
return 0
|
||||
|
||||
state = STATUS_STATE.get(phrase, "error")
|
||||
description = status_description(args.job, phrase, raw)
|
||||
target_url = f"{args.host.rstrip('/')}/{args.repo}/commit/{full_sha}"
|
||||
try:
|
||||
report_status(args.host, args.repo, full_sha, args.token, state,
|
||||
description, target_url)
|
||||
except urllib.error.HTTPError as e:
|
||||
print(f"could not report status: {e.code} "
|
||||
f"{e.read()[:300].decode('utf-8', 'replace')}", file=sys.stderr)
|
||||
return 0
|
||||
print(f"reported to {args.repo}@{sha} as a commit status "
|
||||
"(no pull request found)")
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
|
||||
+106
-16
@@ -13,12 +13,35 @@ hand.
|
||||
**An issue with no dependencies at all is never touched.** Only issues that
|
||||
have at least one dependency are managed:
|
||||
|
||||
* any blocker not positively closed -> ensure the label is present
|
||||
* every blocker closed -> ensure the label is absent
|
||||
* any blocker positively open -> ensure the label is present
|
||||
* every blocker positively closed -> ensure the label is absent
|
||||
* any blocker's state cannot be confirmed either way -> leave the label
|
||||
exactly as it is, and say so in the log
|
||||
|
||||
The asymmetry is deliberate. A dependency in *another* repo is not fully
|
||||
visible to the Actions token, which is scoped to one repo, so "not open" and
|
||||
"closed" are not the same statement.
|
||||
"closed" are not the same statement. Gitea does not error or omit such an
|
||||
entry -- it substitutes a zero-value stub (`id: 0`, every other field blank or
|
||||
null, including `state: ""`), so `d.get("state") != "closed"` used to treat it
|
||||
as "open forever". That kept a cross-repo `Status/Blocked` from ever being
|
||||
removed (weblib-ci#11) and, worse, made the reconciler *reassert* the label
|
||||
after a human removed it by hand (weblib-archive#30, 2026-09-09): it re-ran the
|
||||
same "not closed" test against the same unreadable stub and put the label back.
|
||||
|
||||
Which runs did that is worth stating precisely, because the obvious answer is
|
||||
wrong. It was **not** the every-15-minutes schedule. Scheduled and `issues`
|
||||
events run the workflow from the *default* branch, and weblib-archive's default
|
||||
branch is `main`, which still carries a stale pre-org-label-migration local copy
|
||||
at `tools/sync_blocked_label.py`. That copy resolves the label with
|
||||
`repos/{repo}/labels`, both repos now have **zero** repo-level labels (all 25
|
||||
are org-level), so it returns None and exits with "skipped: no 'Status/Blocked'
|
||||
label in this repo" — writing nothing, ever. Measured 2026-09-09: 377 scheduled
|
||||
plus 54 `issues` runs, all no-ops. The reassertions came from `pull_request` and
|
||||
`workflow_dispatch` runs on branches cut from `development`, which do consume
|
||||
this shared script. So the symptom appears on PR activity, not on a timer.
|
||||
See weblib-archive#70 and cfbypass#35 for the stale copies themselves. Treating "cannot tell" as its own outcome -- change
|
||||
nothing, log the issue number -- fixes both without guessing an answer the
|
||||
token cannot support. See `is_resolved()`.
|
||||
|
||||
That matters because `Status/Blocked` is also applied by hand for reasons the
|
||||
graph knows nothing about -- weblib-archive#29 and #30 are blocked on a
|
||||
@@ -45,6 +68,18 @@ import urllib.request
|
||||
DEFAULT_LABEL = "Status/Blocked"
|
||||
DEFAULT_HOST = "https://git.chaosbit.de"
|
||||
|
||||
#: Statuses that mean a human has decided something about this issue which
|
||||
#: outranks the dependency graph. bit, 2026-09-07: *"the reconciler must not
|
||||
#: touch issues that are already on hold or abandoned"*.
|
||||
#:
|
||||
#: "Not touch" is literal - neither add nor remove. Adding would be actively
|
||||
#: destructive once `Status/*` is exclusive again, because the add would
|
||||
#: *replace* the human's label rather than sit beside it, and `Status/On Hold`
|
||||
#: is precisely what makes the backlog sweep skip an issue. A parked issue
|
||||
#: would silently become an available one, every fifteen minutes, with nothing
|
||||
#: in the log to say so.
|
||||
HANDS_OFF = ("Status/On Hold", "Status/Abandoned")
|
||||
|
||||
|
||||
class Forge:
|
||||
def __init__(self, base, token):
|
||||
@@ -75,18 +110,47 @@ class Forge:
|
||||
|
||||
|
||||
def label_id(forge, repo, name):
|
||||
"""The label's id, or None if this repo has no such label.
|
||||
"""The label's id, or None if neither the repo nor its org has one.
|
||||
|
||||
Repo first, then the organisation. Labels moved to the org on 2026-09-07
|
||||
(weblib-archive#63) and `repos/<r>/labels` now returns `[]` in all five
|
||||
repos, which made this return None everywhere and turned the whole script
|
||||
into a silent no-op - every run printed "skipped" and reported success.
|
||||
Checking both means it does not care how a given instance is arranged.
|
||||
|
||||
Returning None rather than exiting matters when several repos are passed:
|
||||
aborting on the third would leave the first two already modified, which is
|
||||
a worse state than doing nothing. weblib-viewer has no labels at all.
|
||||
a worse state than doing nothing.
|
||||
"""
|
||||
for label in forge.get(f"repos/{repo}/labels?limit=100"):
|
||||
owner = repo.split("/")[0]
|
||||
for path in (f"repos/{repo}/labels?limit=100",
|
||||
f"orgs/{owner}/labels?limit=100"):
|
||||
try:
|
||||
labels = forge.get(path) or []
|
||||
except urllib.error.HTTPError:
|
||||
# A user-owned repo has no org endpoint; not an error worth dying on.
|
||||
continue
|
||||
for label in labels:
|
||||
if label["name"] == name:
|
||||
return label["id"]
|
||||
return None
|
||||
|
||||
|
||||
def is_resolved(dep):
|
||||
"""False for a dependency entry the token could not actually read.
|
||||
|
||||
Gitea's `/dependencies` response omits nothing -- an entry the token
|
||||
cannot see still gets an element, but one where every field is Go's
|
||||
zero value: `id: 0`, `state: ""`, `repository: null`, etc. `id` is the
|
||||
field that can never legitimately be 0 for a real issue, so it is the
|
||||
one thing safe to key on -- `state` happens to be empty today, but a
|
||||
future Gitea populating it while still stubbing the rest would silently
|
||||
revive the bug this guards against if the check were on `state` instead.
|
||||
Measured against a live cross-repo case, 2026-09-09: see weblib-ci#11.
|
||||
"""
|
||||
return dep.get("id", 0) != 0
|
||||
|
||||
|
||||
def open_items(forge, repo):
|
||||
"""Open issues *and* pull requests. They share the /issues endpoint."""
|
||||
out = []
|
||||
@@ -110,23 +174,49 @@ def reconcile(forge, repo, label_name, dry_run):
|
||||
changed = []
|
||||
for item in open_items(forge, repo):
|
||||
number = item["number"]
|
||||
names = {l["name"] for l in item.get("labels") or []}
|
||||
# A human has already ruled on this one. Leave it entirely alone -
|
||||
# neither add nor remove - rather than letting the dependency graph
|
||||
# overwrite a deliberate decision. See HANDS_OFF.
|
||||
held = names.intersection(HANDS_OFF)
|
||||
if held:
|
||||
print(f" hands off {repo}#{number} ({', '.join(sorted(held))})",
|
||||
flush=True)
|
||||
continue
|
||||
deps = forge.get(f"repos/{repo}/issues/{number}/dependencies") or []
|
||||
if not deps:
|
||||
# No edges, no opinion. See the docstring.
|
||||
continue
|
||||
# Fail safe: only an *explicitly closed* dependency counts as done.
|
||||
# A cross-repo blocker is not fully visible to the Actions token, which
|
||||
# is scoped to one repo -- weblib-archive#30 is blocked by
|
||||
# weblib/cfbypass#1, and the runner read that entry as not-open and
|
||||
# took the label off while cfbypass#1 was still open. Anything whose
|
||||
# state we cannot positively confirm keeps the issue blocked.
|
||||
blockers = [d for d in deps if d.get("state") != "closed"]
|
||||
# Three-way split, not two. A dependency the token could not read
|
||||
# comes back as a zero-value stub (see is_resolved) rather than being
|
||||
# omitted, and its blank `state` must not be read as "open" or
|
||||
# "closed" -- either guess has burned us. An *open* blocker anywhere
|
||||
# in the list is decisive on its own (positive evidence, add/keep the
|
||||
# label) regardless of what else is unresolvable; only "should this
|
||||
# come off" needs every entry to be positively closed.
|
||||
open_blockers = [d for d in deps if is_resolved(d) and d.get("state") != "closed"]
|
||||
unresolved = [d for d in deps if not is_resolved(d)]
|
||||
labelled = any(l["name"] == label_name for l in item.get("labels") or [])
|
||||
if blockers and not labelled:
|
||||
action, detail = "add", f"blocked by {', '.join('#%d' % d['number'] for d in blockers)}"
|
||||
if unresolved and not open_blockers:
|
||||
# Cannot confirm every blocker is closed, and no *known* open one
|
||||
# to justify keeping/adding the label on its own merits either.
|
||||
# The reconciler's job is to keep the label in step with what it
|
||||
# can see; when it cannot see part of the graph it has no
|
||||
# business asserting either state. This is what replaces the old
|
||||
# "not closed -> still blocked" fail-safe, which kept a
|
||||
# cross-repo label stuck forever and then started *reasserting*
|
||||
# it over a human's hand-removal (weblib-archive#30, weblib-ci#11)
|
||||
# -- silence was the actual defect, not the direction of the
|
||||
# guess.
|
||||
print(f" unknown {repo}#{number}: {len(unresolved)} blocker(s) unresolvable "
|
||||
f"with this token -- leaving {label_name!r} as-is ({'present' if labelled else 'absent'})",
|
||||
flush=True)
|
||||
continue
|
||||
elif open_blockers and not labelled:
|
||||
action, detail = "add", f"blocked by {', '.join('#%d' % d['number'] for d in open_blockers)}"
|
||||
if not dry_run:
|
||||
forge.post(f"repos/{repo}/issues/{number}/labels", {"labels": [lid]})
|
||||
elif not blockers and labelled:
|
||||
elif not open_blockers and not unresolved and labelled:
|
||||
action, detail = "remove", "every blocker is closed"
|
||||
if not dry_run:
|
||||
# PUT the whole set minus this label, rather than
|
||||
|
||||
Executable
+485
@@ -0,0 +1,485 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Tests for `report_job_log.py`.
|
||||
|
||||
Standard library only, like everything else here, and offline: the Gitea it
|
||||
posts to is a `http.server` on localhost that keeps the comments it is sent, so
|
||||
a test can *re-read* what was written instead of trusting an exit code of 0.
|
||||
|
||||
Run them with any python3:
|
||||
|
||||
python3 test_report_job_log.py
|
||||
|
||||
The point of most of them is the comment **header**. `report_job_log.py` used
|
||||
to hardcode the word "failed", so a step guarded by `if: always()` posted a
|
||||
failure report for a job that had passed (weblib-ci#8, found in
|
||||
weblib-viewer#10). The header is the one thing a reader sees before the log, so
|
||||
it is the one thing worth asserting on.
|
||||
"""
|
||||
|
||||
import contextlib
|
||||
import io
|
||||
import json
|
||||
import os
|
||||
import subprocess
|
||||
import sys
|
||||
import threading
|
||||
import unittest
|
||||
from http.server import BaseHTTPRequestHandler, HTTPServer
|
||||
|
||||
import report_job_log
|
||||
|
||||
HERE = os.path.dirname(os.path.abspath(__file__))
|
||||
SCRIPT = os.path.join(HERE, "report_job_log.py")
|
||||
|
||||
# Environment variables the script reads. Every test clears all of them, so a
|
||||
# stray one in the ambient shell (this suite is meant to be runnable on a
|
||||
# runner, where several of these are set) cannot change a result.
|
||||
ACTIONS_ENV = [
|
||||
"GITHUB_REPOSITORY", "GITHUB_EVENT_PATH", "GITHUB_SERVER_URL",
|
||||
"GITHUB_SHA", "GITHUB_JOB", "GITHUB_HEAD_REF", "GITHUB_REF_NAME",
|
||||
"GITEA_HOST", "GITEA_TOKEN", "GITHUB_TOKEN", "JOB_STATUS",
|
||||
]
|
||||
|
||||
|
||||
class FakeGitea(BaseHTTPRequestHandler):
|
||||
"""Just enough of the API: list open PRs, resolve a commit's merged PR,
|
||||
accept a comment, and accept a commit status."""
|
||||
|
||||
comments = [] # class-level; reset per test
|
||||
statuses = []
|
||||
pulls = []
|
||||
# sha -> pull dict, mirroring GET .../commits/{sha}/pull. A sha absent
|
||||
# from this map 404s, the same shape Gitea returns for a commit that was
|
||||
# never part of any PR.
|
||||
pull_for_sha = {}
|
||||
|
||||
def _send(self, code, payload):
|
||||
raw = json.dumps(payload).encode()
|
||||
self.send_response(code)
|
||||
self.send_header("Content-Type", "application/json")
|
||||
self.send_header("Content-Length", str(len(raw)))
|
||||
self.end_headers()
|
||||
self.wfile.write(raw)
|
||||
|
||||
def do_GET(self):
|
||||
if "/commits/" in self.path and self.path.endswith("/pull"):
|
||||
sha = self.path.split("/commits/", 1)[1].rsplit("/pull", 1)[0]
|
||||
pull = type(self).pull_for_sha.get(sha)
|
||||
if pull is None:
|
||||
return self._send(
|
||||
404, {"message": "pull request does not exist "
|
||||
"[id: 0, issue_id: 0]"})
|
||||
return self._send(200, pull)
|
||||
if "/pulls" in self.path:
|
||||
return self._send(200, type(self).pulls)
|
||||
self._send(404, {"message": "no"})
|
||||
|
||||
def do_POST(self):
|
||||
length = int(self.headers.get("Content-Length") or 0)
|
||||
payload = json.loads(self.rfile.read(length) or b"{}")
|
||||
if "/statuses/" in self.path:
|
||||
type(self).statuses.append({
|
||||
"path": self.path,
|
||||
"auth": self.headers.get("Authorization"),
|
||||
"context": payload.get("context"),
|
||||
"state": payload.get("state"),
|
||||
"description": payload.get("description", ""),
|
||||
"target_url": payload.get("target_url"),
|
||||
})
|
||||
return self._send(201, {"id": 1, "state": payload.get("state")})
|
||||
type(self).comments.append({
|
||||
"path": self.path,
|
||||
"auth": self.headers.get("Authorization"),
|
||||
"body": payload.get("body", ""),
|
||||
})
|
||||
self._send(201, {"html_url": "http://example.invalid/c/1"})
|
||||
|
||||
def log_message(self, *a): # keep the test output readable
|
||||
pass
|
||||
|
||||
|
||||
class ServerTestCase(unittest.TestCase):
|
||||
"""A test that runs the whole entry point against the fake forge."""
|
||||
|
||||
def setUp(self):
|
||||
FakeGitea.comments = []
|
||||
FakeGitea.statuses = []
|
||||
FakeGitea.pulls = []
|
||||
FakeGitea.pull_for_sha = {}
|
||||
self.server = HTTPServer(("127.0.0.1", 0), FakeGitea)
|
||||
self.host = "http://127.0.0.1:%d" % self.server.server_address[1]
|
||||
# `shutdown()` only takes effect on the next poll, so the default
|
||||
# 0.5s interval charged this suite half a second per test -- measured
|
||||
# 4.6s for nine tests, 0.6s after.
|
||||
self.thread = threading.Thread(
|
||||
target=self.server.serve_forever, kwargs={"poll_interval": 0.01},
|
||||
daemon=True)
|
||||
self.thread.start()
|
||||
self.addCleanup(self.server.server_close)
|
||||
self.addCleanup(self.server.shutdown)
|
||||
|
||||
self.log = os.path.join(self.mkdtemp(), "build.log")
|
||||
with open(self.log, "w") as fh:
|
||||
fh.write("nix-build \x1b[31msomething\x1b[0m\nDISTINCTIVE-LINE\n")
|
||||
|
||||
for name in ACTIONS_ENV:
|
||||
os.environ.pop(name, None)
|
||||
|
||||
def mkdtemp(self):
|
||||
import tempfile
|
||||
d = tempfile.mkdtemp()
|
||||
self.addCleanup(lambda: __import__("shutil").rmtree(d,
|
||||
ignore_errors=True))
|
||||
return d
|
||||
|
||||
def posted(self):
|
||||
"""What the server actually stored, read back from the server.
|
||||
|
||||
An exit status of 0 is not evidence that a comment was written: the
|
||||
script deliberately swallows HTTP errors so that a failure to report
|
||||
does not mask the failure being reported, and returns 0 either way.
|
||||
"""
|
||||
return FakeGitea.comments
|
||||
|
||||
def posted_statuses(self):
|
||||
"""Same idea as `posted()`, for the commit-status fallback."""
|
||||
return FakeGitea.statuses
|
||||
|
||||
def run_main(self, *argv):
|
||||
"""The entry point in-process, so a traceback is readable."""
|
||||
out = io.StringIO()
|
||||
with contextlib.redirect_stdout(out):
|
||||
code = report_job_log.main([
|
||||
self.log, "--repo", "weblib/weblib-ci", "--host", self.host,
|
||||
"--token", "t0ken", "--pr", "8", "--job", "tests", *argv])
|
||||
self.assertEqual(code, 0)
|
||||
self.assertIn("reported to weblib/weblib-ci#8", out.getvalue())
|
||||
return code
|
||||
|
||||
def run_cli(self, *argv, env=None):
|
||||
"""The entry point as a subprocess: argv parsing and shebang included."""
|
||||
full = dict(os.environ)
|
||||
full.update(env or {})
|
||||
proc = subprocess.run(
|
||||
[sys.executable, SCRIPT, self.log, "--repo", "weblib/weblib-ci",
|
||||
"--host", self.host, "--token", "t0ken", "--pr", "8",
|
||||
"--job", "tests", *argv],
|
||||
capture_output=True, text=True, env=full, cwd=HERE)
|
||||
return proc
|
||||
|
||||
|
||||
class TestHeader(ServerTestCase):
|
||||
|
||||
def test_default_is_still_failed(self):
|
||||
"""Backward compatibility: the four repos pass only the log path."""
|
||||
self.run_main()
|
||||
body = self.posted()[0]["body"]
|
||||
self.assertIn("**`tests` failed**", body)
|
||||
self.assertNotIn("passed", body)
|
||||
|
||||
def test_status_passed(self):
|
||||
self.run_main("--status", "passed")
|
||||
body = self.posted()[0]["body"]
|
||||
self.assertIn("**`tests` passed**", body)
|
||||
self.assertNotIn("failed", body)
|
||||
|
||||
def test_job_status_expression_success(self):
|
||||
"""`${{ job.status }}` yields `success`, not `passed`."""
|
||||
self.run_main("--status", "success")
|
||||
self.assertIn("**`tests` passed**", self.posted()[0]["body"])
|
||||
|
||||
def test_job_status_expression_failure(self):
|
||||
self.run_main("--status", "failure")
|
||||
self.assertIn("**`tests` failed**", self.posted()[0]["body"])
|
||||
|
||||
def test_cancelled(self):
|
||||
self.run_main("--status", "cancelled")
|
||||
self.assertIn("**`tests` was cancelled**", self.posted()[0]["body"])
|
||||
|
||||
def test_case_and_whitespace_are_not_a_failure_report(self):
|
||||
self.run_main("--status", " Success \n")
|
||||
self.assertIn("**`tests` passed**", self.posted()[0]["body"])
|
||||
|
||||
def test_unknown_status_still_reports(self):
|
||||
"""A status the table has not heard of must not cost us the log.
|
||||
|
||||
`argparse(choices=...)` would exit 2 here, and the log this script
|
||||
exists to surface would never be posted.
|
||||
"""
|
||||
self.run_main("--status", "neutral")
|
||||
body = self.posted()[0]["body"]
|
||||
self.assertIn("**`tests` finished with status `neutral`**", body)
|
||||
self.assertIn("DISTINCTIVE-LINE", body)
|
||||
|
||||
def test_empty_status_falls_back_to_the_default(self):
|
||||
self.run_main("--status", "")
|
||||
self.assertIn("**`tests` failed**", self.posted()[0]["body"])
|
||||
|
||||
def test_sha_and_log_survive_a_non_default_status(self):
|
||||
os.environ["GITHUB_SHA"] = "0123456789abcdef"
|
||||
self.run_main("--status", "passed")
|
||||
body = self.posted()[0]["body"]
|
||||
self.assertIn("**`tests` passed** on `01234567`.", body)
|
||||
self.assertIn("DISTINCTIVE-LINE", body)
|
||||
self.assertNotIn("\x1b[31m", body) # ANSI still stripped
|
||||
|
||||
|
||||
class TestEnvironment(ServerTestCase):
|
||||
|
||||
def test_job_status_env_is_honoured(self):
|
||||
proc = self.run_cli(env={"JOB_STATUS": "success"})
|
||||
self.assertEqual(proc.returncode, 0, proc.stderr)
|
||||
self.assertIn("**`tests` passed**", self.posted()[0]["body"])
|
||||
|
||||
def test_flag_beats_the_environment(self):
|
||||
proc = self.run_cli("--status", "failed", env={"JOB_STATUS": "success"})
|
||||
self.assertEqual(proc.returncode, 0, proc.stderr)
|
||||
self.assertIn("**`tests` failed**", self.posted()[0]["body"])
|
||||
|
||||
def test_unset_environment_is_still_failed(self):
|
||||
proc = self.run_cli()
|
||||
self.assertEqual(proc.returncode, 0, proc.stderr)
|
||||
self.assertIn("**`tests` failed**", self.posted()[0]["body"])
|
||||
|
||||
|
||||
class TestMissingLog(ServerTestCase):
|
||||
|
||||
def test_missing_log_when_failed(self):
|
||||
self.log = os.path.join(self.mkdtemp(), "absent.log")
|
||||
self.run_main()
|
||||
self.assertIn("it failed before the build started",
|
||||
self.posted()[0]["body"])
|
||||
|
||||
def test_missing_log_when_passed_does_not_claim_a_failure(self):
|
||||
self.log = os.path.join(self.mkdtemp(), "absent.log")
|
||||
self.run_main("--status", "passed")
|
||||
body = self.posted()[0]["body"]
|
||||
self.assertIn("nothing was captured", body)
|
||||
self.assertNotIn("failed", body)
|
||||
|
||||
|
||||
class TestEndToEnd(ServerTestCase):
|
||||
"""The CLI, driven the way a workflow drives it: environment, no `--pr`."""
|
||||
|
||||
def _event_file(self, number):
|
||||
path = os.path.join(self.mkdtemp(), "event.json")
|
||||
with open(path, "w") as fh:
|
||||
json.dump({"pull_request": {"number": number}}, fh)
|
||||
return path
|
||||
|
||||
def _run(self, *argv, env=None):
|
||||
full = dict(os.environ)
|
||||
full.pop("GITEA_HOST", None)
|
||||
full.update({
|
||||
"GITHUB_REPOSITORY": "weblib/weblib-ci",
|
||||
"GITHUB_SERVER_URL": self.host,
|
||||
"GITEA_TOKEN": "t0ken",
|
||||
"GITHUB_JOB": "probe",
|
||||
"GITHUB_SHA": "abcdef0123456789",
|
||||
"GITHUB_EVENT_PATH": self._event_file(42),
|
||||
})
|
||||
full.update(env or {})
|
||||
return subprocess.run([sys.executable, SCRIPT, self.log, *argv],
|
||||
capture_output=True, text=True, env=full,
|
||||
cwd=HERE)
|
||||
|
||||
def test_always_guarded_probe_that_passed(self):
|
||||
proc = self._run("--status", "success")
|
||||
self.assertEqual(proc.returncode, 0, proc.stderr)
|
||||
self.assertIn("reported to weblib/weblib-ci#42", proc.stdout)
|
||||
|
||||
# The write is only real if the resource says so.
|
||||
posted = self.posted()
|
||||
self.assertEqual(len(posted), 1, posted)
|
||||
self.assertEqual(posted[0]["path"],
|
||||
"/api/v1/repos/weblib/weblib-ci/issues/42/comments")
|
||||
self.assertEqual(posted[0]["auth"], "token t0ken")
|
||||
body = posted[0]["body"]
|
||||
self.assertIn("**`probe` passed** on `abcdef01`.", body)
|
||||
self.assertNotIn("failed", body)
|
||||
self.assertIn("DISTINCTIVE-LINE", body)
|
||||
|
||||
def test_failure_guarded_job_is_unchanged(self):
|
||||
proc = self._run()
|
||||
self.assertEqual(proc.returncode, 0, proc.stderr)
|
||||
self.assertIn("**`probe` failed** on `abcdef01`.",
|
||||
self.posted()[0]["body"])
|
||||
|
||||
def test_branch_lookup_path_still_works(self):
|
||||
"""No event payload: the PR is found by head branch, and reported."""
|
||||
FakeGitea.pulls = [{"number": 7, "head": {"ref": "fix/x"}}]
|
||||
proc = self._run("--status", "passed",
|
||||
env={"GITHUB_EVENT_PATH": "",
|
||||
"GITHUB_HEAD_REF": "fix/x"})
|
||||
self.assertEqual(proc.returncode, 0, proc.stderr)
|
||||
self.assertIn("reported to weblib/weblib-ci#7", proc.stdout)
|
||||
self.assertIn("**`probe` passed**", self.posted()[0]["body"])
|
||||
|
||||
def test_help_still_renders(self):
|
||||
"""argparse `%`-expands `help=`; a stray `%` there breaks `--help`."""
|
||||
proc = subprocess.run([sys.executable, SCRIPT, "--help"],
|
||||
capture_output=True, text=True, cwd=HERE)
|
||||
self.assertEqual(proc.returncode, 0, proc.stderr)
|
||||
self.assertIn("--status", proc.stdout)
|
||||
|
||||
|
||||
# A commit that landed by merging a now-closed PR into development/main --
|
||||
# weblib-ci#14's actual failure mode. A real 40-char sha, because
|
||||
# pull_request_for_sha silently treats a truncated one the same as "no PR".
|
||||
PUSH_SHA = "c9d6db67d9c20f8ab133f4fb5fea359e9c474274"
|
||||
|
||||
|
||||
class TestPushWithNoOpenPR(ServerTestCase):
|
||||
"""A `push` event: no event payload, no GITHUB_HEAD_REF, and the pushed
|
||||
branch (GITHUB_REF_NAME) is the *target* of a merge, not a PR head -- the
|
||||
exact shape of weblib-ci#14 (weblib-viewer run 1769, PR#27 merged into
|
||||
`development`)."""
|
||||
|
||||
def _run(self, *argv, env=None):
|
||||
full = dict(os.environ)
|
||||
full.pop("GITEA_HOST", None)
|
||||
full.update({
|
||||
"GITHUB_REPOSITORY": "weblib/weblib-ci",
|
||||
"GITHUB_SERVER_URL": self.host,
|
||||
"GITEA_TOKEN": "t0ken",
|
||||
"GITHUB_JOB": "publish",
|
||||
"GITHUB_SHA": PUSH_SHA,
|
||||
"GITHUB_REF_NAME": "development",
|
||||
})
|
||||
full.update(env or {})
|
||||
return subprocess.run([sys.executable, SCRIPT, self.log, *argv],
|
||||
capture_output=True, text=True, env=full,
|
||||
cwd=HERE)
|
||||
|
||||
def test_falls_back_to_the_pr_the_push_merged(self):
|
||||
"""No open PR has `development` as its head -- it never does -- but
|
||||
the merge commit itself resolves to the PR that produced it, even
|
||||
though that PR is now closed."""
|
||||
FakeGitea.pulls = []
|
||||
FakeGitea.pull_for_sha = {PUSH_SHA: {"number": 27, "state": "closed"}}
|
||||
|
||||
proc = self._run()
|
||||
self.assertEqual(proc.returncode, 0, proc.stderr)
|
||||
self.assertIn("reported to weblib/weblib-ci#27", proc.stdout)
|
||||
|
||||
posted = self.posted()
|
||||
self.assertEqual(len(posted), 1, posted)
|
||||
self.assertEqual(posted[0]["path"],
|
||||
"/api/v1/repos/weblib/weblib-ci/issues/27/comments")
|
||||
self.assertIn("DISTINCTIVE-LINE", posted[0]["body"])
|
||||
self.assertEqual(self.posted_statuses(), [],
|
||||
"a resolvable PR must not also get a status")
|
||||
|
||||
def test_branch_lookup_still_wins_when_it_applies(self):
|
||||
"""Ordering: an *open* PR whose head matches the branch is found
|
||||
first, so the (potentially stale) sha lookup is never even tried."""
|
||||
FakeGitea.pulls = [{"number": 5, "head": {"ref": "development"}}]
|
||||
FakeGitea.pull_for_sha = {PUSH_SHA: {"number": 999}}
|
||||
|
||||
proc = self._run()
|
||||
self.assertEqual(proc.returncode, 0, proc.stderr)
|
||||
self.assertIn("reported to weblib/weblib-ci#5", proc.stdout)
|
||||
self.assertNotIn("999", proc.stdout)
|
||||
|
||||
def test_no_pr_anywhere_reports_a_commit_status(self):
|
||||
"""Not even the sha lookup finds anything -- a commit that was never
|
||||
part of any PR. This must still report *somewhere*, not silently
|
||||
exit 0 the way #14 did."""
|
||||
FakeGitea.pulls = []
|
||||
FakeGitea.pull_for_sha = {}
|
||||
|
||||
proc = self._run()
|
||||
self.assertEqual(proc.returncode, 0, proc.stderr)
|
||||
self.assertIn(f"reported to weblib/weblib-ci@{PUSH_SHA[:8]}",
|
||||
proc.stdout)
|
||||
self.assertEqual(self.posted(), [], "no PR exists to comment on")
|
||||
|
||||
statuses = self.posted_statuses()
|
||||
self.assertEqual(len(statuses), 1, statuses)
|
||||
s = statuses[0]
|
||||
self.assertEqual(s["path"],
|
||||
f"/api/v1/repos/weblib/weblib-ci/statuses/{PUSH_SHA}")
|
||||
self.assertEqual(s["auth"], "token t0ken")
|
||||
self.assertEqual(s["state"], "failure")
|
||||
self.assertIn("publish", s["description"])
|
||||
self.assertIn("DISTINCTIVE-LINE", s["description"])
|
||||
self.assertIn(PUSH_SHA, s["target_url"])
|
||||
|
||||
def test_commit_status_state_follows_the_reported_outcome(self):
|
||||
FakeGitea.pulls = []
|
||||
FakeGitea.pull_for_sha = {}
|
||||
|
||||
for status, want_state in [("passed", "success"),
|
||||
("cancelled", "warning"),
|
||||
("skipped", "skipped"),
|
||||
("neutral", "error")]:
|
||||
with self.subTest(status=status):
|
||||
FakeGitea.statuses = []
|
||||
proc = self._run("--status", status)
|
||||
self.assertEqual(proc.returncode, 0, proc.stderr)
|
||||
self.assertEqual(self.posted_statuses()[0]["state"], want_state)
|
||||
|
||||
def test_no_sha_and_no_pr_does_not_crash_or_post_anything(self):
|
||||
"""The one case with nowhere at all to report: still exits 0 and
|
||||
says so, rather than raising or posting a status with an empty sha
|
||||
in the URL."""
|
||||
FakeGitea.pulls = []
|
||||
FakeGitea.pull_for_sha = {}
|
||||
|
||||
proc = self._run(env={"GITHUB_SHA": "", "GITHUB_REF_NAME": "development"})
|
||||
self.assertEqual(proc.returncode, 0, proc.stderr)
|
||||
self.assertIn("nothing to report", proc.stdout)
|
||||
self.assertEqual(self.posted(), [])
|
||||
self.assertEqual(self.posted_statuses(), [])
|
||||
|
||||
def test_existing_pr_event_path_is_unaffected(self):
|
||||
"""The ordinary pull_request-event path -- unrelated to any of the
|
||||
push-triggered fallback -- still goes straight to the comment, and
|
||||
never touches the sha or status endpoints."""
|
||||
path = os.path.join(self.mkdtemp(), "event.json")
|
||||
with open(path, "w") as fh:
|
||||
json.dump({"pull_request": {"number": 11}}, fh)
|
||||
FakeGitea.pull_for_sha = {PUSH_SHA: {"number": 999}} # must be ignored
|
||||
|
||||
proc = self._run(env={"GITHUB_EVENT_PATH": path})
|
||||
self.assertEqual(proc.returncode, 0, proc.stderr)
|
||||
self.assertIn("reported to weblib/weblib-ci#11", proc.stdout)
|
||||
self.assertEqual(self.posted_statuses(), [])
|
||||
|
||||
|
||||
class TestStatusPhrase(unittest.TestCase):
|
||||
"""The unit under the header, without a server."""
|
||||
|
||||
def test_table(self):
|
||||
for status, want in [
|
||||
("failed", "failed"), ("failure", "failed"),
|
||||
("passed", "passed"), ("success", "passed"),
|
||||
("succeeded", "passed"),
|
||||
("cancelled", "was cancelled"), ("canceled", "was cancelled"),
|
||||
("skipped", "was skipped"),
|
||||
("SUCCESS", "passed"), (" failure ", "failed"),
|
||||
(None, "failed"), ("", "failed"),
|
||||
]:
|
||||
with self.subTest(status=status):
|
||||
self.assertEqual(report_job_log.status_phrase(status), want)
|
||||
|
||||
def test_unknown_is_verbatim(self):
|
||||
self.assertEqual(report_job_log.status_phrase("weird"),
|
||||
"finished with status `weird`")
|
||||
|
||||
def test_a_backtick_in_an_unknown_status_cannot_escape_the_code_span(self):
|
||||
"""The verbatim value sits in a code span inside a **bold** header, so
|
||||
a backtick in it would close the span and let the rest render as
|
||||
markdown. `${{ job.status }}` is workflow-author-controlled rather than
|
||||
hostile, but this script is public and shared by four repos."""
|
||||
phrase = report_job_log.status_phrase("x` **loud** `y")
|
||||
self.assertEqual(phrase, "finished with status `x **loud** y`")
|
||||
self.assertEqual(phrase.count("`"), 2)
|
||||
|
||||
def test_default_constant_is_failed(self):
|
||||
"""Named so that changing it is a deliberate act, not a typo."""
|
||||
self.assertEqual(report_job_log.DEFAULT_STATUS, "failed")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main(verbosity=2)
|
||||
@@ -0,0 +1,300 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Tests for `sync_blocked_label.py`.
|
||||
|
||||
Standard library only, like everything else here, and offline: the Gitea it
|
||||
talks to is a `http.server` on localhost that keeps whatever state a test
|
||||
hands it, so assertions read that state back rather than trusting an exit
|
||||
code.
|
||||
|
||||
Run them with any python3:
|
||||
|
||||
python3 test_sync_blocked_label.py
|
||||
|
||||
The point of most of them is the three-way split in `reconcile()` --
|
||||
positively open, positively closed, or unresolvable -- because collapsing it
|
||||
back to two is exactly how weblib-ci#11 happened. A cross-repo dependency the
|
||||
Actions token cannot read comes back from Gitea as a zero-value stub
|
||||
(`id: 0`, `state: ""`, `repository: null`, ...) rather than being omitted or
|
||||
erroring, and the old code's `d.get("state") != "closed"` read that as "still
|
||||
open" forever -- which first meant the label could never come off, and then
|
||||
(weblib-archive#30, 2026-09-09) meant the reconciler *reasserted* the label
|
||||
every 15 minutes over a human's hand-removal.
|
||||
"""
|
||||
|
||||
import contextlib
|
||||
import io
|
||||
import json
|
||||
import os
|
||||
import threading
|
||||
import unittest
|
||||
from http.server import BaseHTTPRequestHandler, HTTPServer
|
||||
|
||||
import sync_blocked_label as sbl
|
||||
|
||||
HERE = os.path.dirname(os.path.abspath(__file__))
|
||||
|
||||
|
||||
def resolved(number, state, repo="weblib/cfbypass"):
|
||||
"""A dependency entry the token *could* read -- the shape a same-repo
|
||||
blocker, or a cross-repo one under a token that can see it, comes back
|
||||
as."""
|
||||
owner, name = repo.split("/")
|
||||
return {
|
||||
"id": 1000 + number, "number": number, "state": state,
|
||||
"repository": {"owner": owner, "name": name, "full_name": repo},
|
||||
}
|
||||
|
||||
|
||||
def unresolvable():
|
||||
"""The zero-value stub Gitea substitutes for an entry the token cannot
|
||||
read. Measured live against weblib-archive#68 and #30, 2026-09-09 --
|
||||
every field blank or zero, `state` included."""
|
||||
return {
|
||||
"id": 0, "url": "", "html_url": "", "number": 0, "user": None,
|
||||
"state": "", "repository": None, "labels": None, "milestone": None,
|
||||
}
|
||||
|
||||
|
||||
class FakeGitea(BaseHTTPRequestHandler):
|
||||
"""Just enough of the API to drive `reconcile()`.
|
||||
|
||||
Class-level state, reset per test in `setUp`:
|
||||
- `labels`: {"repo"/"org": [label dicts]}
|
||||
- `issues`: [issue dicts] -- the open items in the one repo under test
|
||||
- `deps`: {issue number: [dependency entries]}
|
||||
- `label_calls`: every POST/PUT to a labels endpoint, in order
|
||||
"""
|
||||
|
||||
labels_repo = []
|
||||
labels_org = []
|
||||
issues = []
|
||||
deps = {}
|
||||
label_calls = []
|
||||
|
||||
def _send(self, code, payload):
|
||||
raw = json.dumps(payload).encode()
|
||||
self.send_response(code)
|
||||
self.send_header("Content-Type", "application/json")
|
||||
self.send_header("Content-Length", str(len(raw)))
|
||||
self.end_headers()
|
||||
self.wfile.write(raw)
|
||||
|
||||
def do_GET(self):
|
||||
cls = type(self)
|
||||
if self.path.startswith("/api/v1/repos/") and self.path.endswith(
|
||||
"/labels?limit=100"):
|
||||
return self._send(200, cls.labels_repo)
|
||||
if self.path.startswith("/api/v1/orgs/") and "/labels" in self.path:
|
||||
return self._send(200, cls.labels_org)
|
||||
if "/issues?state=open" in self.path:
|
||||
page = int(self.path.rsplit("page=", 1)[1])
|
||||
return self._send(200, cls.issues if page == 1 else [])
|
||||
if "/dependencies" in self.path:
|
||||
number = int(self.path.rsplit("/issues/", 1)[1].split("/")[0])
|
||||
return self._send(200, cls.deps.get(number, []))
|
||||
self._send(404, {"message": "no route: " + self.path})
|
||||
|
||||
def _record_and_ack(self, method):
|
||||
length = int(self.headers.get("Content-Length") or 0)
|
||||
payload = json.loads(self.rfile.read(length) or b"{}")
|
||||
type(self).label_calls.append({
|
||||
"method": method, "path": self.path, "labels": payload["labels"],
|
||||
})
|
||||
self._send(200, [])
|
||||
|
||||
def do_POST(self):
|
||||
self._record_and_ack("POST")
|
||||
|
||||
def do_PUT(self):
|
||||
self._record_and_ack("PUT")
|
||||
|
||||
def log_message(self, *a): # keep the test output readable
|
||||
pass
|
||||
|
||||
|
||||
BLOCKED_LABEL = {"id": 145, "name": "Status/Blocked"}
|
||||
ON_HOLD_LABEL = {"id": 152, "name": "Status/On Hold"}
|
||||
|
||||
|
||||
def issue(number, *label_dicts):
|
||||
return {"number": number, "labels": list(label_dicts)}
|
||||
|
||||
|
||||
class ReconcileTestCase(unittest.TestCase):
|
||||
|
||||
def setUp(self):
|
||||
FakeGitea.labels_repo = []
|
||||
FakeGitea.labels_org = [BLOCKED_LABEL]
|
||||
FakeGitea.issues = []
|
||||
FakeGitea.deps = {}
|
||||
FakeGitea.label_calls = []
|
||||
self.server = HTTPServer(("127.0.0.1", 0), FakeGitea)
|
||||
self.host = "http://127.0.0.1:%d" % self.server.server_address[1]
|
||||
self.thread = threading.Thread(
|
||||
target=self.server.serve_forever, kwargs={"poll_interval": 0.01},
|
||||
daemon=True)
|
||||
self.thread.start()
|
||||
self.addCleanup(self.server.server_close)
|
||||
self.addCleanup(self.server.shutdown)
|
||||
self.forge = sbl.Forge(self.host, "t0ken")
|
||||
|
||||
def reconcile(self, dry_run=False):
|
||||
out = io.StringIO()
|
||||
with contextlib.redirect_stdout(out):
|
||||
changed = sbl.reconcile(self.forge, "weblib/weblib-archive",
|
||||
"Status/Blocked", dry_run)
|
||||
return changed, out.getvalue()
|
||||
|
||||
def calls_for(self, number):
|
||||
needle = f"/issues/{number}/labels"
|
||||
return [c for c in FakeGitea.label_calls if c["path"].endswith(needle)]
|
||||
|
||||
|
||||
class TestNoDependencyEdges(ReconcileTestCase):
|
||||
|
||||
def test_untouched(self):
|
||||
"""An issue with no dependency edges is never touched -- it may be
|
||||
blocked by hand for a reason the graph does not know about
|
||||
(weblib-archive#29/#30's original 'blocked on a decision' case)."""
|
||||
FakeGitea.issues = [issue(1, BLOCKED_LABEL)]
|
||||
FakeGitea.deps = {1: []}
|
||||
changed, _ = self.reconcile()
|
||||
self.assertEqual(changed, [])
|
||||
self.assertEqual(self.calls_for(1), [])
|
||||
|
||||
|
||||
class TestSameRepoUnchanged(ReconcileTestCase):
|
||||
"""Pins the pre-existing, already-correct behaviour so a fix aimed at the
|
||||
cross-repo case cannot quietly change the path that already worked --
|
||||
same-repo dependencies always resolve fully (no id:0 stub ever shows up
|
||||
for them), so `unresolved` is always empty here regardless of the fix."""
|
||||
|
||||
def test_open_blocker_adds_the_label(self):
|
||||
FakeGitea.issues = [issue(5)]
|
||||
FakeGitea.deps = {5: [resolved(1, "open", "weblib/weblib-archive")]}
|
||||
changed, log = self.reconcile()
|
||||
self.assertEqual(changed, [(5, "add", "blocked by #1")])
|
||||
self.assertEqual(self.calls_for(5),
|
||||
[{"method": "POST", "path": "/api/v1/repos/weblib/weblib-archive/issues/5/labels",
|
||||
"labels": [145]}])
|
||||
self.assertIn("add Status/Blocked on weblib/weblib-archive#5", log)
|
||||
|
||||
def test_closed_blocker_removes_the_label(self):
|
||||
FakeGitea.issues = [issue(6, BLOCKED_LABEL)]
|
||||
FakeGitea.deps = {6: [resolved(2, "closed", "weblib/weblib-archive")]}
|
||||
changed, _ = self.reconcile()
|
||||
self.assertEqual(changed, [(6, "remove", "every blocker is closed")])
|
||||
self.assertEqual(self.calls_for(6),
|
||||
[{"method": "PUT", "path": "/api/v1/repos/weblib/weblib-archive/issues/6/labels",
|
||||
"labels": []}])
|
||||
|
||||
def test_already_correct_is_a_no_op_either_direction(self):
|
||||
FakeGitea.issues = [
|
||||
issue(7, BLOCKED_LABEL), # open blocker, already labelled
|
||||
issue(8), # closed blocker, already unlabelled
|
||||
]
|
||||
FakeGitea.deps = {
|
||||
7: [resolved(1, "open", "weblib/weblib-archive")],
|
||||
8: [resolved(1, "closed", "weblib/weblib-archive")],
|
||||
}
|
||||
changed, _ = self.reconcile()
|
||||
self.assertEqual(changed, [])
|
||||
self.assertEqual(FakeGitea.label_calls, [])
|
||||
|
||||
|
||||
class TestUnresolvableCrossRepoBlocker(ReconcileTestCase):
|
||||
"""The fix. Reproduces weblib-archive#68/#30 exactly: a dependency entry
|
||||
that comes back as Gitea's zero-value stub because the Actions token
|
||||
cannot read the other repo."""
|
||||
|
||||
def test_never_removes_a_label_it_cannot_confirm_is_clear(self):
|
||||
"""The original bug: every blocker closed, cross-repo, label stuck
|
||||
forever."""
|
||||
FakeGitea.issues = [issue(30, BLOCKED_LABEL)]
|
||||
FakeGitea.deps = {30: [unresolvable(), unresolvable()]}
|
||||
changed, log = self.reconcile()
|
||||
self.assertEqual(changed, [])
|
||||
self.assertEqual(self.calls_for(30), [])
|
||||
self.assertIn("unknown weblib/weblib-archive#30", log)
|
||||
self.assertIn("2 blocker(s) unresolvable", log)
|
||||
self.assertIn("present", log)
|
||||
|
||||
def test_never_reasserts_a_label_a_human_removed(self):
|
||||
"""The sharper bug found live on 2026-09-09: same dependency graph
|
||||
as above, but a human had already taken the label off. The old code
|
||||
put it straight back on the next run; this must leave it off."""
|
||||
FakeGitea.issues = [issue(30)] # label removed by hand
|
||||
FakeGitea.deps = {30: [unresolvable(), unresolvable()]}
|
||||
changed, log = self.reconcile()
|
||||
self.assertEqual(changed, [])
|
||||
self.assertEqual(self.calls_for(30), [])
|
||||
self.assertIn("absent", log)
|
||||
|
||||
def test_a_known_open_blocker_still_wins_over_an_unresolvable_one(self):
|
||||
"""Positive evidence of an open blocker is decisive on its own --
|
||||
this is not a reason to *loosen* the guard, only to stop guessing
|
||||
where there is no evidence either way."""
|
||||
FakeGitea.issues = [issue(9)]
|
||||
FakeGitea.deps = {9: [resolved(1, "open", "weblib/weblib-archive"),
|
||||
unresolvable()]}
|
||||
changed, _ = self.reconcile()
|
||||
self.assertEqual(changed, [(9, "add", "blocked by #1")])
|
||||
|
||||
def test_an_unresolvable_blocker_blocks_removal_even_with_other_closed(self):
|
||||
"""One confirmed-closed same-repo blocker plus one unresolvable
|
||||
cross-repo one must not remove the label -- the unresolvable one
|
||||
might still be open."""
|
||||
FakeGitea.issues = [issue(10, BLOCKED_LABEL)]
|
||||
FakeGitea.deps = {10: [resolved(1, "closed", "weblib/weblib-archive"),
|
||||
unresolvable()]}
|
||||
changed, log = self.reconcile()
|
||||
self.assertEqual(changed, [])
|
||||
self.assertEqual(self.calls_for(10), [])
|
||||
self.assertIn("unknown weblib/weblib-archive#10", log)
|
||||
|
||||
def test_is_resolved_keys_on_id_not_on_state(self):
|
||||
"""bit, 2026-09-09: key on `id == 0`, not on the empty `state` --
|
||||
`state` is a symptom that happens to be visible today; a future
|
||||
Gitea could populate it while still stubbing the rest of the
|
||||
entry."""
|
||||
self.assertFalse(sbl.is_resolved(unresolvable()))
|
||||
self.assertFalse(sbl.is_resolved({"id": 0, "state": "closed"}))
|
||||
self.assertTrue(sbl.is_resolved({"id": 7, "state": ""}))
|
||||
|
||||
|
||||
class TestHandsOff(ReconcileTestCase):
|
||||
|
||||
def test_on_hold_is_never_touched_even_with_open_deps(self):
|
||||
FakeGitea.issues = [issue(11, ON_HOLD_LABEL)]
|
||||
FakeGitea.deps = {11: [resolved(1, "open", "weblib/weblib-archive")]}
|
||||
changed, log = self.reconcile()
|
||||
self.assertEqual(changed, [])
|
||||
self.assertEqual(self.calls_for(11), [])
|
||||
self.assertIn("hands off weblib/weblib-archive#11", log)
|
||||
|
||||
|
||||
class TestNoLabelInRepo(ReconcileTestCase):
|
||||
|
||||
def test_skips_everything_and_calls_nothing(self):
|
||||
FakeGitea.labels_repo = []
|
||||
FakeGitea.labels_org = []
|
||||
FakeGitea.issues = [issue(1)]
|
||||
changed, log = self.reconcile()
|
||||
self.assertEqual(changed, [])
|
||||
self.assertEqual(FakeGitea.label_calls, [])
|
||||
self.assertIn("skipped: no 'Status/Blocked' label in this repo", log)
|
||||
|
||||
|
||||
class TestDryRun(ReconcileTestCase):
|
||||
|
||||
def test_dry_run_reports_but_does_not_call(self):
|
||||
FakeGitea.issues = [issue(5)]
|
||||
FakeGitea.deps = {5: [resolved(1, "open", "weblib/weblib-archive")]}
|
||||
changed, log = self.reconcile(dry_run=True)
|
||||
self.assertEqual(changed, [(5, "add", "blocked by #1")])
|
||||
self.assertEqual(FakeGitea.label_calls, [])
|
||||
self.assertIn("would add", log)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main(verbosity=2)
|
||||
Reference in New Issue
Block a user