rlsbl v0.113.0 /rlsbl.commands.watch
Updated
On this page

Watch command that polls GitHub Actions CI workflow runs for a given commit SHA, reports pass/fail/in-progress status, and suggests retry on failure.

#rlsbl.commands.watch

#rlsbl.commands.watch

Watch command that polls GitHub Actions CI workflow runs for a given commit SHA and reports pass, fail, or in-progress status.

#_open_url

python
def _open_url(url)

Open a URL in the default browser. Non-fatal if unavailable.

#_release_url

python
def _release_url(repo_slug)

Try to find the latest release tag and return its GitHub URL. Returns None on failure.

#_notify

python
def _notify(title, body, url=None)

Send a desktop notification. If url is provided, opens it only when the user clicks the notification action.

#_classify_failure

python
def _classify_failure(log_text)

Classify a CI failure log tail to decide whether a retry is worthwhile.

Returns one of: - "infra": the run died at the infrastructure layer, below the code under test (no runner acquired, GitHub's action download service failed, or nothing executed at all). Rerun its FAILED jobs once: the run established nothing, and on a resumed release it is the only run the candidate will ever have. - "deterministic": a signature indicating the failure recurs identically on retry (test failures, compile/build errors, config/validation errors, workflow syntax errors, missing-secret/auth denials). Never retry. - "transient": an infrastructure-flake signature (network timeouts, 5xx, rate limits, runner-lost/cancelled). Retry once. - "unknown": no signature matched. Treated by the caller as transient (retry once) -- this DEFAULT preserves the historical blind-retry behavior for failures we don't yet recognize, rather than suppressing a retry that might have succeeded.

Precedence is infra, then deterministic, then transient. Infra comes first because a run that never executed still emits a log tail full of job names, echoed commands and workflow text, which the deterministic signatures match by accident -- that is precisely how a provider-wide outage was read as a code failure and left a resumed release permanently unrunnable. Determin- istic then outranks transient so a log holding both a hard error and incidental network chatter follows the hard error, which is the real cause.

An EMPTY tail is infra, not unknown: --log-failed returning nothing for a failed run means no step ever produced output, so the job died before execution -- runner never acquired, actions never resolved, or the run was cancelled while still queued.

#_fetch_failure_log

python
def _fetch_failure_log(run_id, config=None)

Fetch the tail of the failing step's log for a run via gh.

Runs gh run view <id> --log-failed (through run_gh, so GH_REPO resolution and thread-safe env handling apply) with a bounded timeout, and returns the last _LOG_TAIL_LINES lines joined as a single string. Propagates any exception from the gh call so the caller can emit a loud fallback note.

#_retry_workflow

python
def _retry_workflow(workflow_name, repo_slug, label, failed_run_id, failed_only=False)

Re-run a failed workflow run in place once and watch the new attempt.

Uses gh run rerun <failed_run_id> on the SAME run id. GitHub re-executes the run as a new attempt on the failed run's original commit, so (a) no duplicate check-run is created (a fresh gh workflow run dispatch would poison the publish gate) and (b) the retry runs on the failed commit rather than branch HEAD.

failed_only adds --failed, restarting only the jobs that failed. That is the right shape for an infrastructure-killed run: the jobs that DID acquire a runner and pass keep their result instead of being thrown back into a queue that just proved unreliable. A transient flake keeps the full rerun -- nothing there says which jobs the flake really touched.

Because the run id is unchanged, there is no dispatched-run-identification dance: we simply watch failed_run_id for its new attempt's conclusion.

Returns a result dict with name, passed, and run_id. Returns None if the rerun could not be triggered.

#_watch_single_run

python
def _watch_single_run(ci_run, label, repo_slug, retried_lock=None, retried_workflows=None, timeout=3600)

Watch a single CI run. Returns a dict with name, passed, and run_id.

When retried_lock and retried_workflows are provided, deduplicates retries so that only one retry is dispatched per workflow name even when multiple runs from the same workflow fail concurrently.

Retries are in-place reruns (gh run rerun) that reuse the failed run's own id, so no cross-thread run-id bookkeeping is needed: the late re-poll recognizes the reran run by its unchanged id.

#_watch_runs

python
def _watch_runs(runs, label, repo_slug, retried_lock=None, retried_workflows=None, timeout=3600)

Watch all runs in parallel. Returns list of result dicts.

retried_lock and retried_workflows may be passed in so that retry deduplication state is shared across multiple _watch_runs calls (initial watch + late re-poll watch). Fresh state is created when omitted.

Single-run pools deliberately go through the same thread-pool path so every run participates in the shared retry-dedup machinery.

#_repo_root

python
def _repo_root()

Best-effort git toplevel; falls back to cwd when not in a git repo.

#_has_publish_workflow_on_disk

python
def _has_publish_workflow_on_disk()

Check if any .github/workflows file looks like a publish workflow.

Resolved from the git repo root (not cwd) so monorepo package-dir invocations still find the repo-level workflow files.

#_is_publish_workflow

python
def _is_publish_workflow(name)

Return True if the workflow name matches a publish/deploy/release pattern.

#_print_workflow_audit

python
def _print_workflow_audit(results)

Print a summary of which workflows ran and flag missing publish workflows.

Returns True if a missing-publish warning was printed (for testability).

#_resolve_run_ids

python
def _resolve_run_ids(run_ids)

Resolve run IDs to run info dicts via gh run view.

#poll_runs

python
def poll_runs(commit_sha, max_attempts=30, interval=4)

Poll gh run list until at least one run appears.

Returns a list of run dicts (may be empty if nothing found after all attempts). Default timeout is ~120s (30 attempts * 4s interval).

#_discovery_budget

python
def _discovery_budget(timeout, discovery_grace)

Clamp the discovery grace so it never eats the whole CI budget.

Returns (effective_grace, clamped). At least one poll interval is always granted -- a budget too small to poll even once is the operator's declaration, not a reason to skip discovery entirely.

#_timeout_verdict

python
def _timeout_verdict(results)

Aggregate per-run results into a verdict.

A genuine failure outranks a timeout: if any run definitively failed, the answer is known (:data:CI_RED) and fix-forward is the right remedy, even if a sibling run was still going when the budget ran out.

#CIWaitError

Raised when the CI wait cannot reach a verdict at all.

Distinct from a red verdict: this means the repository declares push-triggered CI but the pushed candidate produced no runs, so there is nothing to gate on and proceeding would publish an unverified commit.

#_workflow_triggers_on_push

python
def _workflow_triggers_on_push(path)

Return True if a workflow file declares a push trigger.

Parsed rather than grepped so a push mentioned in a job step or a comment is not mistaken for a trigger. Unparseable files are treated as NOT push-triggered -- an unreadable workflow cannot be evidence that CI is expected.

#push_triggered_workflows

python
def push_triggered_workflows(repo_root=None)

Return the names of .github/workflows files that trigger on push.

An empty result means the repository has no push-triggered CI: the release flow then proceeds without a CI gate instead of blocking forever on runs that can never appear. The distinction is an observable fact about the repository, not a fallback.

#wait_for_ci_green

python
def wait_for_ci_green(commit_sha, *, timeout, check_filters, log=None, config=None, repo_root=None, label=None, discovery_grace=CI_DISCOVERY_GRACE_SECONDS)

Block until every CI run for commit_sha concludes.

Returns (verdict, results) where verdict is one of :data:CI_GREEN, :data:CI_RED, :data:CI_TIMEOUT, or :data:CI_NOT_CONFIGURED.

Five explicit outcomes, no silent waits:

  • The repository declares no push-triggered workflow -> :data:CI_NOT_CONFIGURED

immediately (nothing can ever run; blocking would hang the release).

  • Push-triggered workflows exist but no run appears for the commit within

the discovery grace -> :class:CIWaitError (hard error).

  • Runs appear and conclude -> :data:CI_GREEN / :data:CI_RED (transient

failures retried once, deterministic ones not).

  • Runs appear but timeout expires with some still unresolved ->

:data:CI_TIMEOUT. Distinct from red on purpose: nothing was proven about those runs, so the remedy is to check their status, not to fix code that may be perfectly fine.

  • Every run concludes green, but the RELEASING PROJECT'S OWN check runs

were absent or did not conclude success (typically skipped by the monorepo CI router's paths filter) -> :class:rlsbl.ci_checks.ProjectCINotRunError (hard error). A green workflow run is not evidence that this project's CI ran: this is the exact predicate the publish gate applies later, checked here so the two gates cannot disagree and tag a version that can never publish.

check_filters is mandatory (a list of :class:rlsbl.ci_checks.CheckFilter, from :func:rlsbl.ci_checks.release_check_filters) precisely because it must never be forgotten: a caller that omitted it would silently re-open the divergence. Pass an empty list only when there is no project to verify.

timeout is the WHOLE budget: discovery is spent inside it and is capped at half of it, so the completion wait always keeps at least half.

#run_cmd

python
def run_cmd(registry, args, flags)

Watch all CI runs for a commit until they complete.

Usage: rlsbl watch [] rlsbl watch --run-id [--run-id ] Defaults to HEAD if no commit SHA is provided.

Search