CI budget: caches, xdist and the run's wall time (#219, #220)
Sources: GitHub’s dependency-caching reference
(https://docs.github.com/en/actions/reference/workflows-and-actions/dependency-caching), astral-sh/setup-uv at our pin
37802ad (v7.6.0: action.yml, README.md), pytest-xdist 3.8.0 docs (https://pytest-xdist.readthedocs.io/en/stable/),
the REST workflow-runs endpoint (https://docs.github.com/en/rest/actions/workflow-runs#get-a-workflow-run), and our own
runs (gh api repos/yellow-pine/noggin/actions/runs/<id>/jobs).
Caches
Section titled “Caches”- Scope: a run restores caches of its own branch or the default branch; a PR run also its base branch. A PR run’s own
save goes to
refs/pull/<n>/mergeand only that PR’s re-runs see it. So only main’s saves seed other PRs and the queue. - Lookup: exact
keyin the run’s branch, thenrestore-keysprefixes (the most recent match), then the same on the default branch. An exact hit setscache-hittotrue; arestore-keysmatch restores but leaves itfalse. A key is written once. ours-convex-:restore-keys: ours-convex-lets a PR that changes a part (oruv.lock) start from main’s latest entry;make oursandmjcf.convexrebuild only what their.keyfiles miss.- setup-uv:
prune-cachedefaults totrue, which runsuv cache prune --cibefore saving: it keeps only wheels built from source. Every wheel we install is pre-built, so main’s entry held ~1.2 KB and every run downloaded everything. setup-uv has norestore-keys; its key hashesuv.lockandpyproject.toml, and it saves only on success.
pytest-xdist
Section titled “pytest-xdist”-n auto: one worker per CPU (4 on a public repo’subuntu-latest).--dist loadgroupsends every test with the samexdist_groupmark to one worker and spreads the rest:model/test_mjcf.pygroups by document, so each worker does not build and sweep every document again.--durations=Nreports across workers.- Each worker sets
PYTEST_XDIST_WORKER(gw0, …). Shared files:mjcf.writetruncates and rewritesmodel/build/mjcf/<doc>.xmlwhile another worker may load it, somodel/conftest.pygives each worker its own folder.mjcf.convexdecomposes a stale part into a shared folder, so a cold model job decomposes everything first (python -m model.mjcf, its own process pool) and the tests only read.
The budget
Section titled “The budget”- The run’s start:
run_started_atofGET /repos/{owner}/{repo}/actions/runs/{run_id}(the latest attempt’s start; needsactions: read). Each job’s last step compares it with now; the run’s wall time is its last job’s. - Changed after main’s run of #238 (2b2db76) waited 20 min for runners behind the queue and failed every budget: the
clock starts at the earliest
started_atof the attempt’s jobs that ran (GET .../runs/{run_id}/attempts/{n}/jobs; queued, waiting, skipped, cancelled and earlier attempts’ jobs left out), and on main an over-budget run warns instead of failing (Cansin). - Changed after #265’s “re-run failed jobs” failed its budget at 239 s for 73 s of work (#267): that attempt’s jobs
list repeats the jobs it kept with the new
run_attemptbut their oldstarted_at, so the clock now starts at the first job that ran from the attempt’srun_started_at(GET .../runs/{run_id}/attempts/{n}) on. - Changed after 14 Site Checks re-runs in 3.5 h (#438, #442): an over-budget PR run warns like main’s instead of failing; a timing budget is not a correctness gate.
- Warm: the parts cache hit its exact key (
actions/cachecache-hit). Arestore-keysmatch is cold: some parts are rebuilt.
Measured (PR #231)
Section titled “Measured (PR #231)”-n autogives 4 workers onubuntu-latest(no psutil:os.sched_getaffinity). Warm, the model tests take ~194 s on 4 workers with the 4 cores saturated (the durations sum to ~720 s): xdist alone cannot bring the model job under 3 min, so it runs as two shards (model hero,model mujoco) behind one requiredmodelcheck.mjcf.specstarts a spawn process pool for every document it builds, and with--dist workstealeach xdist worker built and swept each document again: everytest_mjcf.pytest took ~20 s, and the 47 non-hero model tests 165 s.- The unpruned uv cache is 448 MB and restores in ~7 s;
uv syncthen takes 0.2 s. Without a cache setup-uv takes ~1 s anduv syncdownloads every wheel in ~3 s. So check.yml keeps no uv cache: it would be slower, and eachuv.lockchange would put another 448 MB into the repo’s 10 GB cache, evicting the parts and Blender entries.
noggin v0.2.26-6-g09bf9b7 · 09bf9b7