CI performance dashboard #1182

Maintained by the CI profiler routine, which runs every 4h. Each run replaces this body and carries the history rows over.

Headline (run 2026-10-09 08:17 UTC)

Window: 2026-10-08 08:17 → 2026-10-09 08:17. Now = runs created since the previous profile (04:17).

Sample:

• Job detail for 90 runs (12,459 jobs):
  • 52 CI merge_group runs: every completed one since 04:17, plus every 5th before that
  • 13 CI PR runs (8 non-draft) and 3 CI push runs
  • 1–2 runs for each compatibility workflow
• Run counts per workflow × event come from the total_count of filtered run listings.
• 14 PR timelines.

Waste on merge_group runs (52-run sample):

• So ~40% of merge_group job-minutes went to runs that did not merge.
• Genuine failures since 04:17:
  • test (windows-latest, 1) in 37892957346 at 06:19. This evicted Fix macOS Composer legs red on main via setup-php 2.40.0 #1215 and cancelled the 4 runs behind it, ~2,200 job-min.
  • coverage-docker (sbt) in 37894279963 at 06:34.
• Open ci-janitor PRs for these: Fix Windows flake in read-set replacement test #1237 (Windows flake), Retry HTTP errors in sbt/composer Docker image downloads #1238 (sbt/composer Docker downloads), Retry Docker e2e maven/cargo fixture fetches past registry blips #1222, Retry Bun backtest cells on hosted-fetch 5xx blips #1244.

Job-minutes/day. Estimated as per-run sample × run counts.

• By OS: Linux ~180k, Windows ~40k, macOS ~10.1k.
  • Gradle compat was not re-measured: both sampled PR runs were path-skipped. Last measured: ~58k Linux + ~38k Windows (CI perf: Gradle patch compatibility — PR runs repeat ci.yml's Gradle e2e tier and run nightly-only extras (~17,000 Linux job-min/day) #1177).
• By event:
  • pull_request: 117k Linux, 29k Windows, 0 macOS
  • merge_group: 46k Linux, 8.3k Windows, 7.7k macOS
  • push: 17k Linux, 2.6k Windows, 2.4k macOS
• Largest consumers:
  • CI PR: ~70k Linux, 12k Windows. 311 runs/day; drafts run 1 job, a full non-draft run is ~370 Linux + 44 Windows job-min.
  • CI merge_group: 46k Linux, 8.3k Windows, 7.7k macOS
  • CI push: 17k Linux, 2.6k Windows, 2.4k macOS (CI perf: CI on push to main re-tests the exact SHA the merge queue just passed (~16,000 job-min/day, ~1,800 macOS) #1170)
  • PDM compat PR: 12k Linux
  • Bun compat PR: 7.4k Linux, 8.6k Windows
  • Composer compat PR: 7.9k Linux, 3.1k Windows
  • npm compat PR: 6.2k Linux
  • vlt compat PR: 5.0k Linux, 4.7k Windows
  • The compat numbers rest on 1–2 sampled runs each (CI perf: compatibility workflows (sbt, PDM, vlt, Bun, Composer, +5) — full matrices on ~80% of PR pushes, 0 failures (~80,000 Linux + 26,000 Windows job-min/day) #1198 / PR Run compatibility matrices on PRs only for their own files #1206).

Critical paths now.

• PR: test-release, 24.6 / 30.6 min (p50 / p90). Its Run tests (release) step alone is 29.2 min p50 since 04:17 (CI perf: ci.yml test-release — single 29.5-min job is now the whole PR critical path (~12 min off PR push→ci-ok) #1181).

• merge_group: last job before ci-ok in 27 successful runs:

  last job	runs	detail
  coverage-merge	10	behind coverage-docker (sbt), 11.7 min p50, of which Build sbt image is 5.1 min (CI perf: coverage-docker (sbt) — 5.6-min image rebuild warms 3 unused toolchains; merge-queue critical path (~2.5 min/merge-group run) #1225)
  test (windows-latest, 1/2)	10	shard 2: Run tests 15.0 + Build 2.2 min p50 (CI perf: test (windows/macos) — redundant cargo build step + full debuginfo linking ~240 test binaries (~930 job-min/day, ~300 macOS) #1173)
  Gradle e2e_redirect_gradle_build … gradle_hosted_b	4	13.4 min p50 (CI perf: Gradle e2e shards unbalanced — b-p leg is the merge-queue critical path (~3 min/merge-group run) #1171)
  other Gradle e2e	3

  All three paths finish within 1–2 min of each other, at 20–22 min.

Merge-queue settings (ruleset 24668460): max_entries_to_build 5, max_entries_to_merge 5, ALLGREEN, 120-min timeout.

• New: CI perf (settings): merge queue builds 5 entries at a time — a 13-PR burst took 100 min to drain on a 21-min CI run (~21 min per PR past the 5th) #1248 proposes max_entries_to_build 5 → 8 for human decision.
• The queue advances in lock-step waves of 5, one every ~21 min. In the 06:19–06:52 burst of 13 PRs, the last one merged at 08:01.

History

Open ci-perf issues by ROI

ROI on one scale: (thousands of job-min/day weighted Linux ×1, Windows ×2, macOS ×3, plus 1 per critical-path or PR-latency minute) × confidence ÷ effort (S=1, M=2, L=3). Issues #1176–#1181 carry markers on a different scale; this table re-ranks them.

Merged ci-perf / ci-janitor PRs (last 7 days) and measured effect

Method notes

• GraphQL is blocked in this environment, so merge-queue state came from REST timelines of 14 merged PRs plus merge_group run head branches.
• Paginated job listings follow Link headers to numeric-ID repositories/{id} URLs, which the proxy rejects. Pages 2+ have to be fetched explicitly with &page=N.
• Queued-then-cancelled jobs with no runner count as queue time, not run time.
• About 300 GitHub REST requests this run.

Generated by Claude Code