-
Notifications
You must be signed in to change notification settings - Fork 14.6k
Add PR triage skill #5404
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
tomthorogood
wants to merge
12
commits into
main
Choose a base branch
from
add-pr-triage-skill
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Add PR triage skill #5404
Changes from all commits
Commits
Show all changes
12 commits
Select commit
Hold shift + click to select a range
91fbb2d
Add pr-triage skill documenting PR review workflow
c04fdde
Prioritize blocking PRs before triage table generation
fc9415a
Check for extenuating explanation before rejecting self-promotion PRs
77452cc
Require explicit user request before any PR approval, merge, or close
501edef
Require safe recommendation and consent before CI run approval
274ffea
Restrict trusted github-prefixed PR authors to explicit allowlist
6a1ca20
Allow explicit group requests for triaged PR actions
53f4024
Rerun failed CI after triage table creation and PR merges
acaba89
Permit requested CI approval for human-review PRs
4565198
Prompt for stafftools import after topic merges
d5e7fbc
Revise PR priority order and bot references
tomthorogood c111c42
Avoid automatic CI reruns during PR triage
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,80 @@ | ||
| --- | ||
| name: pr-triage | ||
| description: Triage open pull requests in github/explore, producing a status table with CI results and merge recommendations. Use when asked to review, triage, or summarize open PRs in this repository. | ||
| --- | ||
|
|
||
| # Pull Request Triage Skill | ||
|
|
||
| ## Goal | ||
|
|
||
| Produce a table of open pull requests with CI status and a merge recommendation, following this repository's contribution rules. | ||
|
|
||
| ## Table format | ||
|
|
||
| | PR# | Author login | PR title | CI status | Merge recommendation | Reason | | ||
| |---|---|---|---|---|---| | ||
|
|
||
| - **PR#**: link to the pull request. | ||
| - **CI status**: 🟢 all checks passed, 🟡 checks still running, 🔴 any check failed. | ||
| - **Merge recommendation**: ✔️ looks good, ❌ should be closed without further comment, 🔍 needs human review. | ||
| - **Reason**: fill in for any ❌ recommendation, explaining which rule below applies. | ||
|
|
||
| ## Merge recommendation rules | ||
|
|
||
| 1. **Dependabot PRs** (`app/dependabot`): always ✔️. These are trusted dependency bumps. | ||
| 2. **`github-actions[bot]` automation PRs** (e.g. the collections-renames autofix bot): always ✔️. These are generated by repository-owned workflows, not external submissions. | ||
| 3. **`github-security-bot`**: always ✔️. This is an internal, repository-owned bot, not an external contributor. | ||
| Only these exact two logins are opted in under the `github-` prefix. A `github-` prefix alone does not establish that an account is repository-owned; apply the ordinary contribution rules to any other `github-*` login. | ||
| 4. **PRs editing core repository metadata** (workflows, CI config, `Gemfile`, docs, etc. — anything outside `topics/` or `collections/`) submitted by an external contributor: ❌. This repository only accepts topic/collection contributions from the community; infrastructure changes need maintainer review through other channels. | ||
| 5. **PRs that add or edit a topic or collection**: | ||
| - Check the PR description against `.github/PULL_REQUEST_TEMPLATE.md`. If the required checkboxes for the selected contribution type are not checked, recommend ❌ — per `CONTRIBUTING.md`, incomplete templates are closed without comment. | ||
|
tomthorogood marked this conversation as resolved.
|
||
| - If the checkboxes are complete but the contribution reads as self-promotion (e.g. the author is adding their own repository to a topic or collection), check the PR description and comments for a disclosed, extenuating explanation (e.g. independent evidence of community adoption, or a maintainer acknowledgment) that might justify an exception. Absent such justification, recommend ❌ citing the "Avoid conflicts of interest" guideline in `CONTRIBUTING.md`. | ||
| - If the checkboxes are complete, the change is substantive, and it isn't self-promotion, recommend ✔️. | ||
| - If CI is failing (🔴) for a PR that otherwise looks acceptable, or the contribution's value is ambiguous, recommend 🔍 for human review rather than guessing. | ||
|
|
||
| ## Workflow | ||
|
|
||
| **Never approve, merge, or close PRs automatically. Require an explicit user request identifying the PRs, either by number or by a clearly defined group (e.g. "merge all recommended PRs").** A ✔️ or ❌ recommendation alone is not authorization. Approving CI workflow runs has a separate, restricted consent step below. | ||
|
|
||
| For prioritizing which PRs matter most when the user does ask for merges, note that these often correct data that other PRs' CI depends on, so they're worth flagging as high priority in that order: | ||
|
|
||
| 1. **Autofix PRs** (e.g. the `github-actions[bot]` collections-renames PR). | ||
| 2. **Dependabot PRs** (`dependabot[bot]`). | ||
| 3. **`github-security-bot` PRs** (the only other opted-in `github-` login). | ||
|
|
||
| When asked to "Triage the PRs for github/explore": | ||
|
|
||
| 1. List open PRs with `gh pr list` including CI status (`statusCheckRollup`), read each PR's body/checkboxes, diff, and any bot triage comments (e.g. the maintainer triage comment posted by `explore-triage-commenter`), and apply the merge recommendation rules. Check workflow runs for manual-approval gates. | ||
|
tomthorogood marked this conversation as resolved.
|
||
| 2. In the first response, create the table as an artifact and open it in the editor canvas. Do not wait for CI approvals or branch updates before showing the table. | ||
| 3. Show existing CI results in the table without automatically rerunning failures. If another PR merges, refresh the table using the latest results. Rerun failed CI only when the user requests it or after a targeted diagnosis identifies a reason to retry. | ||
| 4. If any PRs recommended ✔️ have runs requiring approval, prompt the user: "N PRs look safe but have runs requiring approval. Would you like me to approve them to run?" Use the number of qualifying PRs, not the number of runs. Do not ask this for PRs recommended 🔍 or ❌. Do not approve any runs before the user agrees. | ||
|
tomthorogood marked this conversation as resolved.
|
||
| 5. Do not approve, merge, or close PRs — including ones recommended ✔️ or ❌ — unless the user explicitly requests the action for specific PRs or a clearly defined group of PRs. | ||
|
|
||
| After confirming that a PR contributing to `topics/` has merged, prompt the user to import the merged topic via [stafftools](https://admin.github.com/biztools/topics). The link may return 403 or 404 until the user is SSO-authenticated; do not treat that as an import failure or attempt the import on their behalf. | ||
|
|
||
| ## Updating PR branches | ||
|
|
||
| Only when asked to update PRs from the base branch, use `gh pr update-branch <number>` for each open PR. Dependabot, `github-actions[bot]`, and `github-security-bot` PRs are still valid targets for this — being "always accepted" for merge doesn't exempt them from branch updates. | ||
|
|
||
| ## CI check approval | ||
|
|
||
| When the user requests a rerun, or a targeted diagnosis identifies a reason to retry, use `gh run rerun <run_id> --failed -R github/explore` for failed GitHub Actions runs on the open PR's current head commit and update the table with the pending status. Do not rerun older commits' failures or repeatedly rerun the same failure while waiting for results. A run awaiting manual approval is not a failed run to retry; follow the consent rule below instead. | ||
|
|
||
| Workflow runs that require manual approval (e.g. first-time contributors) can be approved with: | ||
|
|
||
| ``` | ||
| gh api -X POST repos/github/explore/actions/runs/<run_id>/approve | ||
| ``` | ||
|
tomthorogood marked this conversation as resolved.
|
||
|
|
||
| Offer the grouped consent prompt only for PRs independently recommended ✔️. Runs for PRs recommended 🔍 may also be approved if the user explicitly requests CI approval for those PRs; never approve runs for PRs recommended ❌. The runs must actually require approval (`action_required` or `waiting`); a 🔴 CI status from a completed, non-blocked run is a real failure, not a pending approval. Before approving, recheck each PR's recommendation and run state. Consent to run CI does not authorize approving or merging the PR. | ||
|
|
||
| ## Diagnosing CI failures | ||
|
|
||
| Before recommending a fix, distinguish between: | ||
|
|
||
| - **A data problem**: the failure stems from stale or inconsistent content already in `main` (e.g. a topic still aliasing another topic that has since been split out on its own). Note this in the analysis; the failing PR may not be at fault, and a separate PR may already fix the underlying data. | ||
| - **A logic problem**: the failure stems from the test/workflow logic itself not accounting for a valid case (e.g. a CI job not exempting an automation bot's own PRs from a check designed for external submissions). | ||
|
|
||
| ## Tone | ||
|
|
||
| This repository is public. Keep all analysis, PR descriptions, and comments matter-of-fact and welcoming toward external contributors — describe issues neutrally (e.g. "the description doesn't check the required boxes") rather than in dismissive or judgmental language. | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.