Skip to content

test(promotion): pin the exact files promoting an assistant writes to the target - #68

Open
scott-lowe-vapi wants to merge 1 commit into
mainfrom
test/promotion-golden-baseline
Open

scott-lowe-vapi wants to merge 1 commit into
mainfrom
test/promotion-golden-baseline

Conversation

@scott-lowe-vapi

@scott-lowe-vapi scott-lowe-vapi commented Oct 3, 2026 •

Copy link
Copy Markdown
Contributor

Value

V.A.L.U.E. tier: small — a tests-only baseline under the remaining promotion changes for TEST-141 (TEST-141). It changes no behaviour. It's small rather than micro because it guards a blast-radius path: promotion into production.

  • Problem: the code that moves assistants from a lower org into production is the riskiest path in gitops. The existing tests check pieces of it (canonicalization, bindings, deletions) but never the full set of files a promotion writes. So a refactor could change what lands in prod without any test failing.

  • Who it affects: teams that promote resources between orgs (dev → staging → prod), and reviewers of the promotion changes stacked above this (fix(promotion): commit the files of transitions that applied when a later one fails #65, feat(promotion): gate promotion out of an org on a passing check #66), which now have to pass this test unchanged.

  • What changes: new tests/promotion-golden.test.ts. It runs one staging → prod promotion through promotionPlanBuild and promotionPlanApply, then asserts the plan and every file left in the target org:

    • source UUIDs canonicalized to names (toolIds, a handoff destination);
    • referenced tools and assistants pulled in as dependencies, and an unreferenced staging-only tool left behind;
    • a credential bound by name, and a phone number bound to the target's;
    • a target file the source no longer has deleted;
    • an unrelated target file left untouched;
    • the markdown prompt body preserved;
    • the source org never written.

    A second test re-plans right after the apply and expects no changes.

Evidence of value

Mutation check: each deliberate break of src/promotion.ts turns the test red, and it's green again once the break is reverted.

Break Golden test
Phone binding returns the source phone ❌ fails
Credential binding returns the source UUID ❌ fails
UUID → name canonicalization disabled ❌ fails
None (main) ✅ passes

The expected output was read line by line against promotion's documented behaviour before it was pinned.

Tests: npm test goes from 474 to 476 passing.

Testing plan

Base of the promotion stack: #68 ← #65 ← #66.

Refs TEST-141

🤖 Generated with Claude Code

… the target

A golden test for the path that moves assistants from a lower org into
production. One staging → prod promotion through promotionPlanBuild and
promotionPlanApply, asserting the plan and every file left in the target
org:

- source UUIDs canonicalized to names (toolIds, handoff destinations);
- referenced tools and assistants pulled in as dependencies, and an
  unreferenced staging-only tool left behind;
- credential bindings by name, and a phone number bound to the target's;
- a target file the source no longer has deleted, an unrelated target
  file untouched, the markdown prompt body preserved;
- the source org never written.

A second test re-plans after the apply and expects no changes.

This changes no behaviour: it is a baseline for the promotion changes
stacked above it, which must pass it unchanged.

Refs TEST-141

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant