fix(bundles): retrieve pinned component releases when a catalog advertises a newer version - #4753
Open
muhammadumer-waheed wants to merge 3 commits into
Open
muhammadumer-waheed wants to merge 3 commits into
muhammadumer-waheed wants to merge 3 commits into
Conversation
…tises a newer version Bundle manifests pin component versions for reproducibility, but extension/preset bundle installation compared the pin against the version the catalog currently advertises and refused the install when they differed, even when the pinned release remained available at its original download URL (github#4712). Catalog entries advertise a single release, so the resolver never attempted to retrieve the pinned one. When the pin differs from the advertised version, the bundler now derives the pinned release's URL from the catalog entry's own download_url by substituting the advertised version token in the URL path (bounded token matching: a v-prefixed tag, a versioned asset filename, or a version glued to an archive suffix) and retrieves that release through the catalog's existing download pipeline (HTTPS validation, size limits, safe cache path). The catalog's SHA-256 covers only the advertised release, so a retrieved pinned release is verified against no digest; the same host and the HTTPS rule still apply. When the pinned release cannot be identified from the catalog URL, or the retrieval fails, the error names the pinned and advertised versions instead of a bare pin mismatch or network error. To expose explicit-URL retrieval without duplicating the download pipeline, the fetch stage of ExtensionCatalog.download_extension and PresetCatalog.download_pack is now download_extension_url / download_pack_url, which the ID-based methods delegate to without behavior change (a catalog entry with a null version now names the cached archive "unknown" instead of erroring). Workflows and bundled-asset installs keep their existing hard pin check: a workflow's URL install path carries an interactive untrusted-source confirmation that a derived-URL fetch would bypass, and a bundled asset has no alternative release to retrieve. New regression tests in tests/specify_cli/bundles/test_primitives.py fail on main (the pin mismatch raised before any retrieval attempt) and pass with this change, plus URL-derivation unit tests and explicit-URL download coverage in tests/test_extensions.py and tests/specify_cli/presets/test_catalog.py. Verified end-to-end with a local catalog advertising 0.5.1 while a 0.4.12 release stays available: `specify bundle install` now installs the pinned 0.4.12 release (on main it fails with the reported error). Fixes github#4712 Assisted-by: opencode (model: Qwen3.8-27B (local), autonomous)
mnriem
added
the
triage-can-wait
Contributor
There was a problem hiding this comment.
Copilot review overview
🟡 Changes recommended
Derived downloads can install the wrong manifest version, and valid v-prefixed versions produce incorrect URLs.
Review effort: Balanced
Findings: 1
Open (2)
What changed in this PR
Fixes stale bundle pins by deriving and downloading historical extension/preset release URLs.
Changes:
- Adds explicit-URL catalog download APIs.
- Adds pin-aware URL derivation and error handling.
- Documents and tests pinned-release retrieval.
| File | Description |
|---|---|
src/specify_cli/bundles/primitives.py |
Implements pinned-release resolution. |
src/specify_cli/extensions/__init__.py |
Adds explicit extension URL downloads. |
src/specify_cli/presets/_catalog.py |
Adds explicit preset URL downloads. |
tests/specify_cli/bundles/test_primitives.py |
Tests pin resolution and failures. |
tests/test_extensions.py |
Tests extension URL downloads. |
tests/specify_cli/presets/test_catalog.py |
Tests preset URL downloads. |
docs/reference/bundles.md |
Documents historical-release behavior. |
💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Collaborator
|
Please address Copilot feedback and fix test & lint errors |
…versions in URL derivation Address the Copilot review findings on this PR: - A successful request to a derived pinned-release URL does not prove the served artifact is the pinned release (a redirect, a fallback response, or a mislabeled historical asset can serve a different component or version). The catalog digest covers only the advertised release and the primitive installers trust the archive manifest, so the bundler now verifies the retrieved archive's manifest declares the pinned component (id and normalized version) before installing, and removes the unverified artifact when it does not. - URL derivation now normalizes the optional v/V prefix on both sides: a v-prefixed pin no longer produces "vv0.4.12" URLs, and a v-prefixed advertised version also rewrites the bare-form version token in asset filenames (e.g. v0.5.1 advertises -> v0.4.12/asset-0.4.12.zip). New tests fail on the pre-review code (verified by stashing the src change): 2 URL-derivation regressions and 3 negative archive-verification cases (extension version/id mismatch, preset version mismatch). E2E: a mislabeled 0.4.12 artifact (manifest declares 0.5.1) is refused with a pin-aware error; the happy path still installs the pinned release. Assisted-by: opencode (model: Qwen3.8-27B (local), autonomous)
Collaborator
|
Please address Copilot feedback |
…version positions Address the second Copilot review round: a bounded version token inside a static path component (a repository named "tool-0.5.1") used to be rewritten as well, so deriving a pinned release from https://github.com/acme/tool-0.5.1/releases/download/v0.5.1/tool.zip moved the component's home repository to tool-0.4.12 and the valid pinned release was never fetched. Substitution is now restricted to recognized version positions: the asset filename (the final path segment) and a segment that is exactly a version token (a release tag or a versioned directory). Version-looking static components in any other position are left untouched. New coverage: a repository named tool-0.5.1 keeps its name in the derived URL (release download and archive tag shapes, plus a non-GitHub host); a segment that is exactly a version token is still treated as a version position. The test fails on the pre-fix code (verified by stashing the src change); full suite and ruff unchanged otherwise. Assisted-by: opencode (model: Qwen3.8-27B (local), autonomous)
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #4712.
Summary
Bundle manifests pin component versions for reproducibility, but
specify bundle installcompared each pin against the version the catalog currently advertises and refused the install when they differed — even when the pinned release remained available at its original download URL. Catalog entries advertise a single release, so the pinned release was never even attempted.This change makes the bundler retrieve the pinned release when a catalog entry has moved on:
download_urlby substituting the advertised version token in the URL path (bounded token matching: a v-prefixed tag, a versioned asset filename, or a version glued to an archive suffix; the optionalv/Vprefix is normalized on both the pin and the advertised version, so av0.4.12pin derivesv0.4.12, nevervv0.4.12, and av0.5.1advertisement also rewrites the bare token inasset-0.5.1.zipfilenames) and retrieves that release through the catalog's existing download pipeline. When the pinned release cannot be identified from the catalog URL, or the retrieval fails, the error names the pinned and advertised versions instead of a bare pin mismatch or network error.ExtensionCatalog.download_extensionandPresetCatalog.download_packis nowdownload_extension_url/download_pack_url(HTTPS validation, size-limited fetch, optional SHA-256, archive-format detection, safe cache path), which the ID-based methods delegate to without behavior change.Intentionally out of scope: workflows keep the hard pin check (their URL install path carries an interactive untrusted-source confirmation that a derived-URL fetch would bypass), bundled assets keep the hard check (no alternative release exists), and steps perform no pin check today.
Reproduction (the issue's scenario)
A local catalog server advertising
specassay-checkv0.5.1 while the 0.4.12 release artifact stays available at its original URL; the bundle pins 0.4.12:Error: Extension 'specassay-check' is pinned to version 0.4.12 in the bundle manifest, but the resolved version is 0.5.1. Update the bundle's pinned version or the source before installing.✓ Installed 'test-bundle' (1 added, 0 already present).— the registry records version0.4.12and the installedextension.ymldeclaresversion: 0.4.12.Error: Extension 'specassay-check' is pinned to version 0.4.12 in the bundle manifest, but the catalog now advertises v0.5.1: the retrieved archive declares version '0.5.1' instead of the pinned version 0.4.12. ...— nothing is installed and the unverified artifact is removed.Test plan
New regression tests fail on
main(the pin mismatch raised before any retrieval attempt) and pass with this change; the review-round tests (archive verification, v-prefix derivation, static-path components) fail on their respective pre-fix commits (verified by stashing the src change); the full suite is otherwise unchanged — the 10 pre-existing*_python_parity"composed" failures reproduce identically on cleanmain:uvx ruff@0.15.0 check src tests→ All checks passed.markdownlint-cli2 docs/reference/bundles.md→ 0 issues.Manual test results
Agent: n/a (CLI-level change; verified by running
specify bundle installdirectly) | OS/Shell: macOS/zshspecify bundle install <bundle.yml> --integration copilotspecify bundle install <bundle.yml>(stale pin, non-derivable catalog URL)specify bundle install <bundle.yml>(pinned URL serves a mislabeled artifact)AI disclosure
Implemented with opencode (model: Qwen3.8-27B, running locally; autonomous within the session under user direction, no human line-by-line review before this PR was opened). Extent: issue triage, source analysis, code, tests, end-to-end reproduction, and docs were all generated by the agent.