Skip to content

feat: discover Rust entity capabilities in the Python adapter - #179

Merged
sidmohan0 merged 3 commits into
devfrom
feature/core-capability-adapter
Sep 28, 2026
Merged

sidmohan0 merged 3 commits into
devfrom
feature/core-capability-adapter

Conversation

@sidmohan0

@sidmohan0 sidmohan0 commented Sep 28, 2026 •

Copy link
Copy Markdown
Contributor

The Rust adapter previously rejected entity labels outside Python's fixed inventory and blocked every German locale. It now consumes Core capability contract 1 to discover text applicability, locale activation and configuration flags. Future entity labels pass through without a duplicate Python inventory. Multiple locales use singular Core scans with deterministic deduplication before existing overlap handling.

Python remains the default. Legacy result classes, aliases, allowlists, filtering order and redaction strategies are preserved. UUID is activated through explicit entity selection; structured-only PERSON selection fails clearly. Validated capability metadata is cached by native reader identity, with independent scan configurations.

Published dependency verified: Core 0.4.0 is now available on PyPI. A fresh normal registry installation passed all 352 focused adapter/legacy contract checks, installed-wheel smoke, dependency checks and the same reviewed parity results. The dependency range remains datafog-core>=0.4.0,<0.5. Hosted CI is running on the final documentation commit. This PR remains unmerged; no DataFog Python release is authorized.

Reviewed limitation: Core returns both PHONE and NPI for the same numeric span. Legacy overlap-before-filter priority keeps PHONE, so explicit NPI selection can return empty. Default legacy redaction still protects the number. Native datafog.v5.scan retains NPI and native transformation supports selecting it. This behavior is explicitly tested; no hardcoded new-label priorities or selection-order changes were introduced.

Validation:

  • Final candidate SHA256: 351ab81ae575b31a6739a43e29135884ccc0d7152f01df568cd1f7608d690b1c, source 60c4636, macOS ARM64 abi3 wheel.
  • Candidate and published-registry adapter/legacy contract suites: 352 passed each.
  • Published macOS ARM64 wheel SHA256: b4217c2a9834cb774d89a13b9543166dd7f38512a233b1c80984ea9c07c5162c, verified via pip installation report and PyPI metadata (Core release source 133bceff0d2a53a7f6d1a75693330c1797285654).
  • Base/CLI regression without native dependencies: 833 passed, 19 skipped, 295 deselected, 19 existing xfails.
  • Frozen native comparison: 77 matches, 2 reviewed detector differences, 1 locale validation-message difference, 31 outside-scope cases. Original 4.8.1 and historical Core 0.3.1 fixtures remain unchanged.
  • Clean installed Python wheel + exact candidate Core wheel: isolated smoke and pip check passed.
  • All pre-commit checks passed; Sphinx build passed with existing static/autodoc warnings.
  • End-to-end benchmarks recorded. Local warm Rust scan medians: 3.23µs short, 9.40µs mixed, 7.25ms for 1MB sparse input; no universal performance claim.

docs/core-0.4-integration.md records the artifact evidence and remaining release gates. Published Core verification is complete. Cross-platform hosted CI results on the current head remain the final readiness gate.

@sidmohan0 sidmohan0 changed the title feat: prepare Core 0.4 capability adapter feat: discover Rust entity capabilities in the Python adapter Sep 28, 2026
@sidmohan0
sidmohan0 marked this pull request as ready for review September 28, 2026 23:30
@sidmohan0
sidmohan0 merged commit 9688401 into dev Sep 28, 2026
65 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant