Skip to content

feat: Add reload infrastructure for file-based flag overrides - #220

Draft
kinyoklion wants to merge 7 commits into
feat/overridesfrom
rlamb/overrides-java-filedata-reloader
Draft

kinyoklion wants to merge 7 commits into
feat/overridesfrom
rlamb/overrides-java-filedata-reloader

Conversation

@kinyoklion

@kinyoklion kinyoklion commented Sep 25, 2026 •

Copy link
Copy Markdown
Member

Summary

The flag overrides feature defined by the OVERRIDE specification needs a file source that reloads reliably: coalesced change notifications, retry after a failed read, retention of the last good data, sequential reloads, and support for files that do not exist yet. This change adds that infrastructure to the integrations package for the file-based override source that follows. It is purely additive: the existing file data source keeps its current behavior.

  • FileDataReloader serializes reloads, debounces change signals with a 100 ms settle window, keeps the last good data on failure by not applying, retries a failed load after one second, reports an identical repeated failure once, and skips a reload whose raw content did not change. Close never waits for an in-flight read. An exception from the apply callback is reported like any other load failure and retried, and nothing from that load is remembered.
  • FileDataPoller detects changes by examining each file's modification time and size on a fixed interval, including a file that appears or disappears. A change callback that throws is logged and polling continues.
  • FileDataWatcher watches parent directories through the file system watch service and signals once after start so a change made between an initial load and the start of watching is not missed. A directory that is missing at start or deleted later is registered again on a one-second schedule once it exists, and one change is signaled so the reload picks up what was written meanwhile.
  • OverrideFileLoader reads a list of files with the override source's rules: a file that does not exist contributes no entries, entries keep the versions that the documents specify, a value-only entry becomes a flag that is on and serves the value by fallthrough, a document that parses but holds a flag the model rejects is reported as a file data error, and the result carries a per-file summary and a content digest. It shares only the parsing code (FlagFileParser) with the file data source, plus two new FlagFactory methods that keep document versions.

The existing file data source keeps its current behavior: FileDataSourceImpl, FileSynchronizer, FileInitializer, FileDataSourceBase, and FileDataSourceParsing.FileDataException are unchanged from feat/overrides (the only edit to a legacy file is the two additive FlagFactory methods). It still requires every configured file to exist, assigns a load version to every entry, expands a value-only entry to an on flag with a single fallthrough variation, reloads on every watch event with no debounce, no retry timer, and no skip, and reports every failure. Two new test classes pin that behavior: FileSynchronizerReloadBehaviorTest (identical rewrite emits another change set, every failed reload is reported and logged as the plain description at error level, no retry without a file change, missing file logged with the path) and FileDataLoadingBehaviorTest (a model-rejected document escapes as a SerializationException, and FileDataException.getDescription requires a cause). The baseline test files pass unmodified.

SDK-3246


Note

Overview
Adds reload infrastructure for an upcoming file-based flag override source, without changing the existing file data source’s runtime behavior.

FileDataReloader owns serialized reloads: debounced triggers (default 100ms), 1s retry after failures, last-good retention on load/apply failure, SHA-256-based skip when raw content is unchanged (with recovery apply after errors), and non-blocking shutdown.

FileDataWatcher and FileDataPoller feed change signals (watch service + optional mtime/size polling), including missing directories/files and a post-start catch-up signal.

OverrideFileLoader merges override files with override-specific rules: optional missing files, document-preserved versions, value-only → on/fallthrough flags, stricter parse errors (including model rejections), per-file summaries, and content digest. FlagFactory gains version-preserving flagFromJson / segmentFromJson overloads.

New tests cover the infrastructure and pin legacy file data source loader/synchronizer error and reload semantics (FileSynchronizerReloadBehaviorTest, FileDataLoadingBehaviorTest).

Reviewed by Cursor Bugbot for commit 8edc7db. Bugbot is set up for automated code reviews on this repo. Configure here.

The flag overrides feature needs a file source that reloads reliably:
coalesced change notifications, retry after a failed read, retention of
the last good data, sequential reloads, and support for files that do
not exist yet. This adds that infrastructure to the integrations package
as a purely additive change. The existing file data source keeps its
current behavior.

- FileDataReloader serializes reloads, debounces change signals, keeps
  the last good data on failure by not applying, retries a failed load
  after a bounded delay, reports an identical repeated failure once, and
  skips a reload whose raw content did not change. Close never waits for
  an in-flight read.
- FileDataPoller detects changes by examining each file's modification
  time and size on a fixed interval, including a file that appears or
  disappears.
- FileDataWatcher watches parent directories and signals once after
  start so a change between an initial load and the start of watching is
  not missed.
- OverrideFileLoader reads a list of files with the override source's
  rules: a missing file contributes no entries, entries keep the
  versions that the documents specify, a value-only entry becomes a flag
  that is off and serves the value, a document that the model rejects is
  a file data error, and the result carries a per-file summary and a
  content digest. Two additive FlagFactory methods keep document
  versions.

FileDataSourceImpl, FileSynchronizer, FileInitializer,
FileDataSourceBase, and FileDataException are unchanged. Two new test
classes pin the file data source's reload and error behavior: a reload
on every watch event with no debounce, no retry, and no skip, every
failure reported and logged as the plain description at error level, a
model-rejected document escaping as a SerializationException, and a
description that requires a cause.
@kinyoklion
kinyoklion force-pushed the rlamb/overrides-java-filedata-reloader branch from f0038ed to 79250a7 Compare September 28, 2026 21:03
@kinyoklion kinyoklion changed the title feat: Add debounce, retry, and last-good retention to file data loading feat: Add reload infrastructure for file-based flag overrides Sep 28, 2026
A directory that does not exist at start, or whose watch key becomes invalid because it was deleted, is remembered as missing and registered again on a fixed schedule from the worker thread. A recovered directory signals one change so that files written while it was not watched are picked up. The worker ends on close.
A RuntimeException from the callback is logged at error level instead of ending the repeating task.
The last good hash and the failure state are recorded only after the handler accepts the result. An exception from apply goes through the same failure path as a read or parse failure, so it is reported once and retried after the retry delay.
The time of the next registration attempt and the wait before it come from System.nanoTime, so a step of the system clock neither delays nor advances the retry.
@kinyoklion

Copy link
Copy Markdown
Member Author

bugbot review

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit 8edc7db. Configure here.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant