fix: Use double precision and integer weight sums in rollout bucketing - #231
Open
tanderson-ld wants to merge 1 commit into
Open
tanderson-ld wants to merge 1 commit into
tanderson-ld wants to merge 1 commit into
Conversation
The bucket value hash division is now performed in double precision, and rollout bucket boundaries are computed by summing the integer weights and dividing at each comparison, per the evaluation spec. Accumulating a single-precision sum allowed rounding error to shift bucket boundaries, which could break mutual exclusivity of experiments sharing a layer. Fixes #94 (SDK-1537)
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Requirements
Related issues
Fixes #94 (internal tracking: SDK-1537)
Describe the solution you've provided
The evaluation spec (FLGEA §II.F steps 6d and 7a) requires double-precision floating-point arithmetic for bucketing, and requires rollout bucket boundaries to be computed by summing the integer weights and then dividing by 100000. This SDK was doing both in single precision, and was accumulating a running float sum of per-weight quotients. With long weight lists (e.g. mutually exclusive experiments in a shared layer), the accumulated rounding error shifts bucket boundaries differently per flag, so a context near a boundary can be placed in overlapping experiments — the behavior reported in #94.
Changes:
EvaluatorBucketing.computeBucketValuenow returnsdouble, and the hash division by 2^60-1 is performed in double precision.Evaluator.getValueForVariationOrRolloutsums the weights as alongand divides by100000.0at each comparison instead of accumulating a float sum.Verification:
EvaluatorBucketingPrecisionTestreproduces the exact scenario from Non-exclusive experiment assignments on layer due to bucketization rounding error #94 (real 551-variation experiment data, seed, and context key). Both tests fail against the previous implementation (0.98308944702...float bucket value and wrong bucket assignment) and pass with this change.RolloutRandomizationConsistencyTest's hard-coded cross-SDK values pass unchanged within their existing 1e-7 tolerances (float→double moves results by at most ~3e-8).java-server-sdktest suite passes.Describe alternatives you've considered
Making the bucketing algorithm configurable (a "BucketerV1"/"BucketerV2" selection, so customers could adopt the fix on their own schedule) was discussed on #94. That adds permanent complexity for a correction whose blast radius is small, and the SDK would remain out of spec by default; the direct fix follows the precedent of previous cross-SDK bucketing-consistency fixes.
Additional context
This is a behavioral change for contexts whose bucket value falls between the old (single-precision) and new (double-precision) bucket boundaries:
Contexts not near a bucket boundary (the overwhelming majority) are unaffected, and assignments remain deterministic after the one-time step at upgrade.
Note for release planning: merging with
fix:will cut a patch release via release-please. The release-notes wording for the assignment discontinuity (and whether any additional comms are needed for experiment-heavy customers) is still pending the experimentation-team review, so hold the merge if that should land first.Note
Overview
Aligns Java server SDK rollout bucketing with the evaluation spec (FLGEA) and other SDKs by fixing two precision bugs that could mis-assign contexts near bucket boundaries—especially in large experiment layers.
computeBucketValuenow returnsdoubleand normalizes the hash with double arithmetic instead of single-precisionfloat.Rollout variation selection no longer accumulates a running float sum of
weight/100000; it sums weights as alongand comparesbucket < weightSum / 100000.0per bucket so rounding cannot drift boundaries differently across flags with long weight lists.Adds
EvaluatorBucketingPrecisionTestreproducing #94 (551-variation experiment); existing bucketing tests are updated fordouble. Contexts not near a boundary stay stable; a small fraction near old float boundaries may get a one-time reassignment at upgrade.Reviewed by Cursor Bugbot for commit 140211b. Bugbot is set up for automated code reviews on this repo. Configure here.