How often does a nominal 95% bootstrap confidence interval actually contain the truth, when the data are autocorrelated?
Not 95%.
dependence naive resampling blocks of 20
none (phi=0) 94.2% 90.1%
moderate (phi=0.5) 71.0% 87.3%
strong (phi=0.8) 48.0% 85.0%
very strong (phi=0.9) 32.3% 78.2%
Read the first row first. It is the control. When the data really are independent, naive resampling lands on 94.2%, which is where a 95% interval belongs. That is the reason to believe the rest of the column: the collapse below it is a property of the data, not a bug in the harness. The script refuses to report at all if that control falls outside 0.92-0.97.
python3 coverage_study.py
One command, one file, numpy only. It takes about five minutes on a laptop and writes
COVERAGE-RESULTS.json. The committed copy of that file is the run the table above came
from, so you can diff your output against it.
The data-generating process is a zero-mean AR(1), so the true mean is exactly 0 and coverage is checkable without estimating anything. For each dependence level the script draws 2,000 independent series of 200 points, builds a 95% percentile interval for the mean by each method, and counts how often the interval contains 0. Seed is fixed at 20260824.
Two methods are compared. IID resampling draws observations with replacement, which destroys the dependence structure. The moving block bootstrap draws overlapping blocks of length L and lays them end to end, which preserves dependence within a block.
The failure is one-sided, and it is the dangerous side. The naive interval does not become noisy as dependence rises; it becomes narrow and confident. A narrow confident interval is what makes someone act.
The fix is real and partial. Blocks recover most of the gap and none of them reach nominal at strong dependence. 78.2% is the honest number, not 95%.
Blocks cost something in the top row. On data that was independent all along, blocks of 20 lose about four points of coverage against naive resampling. That is the price of the insurance and it is small.
This is simulation on one model, chosen because its true mean is known. It tells you about the mechanism, not about your data. Block length matters and choosing it well is its own literature. Nothing here rescues a misspecified model: an honest interval around a wrong point estimate is still wrong.
The reason to run the coverage check on your own estimator is that this repository cannot run it for you.
MIT. See LICENSE.