Iowa State Proves Bagging Quietly Fails on Correlated Data by sqrt(T_mix)
A new theoretical analysis pins down exactly how much bagging breaks on correlated data, and offers a graph-based fix that closes the gap.
PRO- New paper proves standard bagging is provably suboptimal on Markov-dependent data by a
sqrt(T_mix)factor. - Information-theoretic floor for excess classification risk is
Omega(sqrt(T_mix / n))on ergodic chains. - Adaptive spectral routing partitions data via the Fiedler eigenvector and matches the minimax rate.
- Cuts pairwise base-learner covariance by roughly two orders of magnitude on synthetic AR(1) chains.
- Wrapping ROCKET with spectral routing takes top rank on the 128-dataset UCR archive.
- On Atari ensemble DQN, reduces Bellman target variance by 35% and outperforms PER and REM.
Every practitioner who has trained an ensemble on time-series, RL replay buffers, or spatial sensor grids has quietly relied on a lie: that bootstrap sampling produces roughly independent base learners. A new theoretical paper from Iowa State makes the cost of that lie precise, proves that standard bagging cannot recover on its own, and proposes a graph-partitioning algorithm that provably closes the gap.
The paper, Minimax Optimality and Spectral Routing for Majority-Vote Ensembles under Markov Dependence, works through a three-part argument: an information-theoretic floor on how well any ensemble can do under Markov data, a matching lower bound showing standard bagging misses that floor by a square-root factor, and an adaptive algorithm called spectral routing that hits the floor without needing to know the mixing time.
Why bagging quietly breaks on correlated data
The classical variance-reduction story for ensembles assumes i.i.d. samples. With m base learners trained on independent subsamples, the excess classification risk of the majority vote drops as O(1/sqrt(m)). Modern training data is rarely so cooperative. Time-series forecasting relies on sequential data, RL ensembles draw from temporally correlated replay buffers, and spatial sensors produce locally correlated measurements. Under Markov dependence, bootstrap samples share overlapping information that uniform resampling does not account for, which reduces effective diversity among base learners and degrades ensemble performance in ways the i.i.d. analyses miss.
The key quantity is the mixing time T_mix, the number of steps a Markov chain needs to forget its starting state. On a replay buffer with T_mix around 50, consecutive transitions share so much information that a naive bootstrap re-samples the same story dozens of times over.
A tight sandwich of bounds
The paper's central contribution is a tight sandwich of upper and lower bounds. The results, all stated in terms of sample size n and mixing time T_mix, form a clean picture:
- The floor:
This story is for Pro members
You've reached the end of the free preview. Upgrade to AlphaSignal Pro to read the full article - and everything else behind the paywall.
Comments
