Hedging Memory Horizons for Non-Stationary Prediction via Online Aggregation

TL;DR

This paper introduces MELO, a model-agnostic online aggregation method that adaptively hedges memory horizons to improve non-stationary prediction accuracy under distribution shifts.

Contribution

It proposes a novel online aggregation technique combining multiple adaptation scales with theoretical guarantees and practical effectiveness demonstrated on electricity load forecasting.

Findings

MELO reduces RMSE by 34.7% compared to base predictors.

It outperforms models using external COVID policy covariates.

Requires only lightweight recursive updates without retraining.

Abstract

We study online prediction under distribution shift, where inputs arrive chronologically and outcomes are revealed only after prediction. In this setting, predictors must remain stable in quiet regimes yet adapt when regimes shift, and the right adaptation memory is unknown in advance. We propose MELO (Memory-hedged Exponentially Weighted Least-Squares Online aggregation), a model-agnostic method that hedges across adaptation scales: it wraps any non-anticipating base-predictor pool with exponentially weighted least-squares (EWLS) adaptation experts at multiple forgetting factors, and aggregates raw and EWLS-adapted forecasts with MLpol, a parameter-free online aggregation rule. Under boundedness conditions, we establish deterministic oracle inequalities showing that it competes with both the best raw predictor and the best bounded, time-varying affine combinations of the base predictions, up to a path-length-dependent tracking cost and a sublinear aggregation overhead. We evaluate MELO on French national electricity-load forecasting through the COVID-19 lockdown using no regime indicators, lockdown dates, or policy covariates. MELO reduces overall RMSE by 34.7\% relative to base-only MLpol and achieves lower overall RMSE than a TabICL reference supplied with an external COVID policy-response covariate. Moreover, MELO requires only lightweight per-step recursive updates without model retraining.