Optimal Empirical Risk Minimization under Temporal Distribution Shifts

TL;DR

This paper introduces RIDER, a theoretically grounded method for optimal empirical risk minimization under temporal distribution shifts, improving predictive performance in dynamic environments.

Contribution

RIDER provides a novel, theoretically justified framework for weighting data in the presence of temporal distribution shifts, unifying common strategies and demonstrating superior empirical results.

Findings

RIDER improves out-of-sample performance on the Yearbook dataset.

RIDER outperforms standard weighting strategies in stock volatility prediction.

RIDER enhances ride duration forecasting accuracy in NYC taxi data.

Abstract

Temporal distribution shifts pose a key challenge for machine learning models trained and deployed in dynamically evolving environments. This paper introduces RIDER (RIsk minimization under Dynamically Evolving Regimes) which derives optimally-weighted empirical risk minimization procedures under temporal distribution shifts. Our approach is theoretically grounded in the random distribution shift model, where random shifts arise as a superposition of numerous unpredictable changes in the data-generating process. We show that common weighting schemes, such as pooling all data, exponentially weighting data, and using only the most recent data, emerge naturally as special cases in our framework. We demonstrate that RIDER consistently improves out-of-sample predictive performance when applied as a fine-tuning step on the Yearbook dataset, across a range of benchmark methods in Wild-Time. Moreover, we show that RIDER outperforms standard weighting strategies in two other real-world tasks: predicting stock market volatility and forecasting ride durations in NYC taxi data.