TEFL: Prediction-Residual-Guided Rolling Forecasting for Multi-Horizon Time Series

TL;DR

TEFL introduces a residual-based feedback mechanism into deep multi-horizon time series forecasting, significantly improving accuracy and robustness across diverse datasets and architectures by leveraging past prediction residuals during training and evaluation.

Contribution

The paper proposes TEFL, a novel framework that explicitly incorporates historical residuals into deep forecasting models, addressing partial observability and efficiency challenges with a two-stage training process.

Findings

Reduces MAE by 5-10% on average across datasets.

Enhances robustness under abrupt changes and distribution shifts.

Achieves residual reduction up to 19.5% in challenging scenarios.

Abstract

Time series forecasting plays a critical role in domains such as transportation, energy, and meteorology. Despite their success, modern deep forecasting models are typically trained to minimize point-wise prediction loss without leveraging the rich information contained in past prediction residuals from rolling forecasts - residuals that reflect persistent biases, unmodeled patterns, or evolving dynamics. We propose TEFL (Temporal Error Feedback Learning), a unified learning framework that explicitly incorporates these historical residuals into the forecasting pipeline during both training and evaluation. To make this practical in deep multi-step settings, we address three key challenges: (1) selecting observable multi-step residuals under the partial observability of rolling forecasts, (2) integrating them through a lightweight low-rank adapter to preserve efficiency and prevent overfitting, and (3) designing a two-stage training procedure that jointly optimizes the base forecaster and error module. Extensive experiments across 10 real-world datasets and 5 backbone architectures show that TEFL consistently improves accuracy, reducing MAE by 5-10% on average. Moreover, it demonstrates strong robustness under abrupt changes and distribution shifts, with error reductions exceeding 10% (up to 19.5%) in challenging scenarios. By embedding residual-based feedback directly into the learning process, TEFL offers a simple, general, and effective enhancement to modern deep forecasting systems.