RI-Loss: A Learnable Residual-Informed Loss for Time Series Forecasting

TL;DR

This paper introduces RI-Loss, a novel noise-aware loss function for time series forecasting that leverages HSIC to improve robustness and predictive accuracy over traditional MSE-based methods.

Contribution

The paper proposes RI-Loss, a new HSIC-based objective function that explicitly models noise structure, with theoretical guarantees and empirical validation across multiple benchmarks.

Findings

Improves forecasting accuracy on eight real-world datasets.

Provides theoretical bounds with explicit sample complexity.

Enhances robustness of models to noise in data.

Abstract

Time series forecasting relies on predicting future values from historical data, yet most state-of-the-art approaches-including transformer and multilayer perceptron-based models-optimize using Mean Squared Error (MSE), which has two fundamental weaknesses: its point-wise error computation fails to capture temporal relationships, and it does not account for inherent noise in the data. To overcome these limitations, we introduce the Residual-Informed Loss (RI-Loss), a novel objective function based on the Hilbert-Schmidt Independence Criterion (HSIC). RI-Loss explicitly models noise structure by enforcing dependence between the residual sequence and a random time series, enabling more robust, noise-aware representations. Theoretically, we derive the first non-asymptotic HSIC bound with explicit double-sample complexity terms, achieving optimal convergence rates through Bernstein-type concentration inequalities and Rademacher complexity analysis. This provides rigorous guarantees for RI-Loss optimization while precisely quantifying kernel space interactions. Empirically, experiments across eight real-world benchmarks and five leading forecasting models demonstrate improvements in predictive performance, validating the effectiveness of our approach. The code is publicly available at: https://github.com/shang-xl/RI-Loss.