Time-Series Forecasting for Out-of-Distribution Generalization Using Invariant Learning

TL;DR

This paper introduces FOIL, a novel invariant learning framework designed to enhance out-of-distribution generalization in time-series forecasting by addressing unobserved variables and environment inference challenges.

Contribution

FOIL provides a model-agnostic approach with a surrogate loss and environment inference method to improve OOD generalization in TSF tasks.

Findings

FOIL significantly improves TSF model performance, with gains up to 85%.

The framework effectively mitigates the impact of unobserved variables.

FOIL enhances OOD generalization in various real-world time-series datasets.

Abstract

Time-series forecasting (TSF) finds broad applications in real-world scenarios. Due to the dynamic nature of time-series data, it is crucial to equip TSF models with out-of-distribution (OOD) generalization abilities, as historical training data and future test data can have different distributions. In this paper, we aim to alleviate the inherent OOD problem in TSF via invariant learning. We identify fundamental challenges of invariant learning for TSF. First, the target variables in TSF may not be sufficiently determined by the input due to unobserved core variables in TSF, breaking the conventional assumption of invariant learning. Second, time-series datasets lack adequate environment labels, while existing environmental inference methods are not suitable for TSF. To address these challenges, we propose FOIL, a model-agnostic framework that enables timeseries Forecasting for Out-of-distribution generalization via Invariant Learning. FOIL employs a novel surrogate loss to mitigate the impact of unobserved variables. Further, FOIL implements a joint optimization by alternately inferring environments effectively with a multi-head network while preserving the temporal adjacency structure, and learning invariant representations across inferred environments for OOD generalized TSF. We demonstrate that the proposed FOIL significantly improves the performance of various TSF models, achieving gains of up to 85%.