Selective Learning for Deep Time Series Forecasting

TL;DR

This paper introduces a selective learning strategy for deep time series forecasting that improves model generalization by focusing on reliable, non-anomalous, and less uncertain timesteps, reducing overfitting.

Contribution

The paper proposes a dual-mask selective learning framework that filters uncertain and anomalous timesteps, enhancing deep TSF model performance.

Findings

Significant MSE reduction across multiple models and datasets

Improved forecasting accuracy with selective learning approach

Effective filtering of uncertain and anomalous data points

Abstract

Benefiting from high capacity for capturing complex temporal patterns, deep learning (DL) has significantly advanced time series forecasting (TSF). However, deep models tend to suffer from severe overfitting due to the inherent vulnerability of time series to noise and anomalies. The prevailing DL paradigm uniformly optimizes all timesteps through the MSE loss and learns those uncertain and anomalous timesteps without difference, ultimately resulting in overfitting. To address this, we propose a novel selective learning strategy for deep TSF. Specifically, selective learning screens a subset of the whole timesteps to calculate the MSE loss in optimization, guiding the model to focus on generalizable timesteps while disregarding non-generalizable ones. Our framework introduces a dual-mask mechanism to target timesteps: (1) an uncertainty mask leveraging residual entropy to filter uncertain timesteps, and (2) an anomaly mask employing residual lower bound estimation to exclude anomalous timesteps. Extensive experiments across eight real-world datasets demonstrate that selective learning can significantly improve the predictive performance for typical state-of-the-art deep models, including 37.4% MSE reduction for Informer, 8.4% for TimesNet, and 6.5% for iTransformer.