Not All Data are Good Labels: On the Self-supervised Labeling for Time Series Forecasting

TL;DR

This paper introduces SCAM, a self-supervised method for time series forecasting that improves model generalization by re-labeling data and reducing overfitting through adaptive masking and spectral norm regularization.

Contribution

It proposes a novel self-supervised labeling approach with SCAM and SNR to enhance TSF model performance and generalization across diverse datasets.

Findings

SCAM improves accuracy across eleven real-world datasets.

Spectral Norm Regularization reduces overfitting effectively.

Self-supervised pseudo labels enhance model robustness.

Abstract

Time Series Forecasting (TSF) is a crucial task in various domains, yet existing TSF models rely heavily on high-quality data and insufficiently exploit all available data. This paper explores a novel self-supervised approach to re-label time series datasets by inherently constructing candidate datasets. During the optimization of a simple reconstruction network, intermediates are used as pseudo labels in a self-supervised paradigm, improving generalization for any predictor. We introduce the Self-Correction with Adaptive Mask (SCAM), which discards overfitted components and selectively replaces them with pseudo labels generated from reconstructions. Additionally, we incorporate Spectral Norm Regularization (SNR) to further suppress overfitting from a loss landscape perspective. Our experiments on eleven real-world datasets demonstrate that SCAM consistently improves the performance of various backbone models. This work offers a new perspective on constructing datasets and enhancing the generalization of TSF models through self-supervised learning. The code is available at https://github.com/SuDIS-ZJU/SCAM.