ST-MTM: Masked Time Series Modeling with Seasonal-Trend Decomposition for Time Series Forecasting

TL;DR

ST-MTM introduces a seasonal-trend decomposition-based masked modeling framework with novel masking strategies and contrastive learning, significantly improving time series forecasting accuracy by capturing complex temporal semantics.

Contribution

The paper proposes ST-MTM, a novel masked time-series modeling approach that incorporates seasonal-trend decomposition and specialized masking strategies to better learn temporal dependencies.

Findings

Achieves superior forecasting accuracy over existing methods.

Effectively captures complex temporal variations.

Enhances contextual consistency with contrastive learning.

Abstract

Forecasting complex time series is an important yet challenging problem that involves various industrial applications. Recently, masked time-series modeling has been proposed to effectively model temporal dependencies for forecasting by reconstructing masked segments from unmasked ones. However, since the semantic information in time series is involved in intricate temporal variations generated by multiple time series components, simply masking a raw time series ignores the inherent semantic structure, which may cause MTM to learn spurious temporal patterns present in the raw data. To capture distinct temporal semantics, we show that masked modeling techniques should address entangled patterns through a decomposition approach. Specifically, we propose ST-MTM, a masked time-series modeling framework with seasonal-trend decomposition, which includes a novel masking method for the seasonal-trend components that incorporates different temporal variations from each component. ST-MTM uses a period masking strategy for seasonal components to produce multiple masked seasonal series based on inherent multi-periodicity and a sub-series masking strategy for trend components to mask temporal regions that share similar variations. The proposed masking method presents an effective pre-training task for learning intricate temporal variations and dependencies. Additionally, ST-MTM introduces a contrastive learning task to support masked modeling by enhancing contextual consistency among multiple masked seasonal representations. Experimental results show that our proposed ST-MTM achieves consistently superior forecasting performance compared to existing masked modeling, contrastive learning, and supervised forecasting methods.