SimDiff: Simpler Yet Better Diffusion Model for Time Series Point Forecasting

TL;DR

SimDiff introduces a unified Transformer-based diffusion framework that achieves superior point forecasting accuracy for time series by leveraging intrinsic diversity, ensembling, and novel normalization techniques, outperforming existing methods.

Contribution

The paper presents SimDiff, a single-stage, end-to-end diffusion model that improves point forecast accuracy without external regressors, using innovative normalization and ensembling strategies.

Findings

Outperforms existing time series point forecasting methods.

Achieves state-of-the-art accuracy through ensembling and intrinsic diversity.

Demonstrates robustness and adaptability across various datasets.

Abstract

Diffusion models have recently shown promise in time series forecasting, particularly for probabilistic predictions. However, they often fail to achieve state-of-the-art point estimation performance compared to regression-based methods. This limitation stems from difficulties in providing sufficient contextual bias to track distribution shifts and in balancing output diversity with the stability and precision required for point forecasts. Existing diffusion-based approaches mainly focus on full-distribution modeling under probabilistic frameworks, often with likelihood maximization objectives, while paying little attention to dedicated strategies for high-accuracy point estimation. Moreover, other existing point prediction diffusion methods frequently rely on pre-trained or jointly trained mature models for contextual bias, sacrificing the generative flexibility of diffusion models. To address these challenges, we propose SimDiff, a single-stage, end-to-end framework. SimDiff employs a single unified Transformer network carefully tailored to serve as both denoiser and predictor, eliminating the need for external pre-trained or jointly trained regressors. It achieves state-of-the-art point estimation performance by leveraging intrinsic output diversity and improving mean squared error accuracy through multiple inference ensembling. Key innovations, including normalization independence and the median-of-means estimator, further enhance adaptability and stability. Extensive experiments demonstrate that SimDiff significantly outperforms existing methods in time series point forecasting.