UniDiff: A Unified Diffusion Framework for Multimodal Time Series Forecasting

TL;DR

UniDiff introduces a novel multimodal diffusion framework that effectively integrates textual and temporal data for accurate time series forecasting, achieving state-of-the-art results on diverse real-world datasets.

Contribution

The paper presents UniDiff, a unified diffusion model with a cross-attention fusion module and classifier-free guidance for multimodal time series forecasting, addressing prior single-modality limitations.

Findings

Achieves state-of-the-art performance on benchmark datasets

Effectively integrates textual and temporal information

Demonstrates robustness across multiple domains

Abstract

As multimodal data proliferates across diverse real-world applications, leveraging heterogeneous information such as texts and timestamps for accurate time series forecasting (TSF) has become a critical challenge. While diffusion models demonstrate exceptional performance in generation tasks, their application to TSF remains largely confined to modeling single-modality numerical sequences, overlooking the abundant cross-modal signals inherent in complex heterogeneous data. To address this gap, we propose UniDiff, a unified diffusion framework for multimodal time series forecasting. To process the numerical sequence, our framework first tokenizes the time series into patches, preserving local temporal dynamics by mapping each patch to an embedding space via a lightweight MLP. At its core lies a unified and parallel fusion module, where a single cross-attention mechanism adaptively weighs and integrates structural information from timestamps and semantic context from texts in one step, enabling a flexible and efficient interplay between modalities. Furthermore, we introduce a novel classifier-free guidance mechanism designed for multi-source conditioning, allowing for decoupled control over the guidance strength of textual and temporal information during inference, which significantly enhances model robustness. Extensive experiments on real-world benchmark datasets across eight domains demonstrate that the proposed UniDiff model achieves state-of-the-art performance.