Context-Aware Probabilistic Modeling with LLM for Multimodal Time Series Forecasting

TL;DR

CAPTime introduces a novel approach that combines large language models with time series encoders to improve probabilistic multimodal forecasting accuracy and robustness, especially in data-scarce scenarios.

Contribution

It presents a new method integrating textual context with LLMs for probabilistic time series forecasting, addressing limitations of previous shallow or deterministic approaches.

Findings

Outperforms existing methods in accuracy and generalization.

Demonstrates robustness in data-scarce scenarios.

Effectively models multimodal time series with textual context.

Abstract

Time series forecasting is important for applications spanning energy markets, climate analysis, and traffic management. However, existing methods struggle to effectively integrate exogenous texts and align them with the probabilistic nature of large language models (LLMs). Current approaches either employ shallow text-time series fusion via basic prompts or rely on deterministic numerical decoding that conflict with LLMs' token-generation paradigm, which limits contextual awareness and distribution modeling. To address these limitations, we propose CAPTime, a context-aware probabilistic multimodal time series forecasting method that leverages text-informed abstraction and autoregressive LLM decoding. Our method first encodes temporal patterns using a pretrained time series encoder, then aligns them with textual contexts via learnable interactions to produce joint multimodal representations. By combining a mixture of distribution experts with frozen LLMs, we enable context-aware probabilistic forecasting while preserving LLMs' inherent distribution modeling capabilities. Experiments on diverse time series forecasting tasks demonstrate the superior accuracy and generalization of CAPTime, particularly in multimodal scenarios. Additional analysis highlights its robustness in data-scarce scenarios through hybrid probabilistic decoding.