LLM-Integrated Bayesian State Space Models for Multimodal Time-Series Forecasting

TL;DR

This paper introduces LLM-integrated Bayesian State Space models (LBS), a new probabilistic framework combining large language models and state space models for flexible, uncertainty-aware multimodal time-series forecasting that includes textual summaries.

Contribution

The paper presents the first unified framework integrating LLMs with Bayesian state space models for joint numerical and textual multimodal forecasting.

Findings

LBS outperforms previous methods by 13.20% on TextTimeCorpus.

LBS provides human-readable textual summaries of forecasts.

The approach enables flexible lookback and forecast windows with uncertainty quantification.

Abstract

Forecasting in the real world requires integrating structured time-series data with unstructured textual information, but existing methods are architecturally limited by fixed input/output horizons and are unable to model or quantify uncertainty. We address this challenge by introducing LLM-integrated Bayesian State space models (LBS), a novel probabilistic framework for multimodal temporal forecasting. At a high level, LBS consists of two components: (1) a state space model (SSM) backbone that captures the temporal dynamics of latent states from which both numerical and textual observations are generated and (2) a pretrained large language model (LLM) that is adapted to encode textual inputs for posterior state estimation and decode textual forecasts consistent with the latent trajectory. This design enables flexible lookback and forecast windows, principled uncertainty quantification, and improved temporal generalization thanks to the well-suited inductive bias of SSMs toward modeling dynamical systems. Experiments on the TextTimeCorpus benchmark demonstrate that LBS improves the previous state-of-the-art by 13.20% while providing human-readable summaries of each forecast. Our work is the first to unify LLMs and SSMs for joint numerical and textual prediction, offering a novel foundation for multimodal temporal reasoning.