What If TSF: A Benchmark for Reframing Forecasting as Scenario-Guided Multimodal Forecasting

TL;DR

This paper introduces What If TSF, a benchmark for evaluating multimodal time series forecasting models that incorporate textual scenarios, aiming to reflect human-like scenario-based decision making.

Contribution

The paper presents a novel benchmark, WIT, designed to assess models' ability to condition forecasts on textual scenarios, advancing multimodal forecasting research.

Findings

Benchmark enables testing of scenario-guided forecasting.

Models can leverage textual scenarios to produce distinct forecasts.

Provides a new platform for multimodal forecasting evaluation.

Abstract

Time series forecasting is critical to real-world decision making, yet most existing approaches remain unimodal and rely on extrapolating historical patterns. While recent progress in large language models (LLMs) highlights the potential for multimodal forecasting, existing benchmarks largely provide retrospective or misaligned raw context, making it unclear whether such models meaningfully leverage textual inputs. In practice, human experts incorporate what-if scenarios with historical evidence, often producing distinct forecasts from the same observations under different scenarios. Inspired by this, we introduce What If TSF (WIT), a multimodal forecasting benchmark designed to evaluate whether models can condition their forecasts on contextual text, especially future scenarios. By providing expert-crafted plausible or counterfactual scenarios, WIT offers a rigorous testbed for scenario-guided multimodal forecasting. The benchmark is available at https://github.com/jinkwan1115/WhatIfTSF.