Thinking Ahead: Foresight Intelligence in MLLMs and World Models

TL;DR

This paper introduces FSU-QA, a novel dataset for evaluating foresight intelligence in vision-language models, revealing current models' limitations and demonstrating how fine-tuning on this dataset enhances foresight reasoning.

Contribution

The paper presents FSU-QA, the first dataset specifically designed to evaluate and improve foresight intelligence in vision-language models, and provides comprehensive analysis and benchmarks.

Findings

Current models struggle with future reasoning tasks.

Fine-tuning on FSU-QA improves foresight capabilities significantly.

FSU-QA enables assessment of world models through semantic coherence.

Abstract

In this work, we define Foresight Intelligence as the capability to anticipate and interpret future events-an ability essential for applications such as autonomous driving, yet largely overlooked by existing research. To bridge this gap, we introduce FSU-QA, a new Visual Question-Answering (VQA) dataset specifically designed to elicit and evaluate Foresight Intelligence. Using FSU-QA, we conduct the first comprehensive study of state-of-the-art Vision-Language Models (VLMs) under foresight-oriented tasks, revealing that current models still struggle to reason about future situations. Beyond serving as a benchmark, FSU-QA also enables the assessment of world models by measuring the semantic coherence of their generated predictions, quantified through performance gains when VLMs are augmented with such outputs. Our experiments further demonstrate that FSU-QA can effectively enhance foresight reasoning: even small VLMs fine-tuned on FSU-QA surpass much larger, advanced models by a substantial margin. Together, these findings position FSU-QA as a principled foundation for developing next-generation models capable of truly anticipating and understanding future events.