Can Structural Cues Save LLMs? Evaluating Language Models in Massive Document Streams

TL;DR

This paper introduces StreamBench, a new benchmark for evaluating language models in streaming environments with multiple concurrent events, showing that structural cues improve model performance across tasks.

Contribution

The paper presents StreamBench, a comprehensive benchmark for streaming document analysis and demonstrates the effectiveness of structural cues in enhancing model performance.

Findings

Structural cues improve clustering accuracy by up to 4.37%.

Structural cues enhance temporal question answering by up to 9.63%.

Temporal reasoning remains a challenge for current LLMs.

Abstract

Evaluating language models in streaming environments is critical, yet underexplored. Existing benchmarks either focus on single complex events or provide curated inputs for each query, and do not evaluate models under the conflicts that arise when multiple concurrent events are mixed within the same document stream. We introduce StreamBench, a benchmark built from major news stories in 2016 and 2025, comprising 605 events and 15,354 documents across three tasks: Topic Clustering, Temporal Question Answering, and Summarization. To diagnose how models fail, we compare performance with and without structural cues, which organize key facts by event. We find that structural cues improve performance on clustering (up to +4.37%) and temporal QA (up to +9.63%), helping models locate relevant information and separate distinct events. While temporal reasoning remains an open challenge inherent to current LLMs, consistent gains across tasks show that structural cues are a promising direction for future work in massive document streams.