Think-as-You-See: Streaming Chain-of-Thought Reasoning for Large Vision-Language Models

TL;DR

This paper introduces Think-as-You-See (TaYS), a streaming reasoning framework for large vision-language models that enables real-time, concurrent video understanding, outperforming traditional batch and interleaved methods in efficiency and accuracy.

Contribution

We propose TaYS, a novel streaming reasoning paradigm for LVLMs that allows true concurrent processing and improves reasoning performance on video tasks.

Findings

TaYS outperforms batch and interleaved baselines in reasoning accuracy.

TaYS reduces time-to-first-token and overall reasoning delay.

Experimental results confirm the effectiveness of data-aligned streaming reasoning.

Abstract

Large Vision Language Models (LVLMs) exhibit strong Chain-of-Thought (CoT) capabilities, yet most existing paradigms assume full-video availability before inference, a batch-style process misaligned with real-world video streams where information arrives sequentially. Motivated by the streaming nature of video data, we investigate two streaming reasoning paradigms for LVLMs. The first, an interleaved paradigm, alternates between receiving frames and producing partial reasoning but remains constrained by strictly ordered cache updates. To better match streaming inputs, we propose \textbf{Think-as-You-See (TaYS)}, a unified framework enabling true concurrent reasoning. TaYS integrates parallelized CoT generation, stream-constrained training, and stream-parallel inference. It further employs temporally aligned reasoning units, streaming attention masks and positional encodings, and a dual KV-cache that decouples visual encoding from textual reasoning. We evaluate all paradigms on the Qwen2.5-VL family across representative video CoT tasks, including event dynamics analysis, causal reasoning, and thematic understanding. Experiments show that TaYS consistently outperforms both batch and interleaved baselines, improving reasoning performance while substantially reducing time-to-first-token (TTFT) and overall reasoning delay. These results demonstrate the effectiveness of data-aligned streaming reasoning in enabling efficient and responsive video understanding for LVLMs. We release our code at https://github.com/EIT-NLP/StreamingLLM/tree/main/TaYS