From Static Inference to Dynamic Interaction: A Survey of Streaming Large Language Models

TL;DR

This survey reviews the emerging streaming large language models, clarifies their definitions, proposes a taxonomy, discusses methodologies, applications, and future research directions.

Contribution

It provides a unified definition, systematic taxonomy, and comprehensive analysis of streaming LLMs, addressing existing fragmentation in the field.

Findings

Established a unified definition based on data flow and interaction.

Proposed a systematic taxonomy of streaming LLMs.

Discussed applications and future research directions.

Abstract

Standard Large Language Models (LLMs) are predominantly designed for static inference with pre-defined inputs, which limits their applicability in dynamic, real-time scenarios. To address this gap, the streaming LLM paradigm has emerged. However, existing definitions of streaming LLMs remain fragmented, conflating streaming generation, streaming inputs, and interactive streaming architectures, while a systematic taxonomy is still lacking. This paper provides a comprehensive overview and analysis of streaming LLMs. First, we establish a unified definition of streaming LLMs based on data flow and dynamic interaction to clarify existing ambiguities. Building on this definition, we propose a systematic taxonomy of current streaming LLMs and conduct an in-depth discussion on their underlying methodologies. Furthermore, we explore the applications of streaming LLMs in real-world scenarios and outline promising research directions to support ongoing advances in streaming intelligence. We maintain a continuously updated repository of relevant papers at https://github.com/EIT-NLP/Awesome-Streaming-LLMs.