Dynamic Chunking and Selection for Reading Comprehension of Ultra-Long Context in Large Language Models

TL;DR

This paper introduces a dynamic chunking and selection method for large language models to improve comprehension of ultra-long texts by adaptively dividing contexts and selecting question-relevant chunks, outperforming fixed truncation approaches.

Contribution

It presents a novel adaptive chunking and question-aware selection technique that enhances LLM understanding of very long texts, maintaining robustness across extensive input lengths.

Findings

Outperforms strong baselines on question-answering benchmarks.

Maintains robustness with sequences up to 256k tokens.

Effective in both single-hop and multi-hop QA tasks.

Abstract

Large language models (LLMs) often struggle to accurately read and comprehend extremely long texts. Current methods for improvement typically rely on splitting long contexts into fixed-length chunks. However, fixed truncation risks separating semantically relevant content, leading to ambiguity and compromising accurate understanding. To overcome this limitation, we propose a straightforward approach for dynamically separating and selecting chunks of long context, facilitating a more streamlined input for LLMs. In particular, we compute semantic similarities between adjacent sentences, using lower similarities to adaptively divide long contexts into variable-length chunks. We further train a question-aware classifier to select sensitive chunks that are critical for answering specific questions. Experimental results on both single-hop and multi-hop question-answering benchmarks show that the proposed approach consistently outperforms strong baselines. Notably, it maintains robustness across a wide range of input lengths, handling sequences of up to 256k tokens. Our datasets and code are available at the following link: https://github.com/ECNU-Text-Computing/DCS