Read As Human: Compressing Context via Parallelizable Close Reading and Skimming

TL;DR

This paper introduces RAM, a context compression framework inspired by human reading, which selectively fully reads important segments and skims less relevant ones to improve efficiency and performance in long-input tasks.

Contribution

RAM employs an adaptive hybrid reading strategy with parallel encoding and contrastive learning to enhance long-context processing in language models.

Findings

RAM outperforms baselines on question answering and summarization tasks.

Achieves up to 12x speedup on long inputs.

Maintains high performance with compressed context.

Abstract

Large Language Models (LLMs) demonstrate exceptional capability across diverse tasks. However, their deployment in long-context scenarios is hindered by two challenges: computational inefficiency and redundant information. We propose RAM (Read As HuMan), a context compression framework that adopts an adaptive hybrid reading strategy, to address these challenges. Inspired by human reading behavior (i.e., close reading important content while skimming less relevant content), RAM partitions the context into segments and encodes them with the input query in parallel. High-relevance segments are fully retained (close reading), while low-relevance ones are query-guided compressed into compact summary vectors (skimming). Both explicit textual segments and implicit summary vectors are concatenated and fed into decoder to achieve both superior performance and natural language format interpretability. To refine the decision boundary between close reading and skimming, we further introduce a contrastive learning objective based on positive and negative query-segment pairs. Experiments demonstrate that RAM outperforms existing baselines on multiple question answering and summarization benchmarks across two backbones, while delivering up to a 12x end-to-end speedup on long inputs (average length 16K; maximum length 32K).