ArcAligner: Adaptive Recursive Aligner for Compressed Context Embeddings in RAG

TL;DR

ArcAligner is a lightweight adaptive module integrated into language models that improves the utilization of compressed context representations, enhancing performance on knowledge-intensive tasks while maintaining speed.

Contribution

It introduces ArcAligner, a novel adaptive recursive aligner that enhances compressed context understanding in RAG models, outperforming existing compression methods.

Findings

Outperforms compression baselines on QA benchmarks

Effective on multi-hop and long-tail tasks

Maintains speed with adaptive gating

Abstract

Retrieval-Augmented Generation (RAG) helps LLMs stay accurate, but feeding long documents into a prompt makes the model slow and expensive. This has motivated context compression, ranging from token pruning and summarization to embedding-based compression. While researchers have tried ''compressing'' these documents into smaller summaries or mathematical embeddings, there is a catch: the more you compress the data, the more the LLM struggles to understand it. To address this challenge, we propose ArcAligner (Adaptive recursive context *Aligner*), a lightweight module integrated into the language model layers to help the model better utilize highly compressed context representations for downstream generation. It uses an adaptive ''gating'' system that only adds extra processing power when the information is complex, keeping the system fast. Across knowledge-intensive QA benchmarks, ArcAligner consistently beats compression baselines at comparable compression rates, especially on multi-hop and long-tail settings. The source code is publicly available.