CLaRa: Bridging Retrieval and Generation with Continuous Latent Reasoning

TL;DR

CLaRa introduces a unified framework that combines retrieval and generation in a shared continuous space, improving knowledge integration and compression efficiency for large language models in question answering tasks.

Contribution

The paper presents CLaRa, a novel method that jointly optimizes retrieval and generation with continuous embeddings, enabling better compression and relevance alignment.

Findings

Achieves state-of-the-art compression and reranking performance.

Outperforms text-based fine-tuned baselines at 16x compression.

Unified optimization improves answer quality and retrieval relevance.

Abstract

Retrieval-augmented generation (RAG) enhances large language models (LLMs) with external knowledge but still suffers from long contexts and disjoint retrieval-generation optimization. In this work, we propose CLaRa (Continuous Latent Reasoning), a unified framework that performs embedding-based compression and joint optimization in a shared continuous space. To obtain semantically rich and retrievable compressed vectors, thereby reducing the document length fed into the generator, we introduce SCP, a key-preserving data synthesis framework based on question answering and paraphrase supervision. CLaRa then trains the reranker and generator end-to-end via a single language modeling loss, with gradients flowing through both modules using a differentiable top-k estimator. Theoretically, this unified optimization aligns retrieval relevance with answer quality. Experiments across multiple QA benchmarks show that CLaRa achieves state-of-the-art compression and reranking performance, even at a text compression rate of 16, outperforming text-based fine-tuned baselines.