BalanceRAG: Joint Risk Calibration for Cascaded Retrieval-Augmented Generation

TL;DR

BalanceRAG introduces a joint risk calibration method for cascaded retrieval-augmented generation, optimizing when to use retrieval based on uncertainty to improve factuality and efficiency.

Contribution

It develops a novel framework for joint risk calibration in cascaded RAG systems, enabling adaptive thresholding and multi-risk control based on uncertainty scores.

Findings

BalanceRAG meets prescribed risk levels across benchmarks.

It preserves higher coverage and correct answers.

It reduces unnecessary retrieval calls compared to always-on RAG.

Abstract

Large language models (LLMs) can enhance factuality via retrieval-augmented generation (RAG), but applying RAG to every query is unnecessary when the model-only answer is reliable. This motivates cascaded RAG: each query is first handled by an LLM-only branch, escalated to a RAG fallback only if the primary branch is uncertain, and abstained from when neither branch is sufficiently trustworthy. However, calibrating such cascades stage by stage may be conservative, since the final utility depends on joint uncertainty thresholding of LLM-only and RAG. In this work, we develop BalanceRAG to certify threshold pairs at a target risk level. Given uncertainty scores from the two branches, BalanceRAG frames each threshold pair as an operating point on a two-dimensional lattice and identifies safe operating points using sequential graphical testing. This enables risk-adaptive threshold calibration, controlling the system-level error rate among accepted points, while retaining more examples. Furthermore, BalanceRAG extends to multi-risk calibration, allowing retrieval usage to be bounded together with the selection-conditioned risk. Experiments on three open-domain question answering (QA) benchmarks across multiple LLM backbones demonstrate that BalanceRAG meets prescribed risk levels, preserves higher coverage and more accepted correct examples, and reduces unnecessary retrieval calls compared with always-on RAG.