RAGE Against the Machine: Retrieval-Augmented LLM Explanations

TL;DR

This paper introduces RAGE, an interactive tool that explains retrieval-augmented LLMs by identifying influential input parts through counterfactual analysis, enhancing understanding of model responses.

Contribution

The paper presents RAGE, a novel interactive explanation system for retrieval-augmented LLMs, incorporating pruning techniques to explore counterfactual explanations and provenance of answers.

Findings

RAGE effectively identifies key input components affecting LLM outputs.

The tool enables users to trace answer provenance and understand model reasoning.

Counterfactual explanations improve interpretability of retrieval-augmented LLMs.

Abstract

This paper demonstrates RAGE, an interactive tool for explaining Large Language Models (LLMs) augmented with retrieval capabilities; i.e., able to query external sources and pull relevant information into their input context. Our explanations are counterfactual in the sense that they identify parts of the input context that, when removed, change the answer to the question posed to the LLM. RAGE includes pruning methods to navigate the vast space of possible explanations, allowing users to view the provenance of the produced answers.