A Two-Stage LLM Framework for Accessible and Verified XAI Explanations

TL;DR

This paper introduces a two-stage LLM framework that enhances the trustworthiness and accessibility of XAI explanations by verifying and refining natural-language narratives generated from raw XAI outputs.

Contribution

It proposes a novel meta-verification framework with an Explainer and Verifier LLMs, improving explanation quality and reliability in XAI systems.

Findings

Verification filters unreliable explanations effectively.

Refinement improves linguistic accessibility of explanations.

Feedback guides the Explainer toward more coherent reasoning.

Abstract

Large Language Models (LLMs) are increasingly used to translate the technical outputs of eXplainable Artificial Intelligence (XAI) methods into accessible natural-language explanations. However, existing approaches often lack guarantees of accuracy, faithfulness, and completeness. At the same time, current efforts to evaluate such narratives remain largely subjective or confined to post-hoc scoring, offering no safeguards to prevent flawed explanations from reaching end-users. To address these limitations, this paper proposes a Two-Stage LLM Meta-Verification Framework that consists of (i) an Explainer LLM that converts raw XAI outputs into natural-language narratives, (ii) a Verifier LLM that assesses them in terms of faithfulness, coherence, completeness, and hallucination risk, and (iii) an iterative refeed mechanism that uses the Verifier's feedback to refine and improve them. Experiments across five XAI techniques and datasets, using three families of open-weight LLMs, show that verification is crucial for filtering unreliable explanations while improving linguistic accessibility compared with raw XAI outputs. In addition, the analysis of the Entropy Production Rate (EPR) during the refinement process indicates that the Verifier's feedback progressively guides the Explainer toward more stable and coherent reasoning. Overall, the proposed framework provides an efficient pathway toward more trustworthy and democratized XAI systems.