A Necessary Step toward Faithfulness: Measuring and Improving Consistency in Free-Text Explanations

TL;DR

This paper introduces PEX, a measure for evaluating and improving the consistency of free-text explanations in AI, significantly enhancing explanation faithfulness and transparency in high-stakes decision-making.

Contribution

It presents a novel measure for explanation consistency, demonstrates its application to improve explanation faithfulness, and provides empirical evidence of its effectiveness across multiple models.

Findings

Over 62% of explanations lack consistency

Preference optimization improves explanation consistency by up to 292.3%

Optimizing consistency enhances explanation faithfulness by up to 9.7%

Abstract

Faithful free-text explanations are important to ensure transparency in high-stakes AI decision-making contexts, but they are challenging to generate by language models and assess by humans. In this paper, we present a measure for Prediction-EXplanation (PEX) consistency, by extending the concept of weight of evidence. This measure quantifies how much a free-text explanation supports or opposes a prediction, serving as an important aspect of explanation faithfulness. Our analysis reveals that more than 62% explanations generated by large language models lack this consistency. We show that applying direct preference optimization improves the consistency of generated explanations across three model families, with improvement ranging from 43.1% to 292.3%. Furthermore, we demonstrate that optimizing this consistency measure can improve explanation faithfulness by up to 9.7%.