Investigating Training and Generalization in Faithful Self-Explanations of Large Language Models

TL;DR

This paper investigates methods to improve the faithfulness of self-generated explanations in large language models, demonstrating that training enhances faithfulness across styles and tasks, with evidence of generalization and cross-style transfer.

Contribution

It introduces a training approach using pseudo-faithful explanations to improve and generalize faithfulness in LLM self-explanations across multiple styles and tasks.

Findings

Training improves explanation faithfulness across tasks and styles

Improvements generalize to multi-word explanations and unseen tasks

Cross-style generalization indicates broader faithfulness enhancement

Abstract

Large language models have the potential to generate explanations for their own predictions in a variety of styles based on user instructions. Recent research has examined whether these self-explanations faithfully reflect the models' actual behavior and has found that they often lack faithfulness. However, the question of how to improve faithfulness remains underexplored. Moreover, because different explanation styles have superficially distinct characteristics, it is unclear whether improvements observed in one style also arise when using other styles. This study analyzes the effects of training for faithful self-explanations and the extent to which these effects generalize, using three classification tasks and three explanation styles. We construct one-word constrained explanations that are likely to be faithful using a feature attribution method, and use these pseudo-faithful self-explanations for continual learning on instruction-tuned models. Our experiments demonstrate that training can improve self-explanation faithfulness across all classification tasks and explanation styles, and that these improvements also show signs of generalization to the multi-word settings and to unseen tasks. Furthermore, we find consistent cross-style generalization among three styles, suggesting that training may contribute to a broader improvement in faithful self-explanation ability.