LLMs for Explainable AI: A Comprehensive Survey

TL;DR

This survey explores how Large Language Models can enhance Explainable AI by generating human-readable explanations, addressing transparency issues in complex AI systems, and discussing current approaches, challenges, and future directions.

Contribution

It provides a comprehensive overview of LLM-based XAI methods, evaluation techniques, challenges, and real-world applications, highlighting future research needs.

Findings

LLMs improve interpretability of AI models.

Evaluation techniques for LLM explanations are evolving.

Challenges include ensuring explanation quality and user trust.

Abstract

Large Language Models (LLMs) offer a promising approach to enhancing Explainable AI (XAI) by transforming complex machine learning outputs into easy-to-understand narratives, making model predictions more accessible to users, and helping bridge the gap between sophisticated model behavior and human interpretability. AI models, such as state-of-the-art neural networks and deep learning models, are often seen as "black boxes" due to a lack of transparency. As users cannot fully understand how the models reach conclusions, users have difficulty trusting decisions from AI models, which leads to less effective decision-making processes, reduced accountabilities, and unclear potential biases. A challenge arises in developing explainable AI (XAI) models to gain users' trust and provide insights into how models generate their outputs. With the development of Large Language Models, we want to explore the possibilities of using human language-based models, LLMs, for model explainabilities. This survey provides a comprehensive overview of existing approaches regarding LLMs for XAI, and evaluation techniques for LLM-generated explanation, discusses the corresponding challenges and limitations, and examines real-world applications. Finally, we discuss future directions by emphasizing the need for more interpretable, automated, user-centric, and multidisciplinary approaches for XAI via LLMs.