A Taxonomy for Design and Evaluation of Prompt-Based Natural Language Explanations

TL;DR

This paper develops a comprehensive taxonomy for prompt-based Natural Language Explanations (NLEs) in AI, aiding stakeholders in designing, evaluating, and governing transparent AI systems effectively.

Contribution

It introduces an updated XAI taxonomy tailored for prompt-based NLEs, covering context, generation, presentation, and evaluation dimensions.

Findings

Provides a structured framework for NLE characterization.

Facilitates improved design and assessment of NLEs.

Supports AI governance and transparency efforts.

Abstract

Effective AI governance requires structured approaches for stakeholders to access and verify AI system behavior. With the rise of large language models, Natural Language Explanations (NLEs) are now key to articulating model behavior, which necessitates a focused examination of their characteristics and governance implications. We draw on Explainable AI (XAI) literature to create an updated XAI taxonomy, adapted to prompt-based NLEs, across three dimensions: (1) Context, including task, data, audience, and goals; (2) Generation and Presentation, covering generation methods, inputs, interactivity, outputs, and forms; and (3) Evaluation, focusing on content, presentation, and user-centered properties, as well as the setting of the evaluation. This taxonomy provides a framework for researchers, auditors, and policymakers to characterize, design, and enhance NLEs for transparent AI systems.