Explaining Large Language Models with gSMILE

TL;DR

gSMILE is a perturbation-based, model-agnostic framework that provides interpretable token-level explanations for large language models, enhancing transparency and trustworthiness in AI outputs.

Contribution

The paper introduces gSMILE, a novel interpretability method that extends SMILE using controlled perturbations and Wasserstein metrics for reliable token attribution in LLMs.

Findings

gSMILE produces human-aligned, reliable attributions.

Claude 2.1 shows high attention fidelity.

GPT-3.5 exhibits high output consistency.

Abstract

Large Language Models (LLMs) such as GPT, LLaMA, and Claude achieve remarkable performance in text generation but remain opaque in their decision-making processes, limiting trust and accountability in high-stakes applications. We present gSMILE (generative SMILE), a model-agnostic, perturbation-based framework for token-level interpretability in LLMs. Extending the SMILE methodology, gSMILE uses controlled prompt perturbations, Wasserstein distance metrics, and weighted linear surrogates to identify input tokens with the most significant impact on the output. This process enables the generation of intuitive heatmaps that visually highlight influential tokens and reasoning paths. We evaluate gSMILE across leading LLMs (OpenAI's gpt-3.5-turbo-instruct, Meta's LLaMA 3.1 Instruct Turbo, and Anthropic's Claude 2.1) using attribution fidelity, attribution consistency, attribution stability, attribution faithfulness, and attribution accuracy as metrics. Results show that gSMILE delivers reliable human-aligned attributions, with Claude 2.1 excelling in attention fidelity and GPT-3.5 achieving the highest output consistency. These findings demonstrate gSMILE's ability to balance model performance and interpretability, enabling more transparent and trustworthy AI systems.