A Mechanistic Explanatory Strategy for XAI

TL;DR

This paper proposes a mechanistic explanatory strategy for XAI, emphasizing the identification of functional components in deep learning systems to improve interpretability within a philosophical framework.

Contribution

It introduces a mechanistic approach to explain deep neural networks, integrating philosophical insights with practical case studies from image and language models.

Findings

Mechanistic explanations can reveal overlooked elements in AI systems.

Decomposition and localization aid in understanding neural network functions.

Case studies demonstrate the approach's alignment with interpretability research.

Abstract

Despite significant advancements in XAI, scholars note a persistent lack of solid conceptual foundations and integration with broader scientific discourse on explanation. In response, emerging research draws on explanatory strategies from various sciences and the philosophy of science literature to fill these gaps. This paper outlines a mechanistic strategy for explaining the functional organization of deep learning systems, situating recent developments in explainable AI within a broader philosophical context. According to the mechanistic approach, the explanation of opaque AI systems involves identifying mechanisms that drive decision making. For deep neural networks, this means discerning functionally relevant components, such as neurons, layers, circuits, or activation patterns, and understanding their roles through decomposition, localization, and recomposition. Proof-of-principle case studies from image recognition and language modeling align these theoretical approaches with mechanistic interpretability research from OpenAI and Anthropic. The findings suggest that pursuing mechanistic explanations can uncover elements that traditional explainability techniques may overlook, ultimately contributing to more thoroughly explainable AI