False Sense of Security in Explainable Artificial Intelligence (XAI)

TL;DR

This paper critically examines the limitations of current explainable AI methods and highlights how legislative policies may create a false sense of security without addressing technical challenges, risking ineffective AI governance.

Contribution

It provides a comprehensive analysis of AI regulations in the US and EU, emphasizing the disconnect between legal expectations and technical realities of XAI.

Findings

Current XAI methods often produce misleading explanations.

Legislative policies may foster a false sense of security in AI explainability.

Effective AI governance requires aligning legal standards with technical capabilities.

Abstract

A cautious interpretation of AI regulations and policy in the EU and the USA place explainability as a central deliverable of compliant AI systems. However, from a technical perspective, explainable AI (XAI) remains an elusive and complex target where even state of the art methods often reach erroneous, misleading, and incomplete explanations. "Explainability" has multiple meanings which are often used interchangeably, and there are an even greater number of XAI methods - none of which presents a clear edge. Indeed, there are multiple failure modes for each XAI method, which require application-specific development and continuous evaluation. In this paper, we analyze legislative and policy developments in the United States and the European Union, such as the Executive Order on the Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence, the AI Act, the AI Liability Directive, and the General Data Protection Regulation (GDPR) from a right to explanation perspective. We argue that these AI regulations and current market conditions threaten effective AI governance and safety because the objective of trustworthy, accountable, and transparent AI is intrinsically linked to the questionable ability of AI operators to provide meaningful explanations. Unless governments explicitly tackle the issue of explainability through clear legislative and policy statements that take into account technical realities, AI governance risks becoming a vacuous "box-ticking" exercise where scientific standards are replaced with legalistic thresholds, providing only a false sense of security in XAI.