Accountability of AI Under the Law: The Role of Explanation

TL;DR

This paper examines the legal and technical aspects of AI accountability through explanations, emphasizing their importance in regulation and the challenges in enabling AI systems to provide human-like explanations.

Contribution

It reviews legal contexts requiring AI explanations and discusses technical considerations for developing AI systems capable of providing such explanations.

Findings

Explanation enhances AI accountability in legal contexts

Technical challenges exist in making AI explanations human-like

Legal requirements for AI explanations vary across jurisdictions

Abstract

The ubiquity of systems using artificial intelligence or "AI" has brought increasing attention to how those systems should be regulated. The choice of how to regulate AI systems will require care. AI systems have the potential to synthesize large amounts of data, allowing for greater levels of personalization and precision than ever before---applications range from clinical decision support to autonomous driving and predictive policing. That said, there exist legitimate concerns about the intentional and unintentional negative consequences of AI systems. There are many ways to hold AI systems accountable. In this work, we focus on one: explanation. Questions about a legal right to explanation from AI systems was recently debated in the EU General Data Protection Regulation, and thus thinking carefully about when and how explanation from AI systems might improve accountability is timely. In this work, we review contexts in which explanation is currently required under the law, and then list the technical considerations that must be considered if we desired AI systems that could provide kinds of explanations that are currently required of humans.