The explanation dialogues: an expert focus study to understand requirements towards explanations within the GDPR

TL;DR

This study explores legal experts' expectations of explainable AI within the GDPR, revealing gaps in current explanations and providing recommendations for improving explanation design and legal compliance.

Contribution

It presents the first expert focus study on legal expectations of XAI in the EU context, offering insights and guidelines for developers and legal practitioners.

Findings

Explanations are often hard to understand and lack sufficient information.

Legal experts highlight issues related to transparency, contestability, and intellectual property rights.

Recommendations address explanation presentation, content, and legal considerations.

Abstract

Explainable AI (XAI) provides methods to understand non-interpretable machine learning models. However, we have little knowledge about what legal experts expect from these explanations, including their legal compliance with, and value against European Union legislation. To close this gap, we present the Explanation Dialogues, an expert focus study to uncover the expectations, reasoning, and understanding of legal experts and practitioners towards XAI, with a specific focus on the European General Data Protection Regulation. The study consists of an online questionnaire and follow-up interviews, and is centered around a use-case in the credit domain. We extract both a set of hierarchical and interconnected codes using grounded theory, and present the standpoints of the participating experts towards XAI. We find that the presented explanations are hard to understand and lack information, and discuss issues that can arise from the different interests of the data controller and subject. Finally, we present a set of recommendations for developers of XAI methods, and indications of legal areas of discussion. Among others, recommendations address the presentation, choice, and content of an explanation, technical risks as well as the end-user, while we provide legal pointers to the contestability of explanations, transparency thresholds, intellectual property rights as well as the relationship between involved parties.