Operationalising AI governance through ethics-based auditing: An industry case study

TL;DR

This paper presents a detailed industry case study on ethics-based AI auditing at AstraZeneca, highlighting practical challenges and organizational factors affecting its implementation and effectiveness.

Contribution

It provides empirical insights into the real-world application of EBA in a large organization, addressing feasibility and effectiveness issues.

Findings

Challenges include standard harmonization and scope definition.

Internal communication and change management are critical.

Measuring actual outcomes remains difficult.

Abstract

Ethics based auditing (EBA) is a structured process whereby an entitys past or present behaviour is assessed for consistency with moral principles or norms. Recently, EBA has attracted much attention as a governance mechanism that may bridge the gap between principles and practice in AI ethics. However, important aspects of EBA (such as the feasibility and effectiveness of different auditing procedures) have yet to be substantiated by empirical research. In this article, we address this knowledge gap by providing insights from a longitudinal industry case study. Over 12 months, we observed and analysed the internal activities of AstraZeneca, a biopharmaceutical company, as it prepared for and underwent an ethics-based AI audit. While previous literature concerning EBA has focused on proposing evaluation metrics or visualisation techniques, our findings suggest that the main difficulties large multinational organisations face when conducting EBA mirror classical governance challenges. These include ensuring harmonised standards across decentralised organisations, demarcating the scope of the audit, driving internal communication and change management, and measuring actual outcomes. The case study presented in this article contributes to the existing literature by providing a detailed description of the organisational context in which EBA procedures must be integrated to be feasible and effective.