On the meaning of uncertainty for ethical AI: philosophy and practice

TL;DR

This paper explores how explicitly acknowledging the statistical foundations of AI can improve accountability, transparency, and ethical considerations, demonstrated through COVID-19 model analysis for the UK government.

Contribution

It introduces a method to incorporate ethical considerations into statistical reasoning by extending Posterior Belief Assessment for complex AI structures.

Findings

Enhanced model responsiveness to feedback

Improved interpretation of uncertainty in AI outputs

Increased transparency for evaluation

Abstract

Whether and how data scientists, statisticians and modellers should be accountable for the AI systems they develop remains a controversial and highly debated topic, especially given the complexity of AI systems and the difficulties in comparing and synthesising competing claims arising from their deployment for data analysis. This paper proposes to address this issue by decreasing the opacity and heightening the accountability of decision making using AI systems, through the explicit acknowledgement of the statistical foundations that underpin their development and the ways in which these dictate how their results should be interpreted and acted upon by users. In turn, this enhances (1) the responsiveness of the models to feedback, (2) the quality and meaning of uncertainty on their outputs and (3) their transparency to evaluation. To exemplify this approach, we extend Posterior Belief Assessment to offer a route to belief ownership from complex and competing AI structures. We argue that this is a significant way to bring ethical considerations into mathematical reasoning, and to implement ethical AI in statistical practice. We demonstrate these ideas within the context of competing models used to advise the UK government on the spread of the Omicron variant of COVID-19 during December 2021.