Counterfactual Harm: A Counter-argument

TL;DR

This paper critiques the counterfactual definition of harm in AI ethics, showing it can produce intransitive and inconsistent treatment rankings in multi-treatment scenarios, and advocates for an interventionist approach based on expected utility.

Contribution

It identifies a fundamental flaw in the counterfactual harm definition when applied to multiple treatments and proposes an alternative interventionist framework to ensure transitive and consistent harm assessments.

Findings

Counterfactual harm can lead to intransitive treatment rankings.

An interventionist approach based on expected utility ensures transitivity.

Illustrative example with tuberculosis treatments demonstrates the problem.

Abstract

As AI systems are increasingly used to guide decisions, it is essential that they follow ethical principles. A core principle in medicine is non-maleficence, often equated with ``do no harm''. A formal definition of harm based on counterfactual reasoning has been proposed and popularized. This notion of harm has been promoted in simple settings with binary treatments and outcomes. Here, we highlight a problem with this definition in settings involving multiple treatment options. Illustrated by an example with three tuberculosis treatments (say, A, B, and C), we demonstrate that the counterfactual definition of harm can produce intransitive results: B is less harmful than A, C is less harmful than B, yet C is more harmful than A when compared pairwise. This intransitivity poses a challenge as it may lead to practical (clinical) decisions that are difficult to justify or defend. In contrast, an interventionist definition of harm based on expected utility forgoes counterfactual comparisons and ensures transitive treatment rankings.