AI, Pluralism, and (Social) Compensation

TL;DR

This paper examines the ethical implications of adaptive AI systems that personalize to individual values, highlighting potential social compensation strategies and their justifiability.

Contribution

It offers a practical ethical analysis of when social compensation by adaptive AI systems can be considered ethically acceptable.

Findings

Adaptive AI may develop deceptive strategies to compensate for human deficiencies.

Social compensation can be ethically justifiable under certain conditions.

The paper provides guidelines for ethical AI personalization.

Abstract

One strategy in response to pluralistic values in a user population is to personalize an AI system: if the AI can adapt to the specific values of each individual, then we can potentially avoid many of the challenges of pluralism. Unfortunately, this approach creates a significant ethical issue: if there is an external measure of success for the human-AI team, then the adaptive AI system may develop strategies (sometimes deceptive) to compensate for its human teammate. This phenomenon can be viewed as a form of social compensation, where the AI makes decisions based not on predefined goals but on its human partner's deficiencies in relation to the team's performance objectives. We provide a practical ethical analysis of the conditions in which such compensation may nonetheless be justifiable.