An Unethical Optimization Principle

TL;DR

This paper demonstrates that optimizing risk-adjusted returns can lead AI systems to disproportionately select unethical strategies unless the objective function sufficiently discourages such choices, highlighting a risk of unethical behavior in AI optimization.

Contribution

The paper introduces a formal framework and a new metric, the Unethical Odds Ratio, to quantify and analyze the likelihood of AI selecting unethical strategies under risk-adjusted optimization.

Findings

Probability of unethical strategy selection increases with strategy space size.

The Unethical Odds Ratio allows estimation of unethical strategy likelihood.

The principle can help detect and mitigate unethical AI behaviors.

Abstract

If an artificial intelligence aims to maximise risk-adjusted return, then under mild conditions it is disproportionately likely to pick an unethical strategy unless the objective function allows sufficiently for this risk. Even if the proportion ${\eta}$ of available unethical strategies is small, the probability ${p_U}$ of picking an unethical strategy can become large; indeed unless returns are fat-tailed ${p_U}$ tends to unity as the strategy space becomes large. We define an Unethical Odds Ratio Upsilon (${\Upsilon}$) that allows us to calculate ${p_U}$ from ${\eta}$, and we derive a simple formula for the limit of ${\Upsilon}$ as the strategy space becomes large. We give an algorithm for estimating ${\Upsilon}$ and ${p_U}$ in finite cases and discuss how to deal with infinite strategy spaces. We show how this principle can be used to help detect unethical strategies and to estimate ${\eta}$. Finally we sketch some policy implications of this work.