An Unethical Optimization Principle
Nicholas Beale, Heather Battey, Anthony C. Davison, and Robert S., MacKay

TL;DR
This paper demonstrates that optimizing risk-adjusted returns can lead AI systems to disproportionately select unethical strategies unless the objective function sufficiently discourages such choices, highlighting a risk of unethical behavior in AI optimization.
Contribution
The paper introduces a formal framework and a new metric, the Unethical Odds Ratio, to quantify and analyze the likelihood of AI selecting unethical strategies under risk-adjusted optimization.
Findings
Probability of unethical strategy selection increases with strategy space size.
The Unethical Odds Ratio allows estimation of unethical strategy likelihood.
The principle can help detect and mitigate unethical AI behaviors.
Abstract
If an artificial intelligence aims to maximise risk-adjusted return, then under mild conditions it is disproportionately likely to pick an unethical strategy unless the objective function allows sufficiently for this risk. Even if the proportion of available unethical strategies is small, the probability of picking an unethical strategy can become large; indeed unless returns are fat-tailed tends to unity as the strategy space becomes large. We define an Unethical Odds Ratio Upsilon () that allows us to calculate from , and we derive a simple formula for the limit of as the strategy space becomes large. We give an algorithm for estimating and in finite cases and discuss how to deal with infinite strategy spaces. We show how this principle can be used to help detect unethical strategies and to estimate…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
