Shutdownable Agents through POST-Agency

TL;DR

The paper introduces POST-Agents, a method to ensure AI agents remain shutdownable by training them to satisfy preferences only between same-length trajectories, supported by theoretical proofs of their properties.

Contribution

It proposes the POST-Agents framework and proves that it guarantees shutdownability while maintaining usefulness, addressing a key safety concern in AI development.

Findings

POST-Agents satisfy Preferences Only Between Same-Length Trajectories.

POST-Agents imply Neutrality+, ensuring agents ignore trajectory-length probabilities.

Neutrality+ maintains agent shutdownability and utility maximization.

Abstract

Many fear that future artificial agents will resist shutdown. I present an idea - the POST-Agents Proposal - for ensuring that doesn't happen. I propose that we train agents to satisfy Preferences Only Between Same-Length Trajectories (POST). I then prove that POST - together with other conditions - implies Neutrality+: the agent maximizes expected utility, ignoring the probability distribution over trajectory-lengths. I argue that Neutrality+ keeps agents shutdownable and allows them to be useful.