Resource Rational Contractualism Should Guide AI Alignment

TL;DR

This paper introduces Resource-Rational Contractualism, a framework enabling AI systems to efficiently approximate human agreements by using heuristics, facilitating adaptive and scalable alignment in complex social environments.

Contribution

It proposes a novel resource-rational approach to contractualist AI alignment, combining heuristics and normative principles for scalable decision-making.

Findings

Framework enables efficient approximation of human agreements

Allows AI to adapt dynamically to social changes

Balances effort and accuracy in decision processes

Abstract

AI systems will soon have to navigate human environments and make decisions that affect people and other AI agents whose goals and values diverge. Contractualist alignment proposes grounding those decisions in agreements that diverse stakeholders would endorse under the right conditions, yet securing such agreement at scale remains costly and slow -- even for advanced AI. We therefore propose Resource-Rational Contractualism (RRC): a framework where AI systems approximate the agreements rational parties would form by drawing on a toolbox of normatively-grounded, cognitively-inspired heuristics that trade effort for accuracy. An RRC-aligned agent would not only operate efficiently, but also be equipped to dynamically adapt to and interpret the ever-changing human social world.