An AGI with Time-Inconsistent Preferences

TL;DR

This paper highlights a critical flaw in AGI design assumptions, showing that assuming time-consistent preferences is unrealistic, and that AGI with time-inconsistent preferences faces unique decision-making challenges.

Contribution

It introduces the concept that AGI may have time-inconsistent preferences, challenging standard economic assumptions used in AI decision-making models.

Findings

Time-inconsistent preferences complicate AGI planning.

AGI with such preferences cannot fully trust future selves.

Standard discounting methods may be inadequate for AGI.

Abstract

This paper reveals a trap for artificial general intelligence (AGI) theorists who use economists' standard method of discounting. This trap is implicitly and falsely assuming that a rational AGI would have time-consistent preferences. An agent with time-inconsistent preferences knows that its future self will disagree with its current self concerning intertemporal decision making. Such an agent cannot automatically trust its future self to carry out plans that its current self considers optimal.