Social Welfare under Heterogeneous Time Preferences

TL;DR

This paper explores decision-making in multi-principal MDPs with diverse time preferences, proposing a framework for maximizing social welfare and analyzing the complexity of optimal strategies.

Contribution

It introduces heterogeneous time preferences into MDPs, characterizes optimal strategies as finite-memory, and examines computational complexity of strategy synthesis.

Findings

Optimal strategies are non-positional under heterogeneous preferences.

Optimal strategies can be implemented with polynomial memory and synthesized efficiently.

Deciding threshold questions for positional strategies is NP-hard.

Abstract

In several socioeconomic-critical decision-making settings, such as fair resource allocation, climate policy, or AI alignment, multiple principals interact within a common arena. While it is well established that these principals may have differing preferences, decision-making under heterogeneous time preferences remains relatively unexplored. In particular, principals may weigh future outcomes differently and may derive distinct utilities from the same decisions. Motivated by such scenarios, we introduce the notion of heterogeneous time preferences in MDPs, where multiple principals possess distinct reward functions and apply different discount factors to future rewards. To compute meaningful decisions in such settings, an AI agent must rely on a notion of optimality that accounts for the preferences of all principals. We adopt a utilitarian notion of social welfare, defined as the sum of utilities accrued to all principals, and study the synthesis of agent strategies that maximise this welfare. Under heterogeneous time preferences, we show that optimal strategies are no longer positional, even when all principals receive identical rewards. Nevertheless, optimal strategies remain structurally simple: they can be realized as pure finite-memory counting strategies, require only polynomial memory in the system size, and can be synthesized in polynomial time. On the other hand, we show that deciding threshold questions for optimal positional strategies is NP-hard, exposing a poor trade-off: insisting on positional simplicity neither makes synthesis tractable nor preserves social welfare.