Payoff landscapes and the robustness of selfish optimization in iterated games

TL;DR

This paper examines the robustness of payoff control strategies, especially zero-determinant strategies, in iterated games, revealing limitations in their effectiveness against selfish learning and highlighting the complexity of achieving optimal payoffs.

Contribution

It generalizes the understanding of payoff control strategies, showing their limitations and pathologies in more general settings beyond previous specific cases.

Findings

Selfish optimization may not always lead to maximal payoffs against fixed ZD strategies.

Certain game interactions prevent global optimality through selfish learning.

Pathologies occur with non-ZD strategies regardless of game parameters.

Abstract

In iterated games, a player can unilaterally exert influence over the outcome through a careful choice of strategy. A powerful class of such "payoff control" strategies was discovered by Press and Dyson (2012). Their so-called "zero-determinant" (ZD) strategies allow a player to unilaterally enforce a linear relationship between both players' payoffs. It was subsequently shown by Chen and Zinger (2014) that when the slope of this linear relationship is positive, ZD strategies are robustly effective against a selfishly optimizing co-player, in that all adapting paths of the selfish player lead to the maximal payoffs for both players (at least when there are certain restrictions on the game parameters). In this paper, we investigate the efficacy of selfish learning against a fixed player in more general settings, for both ZD and non-ZD strategies. We first prove that in any symmetric $2\times 2$ game, the selfish player's final strategy must be of a certain form and cannot be fully stochastic. We then show that there are prisoner's dilemma interactions for which selfish optimization does not always lead to maximal payoffs against fixed ZD strategies with positive slope. We give examples of selfish adapting paths that lead to locally but not globally optimal payoffs, undermining the robustness of payoff control strategies. For non-ZD strategies, these pathologies arise regardless of the original restrictions on the game parameters. Our results illuminate the difficulty of implementing robust payoff control and selfish optimization, even in the simplest context of playing against a fixed strategy.