Using a theory of mind to find best responses to memory-one strategies

TL;DR

This paper explores how memory-one strategies in the Iterated Prisoner's Dilemma can be optimized using a theory of mind, revealing limitations of such strategies and the potential benefits of longer memory in multi-agent interactions.

Contribution

It introduces a comprehensive framework for analyzing and optimizing memory-one strategies with a theory of mind, extending understanding beyond extortionate tactics.

Findings

Optimal play is often not extortionate.

Memory-one strategies are limited in multi-agent settings.

Longer memory strategies can outperform memory-one strategies.

Abstract

Memory-one strategies are a set of Iterated Prisoner's Dilemma strategies that have been praised for their mathematical tractability and performance against single opponents. This manuscript investigates best response memory-one strategies with a theory of mind for their opponents. The results add to the literature that has shown that extortionate play is not always optimal by showing that optimal play is often not extortionate. They also provide evidence that memory-one strategies suffer from their limited memory in multi agent interactions and can be out performed by optimised strategies with longer memory. We have developed a theory that has allowed to explore the entire space of memory-one strategies. The framework presented is suitable to study memory-one strategies in the Prisoner's Dilemma, but also in evolutionary processes such as the Moran process, Furthermore, results on the stability of defection in populations of memory-one strategies are also obtained.