Reactive learning strategies for iterated games
Alex McAvoy, Martin A. Nowak
TL;DR
This paper introduces reactive learning strategies for iterated games, showing they can effectively restrict payoff regions and are mathematically equivalent to certain memory-one strategies, with implications for game outcome control.
Contribution
It establishes a formal connection between reactive learning strategies and memory-one strategies, demonstrating their effectiveness in constraining feasible payoffs in iterated games.
Findings
Feasible payoff region is convex hull of at most 11 points against a memory-one strategy.
Reactive learning strategies' payoff regions are subsets of those from equivalent memory-one strategies.
Reactive learning strategies are powerful tools for outcome restriction in iterated games.
Abstract
In an iterated game between two players, there is much interest in characterizing the set of feasible payoffs for both players when one player uses a fixed strategy and the other player is free to switch. Such characterizations have led to extortionists, equalizers, partners, and rivals. Most of those studies use memory-one strategies, which specify the probabilities to take actions depending on the outcome of the previous round. Here, we consider "reactive learning strategies," which gradually modify their propensity to take certain actions based on past actions of the opponent. Every linear reactive learning strategy, , corresponds to a memory one-strategy, , and vice versa. We prove that for evaluating the region of feasible payoffs against a memory-one strategy, , we need to check its performance against at most …
| point | ||
|---|---|---|
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Reactive learning strategies for iterated games
Alex McAvoy and Martin A. Nowak
Abstract.
In an iterated game between two players, there is much interest in characterizing the set of feasible payoffs for both players when one player uses a fixed strategy and the other player is free to switch. Such characterizations have led to extortionists, equalizers, partners, and rivals. Most of those studies use memory-one strategies, which specify the probabilities to take actions depending on the outcome of the previous round. Here, we consider “reactive learning strategies,” which gradually modify their propensity to take certain actions based on past actions of the opponent. Every linear reactive learning strategy, , corresponds to a memory one-strategy, , and vice versa. We prove that for evaluating the region of feasible payoffs against a memory-one strategy, , we need to check its performance against at most other strategies. Thus, is the convex hull in of at most points. Furthermore, if is a memory-one strategy, with feasible payoff region , and is the corresponding reactive learning strategy, with feasible payoff region , then is a subset of . Reactive learning strategies are therefore powerful tools in restricting the outcomes of iterated games.
Keywords: adaptive strategy; iterated game; memory-one strategy; social dilemma
1. Introduction
Since the discovery of zero-determinant strategies for iterated games by Press and Dyson [1], there has been a growing interest in the set of possible payoffs that can be achieved against a fixed strategy. Imagine that Alice uses a particular strategy, while Bob can try out any conceivable strategy. The resulting set of payoffs for both Alice and Bob define the “feasible region” of Alice’s strategy. If Alice uses a so-called zero-determinant strategy [1], then the feasible region is a line. In general, the feasible region is a two-dimensional convex subset of the feasible payoff region of the game (Fig. 1). Using the geometric intuition put forth by Press and Dyson [1], subsequent work has explored strategies that generate two-dimensional feasible regions, defined by linear inequalities rather than strict equations [2, 3, 4]. However, a general description of what this region looks like, as it relates to the type of strategy played, is currently not well-understood. In this study, we characterize the feasible regions for the well-known class of memory-one strategies [5] and consider their relationships to those of a new class of “reactive learning strategies.”
Iterated games have many applications across the social sciences and biology, and with them has come a proliferation of strategy classes of various complexities [6, 7, 8, 9, 10]. The type of strategy a player uses for dealing with repeated encounters depends on many factors, including the cognitive capacity of the player and the nature of the underlying “one-shot” (or “stage”) games. In applications to theoretical biology, the most well-studied type of strategy is known as “memory-one” because it takes into account the outcome of only the previous encounter when determining how to play in the next round [5, 11]. This class of strategies, while forming only a small subset of all possible ways to play an iterated game [12], has several advantages over more complicated strategies. They permit rich behavior in iterated play, such as punishment for exploitation and reward for cooperation [5, 13, 14, 15, 16, 17, 18]; but, owing to their simple memory requirements, they are also straightforward to implement in practice and analyze mathematically.
Memory, however, can apply to more than just the players’ actions in the previous round. Since the action a player chooses in any particular encounter is typically chosen stochastically rather than deterministically, a player can also take into account how they chose their previous action rather than just the result. In a social dilemma, for instance, each player chooses an action (“cooperate,” , or “defect,” ) in a given round and receives a payoff for this action against that of the opponent. The distribution with which this action is chosen is referred to as a “mixed action” and can be specified by a single number between [math] and , representing the tendency to cooperate. A standard memory-one strategy for player is given by a five-tuple, , where is the probability of cooperation in the initial round and is the probability of cooperation following an outcome in which uses action and the opponent, , uses action . We consider a variation on this theme, where instead of using and to determine the next mixed action, uses the opponent’s action, , to update their own mixed action, , that was used previously to generate . We refer to a strategy of this form as a “reactive learning strategy.”
Such a strategy is “reactive” because it takes into account the realized action of just the opponent, and it is “learning” because it adapts to this external stimulus. Like a memory-one strategy, a reactive learning strategy for requires knowledge of information one round into the past, namely ’s mixed action, , and ’s realized action, . Unlike a memory-one strategy, in which the probability of cooperation is in the set in every round of the game, a reactive learning strategy can result in a broad range of cooperation tendencies for over the duration of an iterated game. Moreover, these tendencies can be gradually changed over the course of many rounds, resulting (for example) in high probabilities of cooperation only after the opponent has demonstrated a sufficiently long history of cooperating. Punishment for defection can be similarly realized over a number of interactions. Remembering a probability, , and an action, , instead of just two actions, and , can thus lead to more complex behaviors.
This adaptive approach to iterated games is similar to the Bush-Mosteller reinforcement learning algorithm [19, 20, 21], but there are important distinctions. For one, a reactive learning strategy does not necessarily reinforce behavior resulting in higher payoffs. Furthermore, it completely disregards the focal player’s realized action, using only that of the opponent in the update mechanism. But there are certainly reactive learning strategies that are more closely related to reinforcement learning, and we give an example using a variation on the memory-one strategy tit-for-tat (TFT), which we call “learning tit-for-tat (LTFT).”
In this study, we establish some basic properties of reactive learning strategies relative to the memory-one space. We first characterize the feasible region of a memory-one strategy as the convex hull of at most points. When then show that there is an embedding of the set of memory-one strategies in the set of reactive learning strategies with the following property: if is a memory-one strategy and is the corresponding reactive learning strategy, then the feasible region of contains the feasible region of . Moreover, the image of the map is the set of linear reactive learning strategies, which consists of those strategies that send a player’s mixed action, , to for some . As a consequence, if the goal of a player is to restrict the region of payoffs attainable by the players, then this player should prefer using a linear reactive learning strategy over the corresponding memory-one strategy.
2. Memory-one strategies
Consider an iterated game between two players, and . In every round, each player chooses an action from the set (“cooperate” or “defect”). They receive payoffs based on the values in the matrix
[TABLE]
Over many rounds, these payoffs are averaged to arrive at an expected payoff for each player.
Whereas an action specifies the behavior of a player in one particular encounter, a strategy specifies how a player behaves over the course of many encounters. One of the simplest and best-studied strategies for iterated games is a memory-one strategy [5], which for player is defined as follows: for every observed as action outcomes of a given round, devises a mixed action for the next round. The notation indicates that this mixed action depends on the (pure) actions of both players in the previous round, not how they arrived at those actions (e.g. by generating an action probabilistically). The term “strategy” is reserved for the players’ behaviors in the iterated game.
Let be the space of all memory-one strategies for player in an iterated game. With just two actions, and , we have , i.e. the space of all . A pair of memory-one strategies, and , for and , respectively, yield a Markov chain on the space of all action pairs, , whose transition matrix is
[TABLE]
and whose initial distribution is . If for every , then this chain is ergodic and has a unique stationary distribution, , which is independent of . In particular, the expected payoffs, and , are independent of and . In this case, and are functions of just the response probabilities, and .
A useful way of thinking about a strategy is through its feasible region, i.e. the set of all possible payoff pairs (for and ) that can be achieved against it. For any memory-one strategy of , let
[TABLE]
be this feasible region. (Note that, if uses a memory-one strategy, then it suffices to assume that uses a memory-one strategy by the results of Press and Dyson [1].) This subset of the feasible region represents the “geometry” of strategy in the sense that it captures all possible payoff pairs against an opponent.
In this section, we show that the feasible region for with is characterized by playing against the following strategies: , , , , , , , , , , and . In other words, is the convex hull of points (see Fig. 1). Therefore, any generates a simple polygon in whose number of extreme points is uniformly bounded over all game-strategy pairs, .
Lemma 1**.**
For and , let be the strategy obtained from by changing to . If , , and , then the point falls on the line joining and .
Proof.
Let and . Since the transition matrix of Eq. 2 is just , one can directly solve for its stationary distribution, (e.g. by using Gaussian elimination or the determinant formula of Press and Dyson [1]). For example, suppose that . Then, with
[TABLE]
one has
[TABLE]
Provided , we also have and . Moreover, one can check that, under this condition, is nowhere equal to [math], and for every . The other cases with are analogous. ∎
Remark 1**.**
Even when is uniformly distributed between [math] and , the corresponding points in the feasible region need not be uniformly distributed between the endpoints corresponding to and , respectively (see Fig. 2). This result is therefore somewhat different from the analogous situation of playing against a mixed action in a stage game, where, for a payoff function and mixed action and , one has due to linearity.
Proposition 1**.**
For any with , is the convex hull of the following points:
[TABLE]
Proof.
Press and Dyson [1] show that if uses a memory-one strategy, , then any strategy of the opponent, , can be replaced by a memory-one strategy, , without changing the payoffs to and ; thus, if uses a memory-one strategy, one may assume without a loss of generality that also uses a memory-one strategy. If and , the fact that can be written as a convex combination of the points then follows immediately from Lemma 1. Moreover, the points corresponding to , , and are the same, as are the points corresponding to , , and ; thus, we can eliminate four points. Furthermore, we can remove the point associated to because it lies on the line connecting the points associated to and . One can easily check that the remaining points have the following property: if point is removed, then there exist and for which is not the convex hull of the points different from (Table 1). Thus, for a general and payoff matrix, all of these points are required.
∎
Remark 2**.**
enforces a linear payoff relationship if and only if these points are collinear.
Remark 3**.**
One needs all of these points for general and . However, for any particular game-strategy pair, it is often the case that several of these points are unnecessary because they lie within the convex hull of some other subset of these points; they are typically not all extreme points of .
3. Reactive learning strategies
In a traditional memory-one strategy, ’s probability of playing depends on the realized actions of the two players, and . However, can observe more than just their pure action against the opponent’s; they also know how they arrived at (i.e. they know the mixed action, , that resulted in in the previous round). Of course, need not be able to see ’s mixed action, but they can still observe the pure action played. Therefore, an alternative notion of a memory-one strategy for player could be defined as follows: after plays and plays , then chooses a new action based on the distribution . In this formulation, is a map from to . We refer to such a map, , together with ’s initial probability of playing , , as a “reactive learning strategy” for player (Fig. 3).
In other words, in contrast to , which can be alternatively described as
[TABLE]
we define the space of reactive learning strategies as
[TABLE]
where indicates the space of mixed actions for and indicates the action space for . Although is a much larger space than , the updates of mixed actions can be easier to specify using reactive learning strategies since they allow for adaptive modification of an existing mixed action (without the need to devise a new mixed action from scratch after every observed history of play).
Example 1**.**
Suppose that player starts by playing and with equal probability, i.e. . For fixed (the “learning rate”), cooperation from the opponent leads to while defection leads to . Thus, a long pattern of exploitation by leads to defect more often. On the other hand, does not immediately forgive such behavior but rather requires to cooperate repeatedly to bring back up to higher levels of cooperation. For example, if starts with and defects times in a row, then subsequently cooperates with probability . In order to bring ’s probability of cooperation above once again, must then cooperate for rounds, where
[TABLE]
We refer to this strategy as “learning tit-for-tat (LTFT)” because it pushes a player’s cooperation probability in the direction of the opponent’s last move (see Fig. 4). In this way, a reactive learning strategy can encode more complicated behavior than a memory-one strategy. Conversely, memory-one strategies can also encode behavior not captured by reactive learning strategies, which we discuss further in §3.3.
3.1. Linear reactive learning strategies
A pertinent question at this point is whether there is a “natural” map from to . Let be a memory-one strategy. If is the corresponding reactive learning strategy, then the first requirement we impose is . If , then plays with probability one. It is therefore reasonable to insist that . Similarly, plays with probability one when , and we insist that . Suppose now that and are two mixed actions for . If plays , then the responses for corresponding to and are and , respectively. If plays with probability and with probability , then it is also natural to insist that the response is with probability and with probability . Thus, for any and , with these requirements can be written uniquely in terms of as
[TABLE]
Using this map, one can naturally identify with the set of linear reactive learning strategies, , consisting of those functions for which there exist with
[TABLE]
Clearly, any such must lie in since for every and .
Under this correspondence, the strategy of Example 1 has parameters . But note that this map, , is not surjective due to the fact that not every reactive learning strategy is linear. For example, if and is the quadratic response function defined by
[TABLE]
then there exists no mapping to provided or .
3.2. Stationary distributions
Suppose that and are reactive learning strategies for and , respectively. These strategies generate a Markov chain on the (infinite) space with transition probabilities between \Big{(}\left(x,y\right),\left(\sigma_{X},\sigma_{Y}\right)\Big{)},\Big{(}\left(x^{\prime},y^{\prime}\right),\left(p_{\sigma_{X}y}^{\ast},q_{\sigma_{Y}x}^{\ast}\right)\Big{)}\in\left\{C,D\right\}^{2}\times\left[0,1\right]^{2} given by
[TABLE]
To simplify notation, we can also denote the right-hand side of this equation by .
If is a stationary distribution of this chain, then, for any \Big{(}\left(x,y\right),\left(\sigma_{X},\sigma_{Y}\right)\Big{)}\in\left\{C,D\right\}^{2}\times\left[0,1\right]^{2},
[TABLE]
In general, is difficult to give explicitly. However, it is possible to understand the marginal distributions on and in more detail (see Appendix). In any case, having an explicit formula for is not necessary for obtaining our main result on feasible payoff regions, which we turn to in the next section.
3.3. Feasible payoff regions
By looking at the feasible region of a strategy, we uncover a nice relationship between a memory-one strategy, , and its corresponding (linear) reactive learning strategy, . Namely, for every , we have . In this section, we give a proof of this fact and illustrate some of its consequences.
For , let be the history of play from time [math] through time [12]. When , , where denotes the “empty” history, indicating that no play came before the present encounter. A behavioral strategy for a player specifies, for every possible history of play, a probability of using in the next encounter. That is, if , then a behavioral strategy is a map . The following lemma shows that when considering the feasible region of a memory-one or reactive learning strategy, one can assume without a loss of generality that the opponent is playing a Markov strategy:
Lemma 2**.**
Let be the set of all Markov strategies, i.e.
[TABLE]
For any , we have .
Proof.
When , the lemma follows from [1, Appendix A]. Specifically, when plays against , consider the time- distributions on and on . For ,
[TABLE]
Therefore, the same sequence of distributions arises when uses the Markov strategy defined by
[TABLE]
If is a reactive learning strategy that uses against , then for every there are distributions on , on , and on . For ,
[TABLE]
Consider the Markov strategy for with and . For , let
[TABLE]
If and are the analogues of and for against , then clearly and for . Suppose that for some , we have and . It follows, then, that at time ,
[TABLE]
which gives the desired result for . ∎
This lemma leads to a straightforward proof of our main result:
Theorem 1**.**
for every .
Proof.
By Lemma 2, for , we may assume the opponent’s strategy is Markovian, meaning that it has a memory of one round into the past but can depend on the current round, . This dependence on distinguishes a Markov strategy from a memory-one strategy, the latter of which also has memory of one round into the past but is independent of . We denote by the set of all Markov strategies (Eq. 15).
Let be a linear reactive learning strategy for and suppose that . For every , these strategies generate a distribution over . For any strategy against , there is a sequence of distributions on generated by these two strategies. We prove the proposition by finding such that for every and .
Let be the (marginal) distribution on at time . For , denote by this distribution conditioned on using action at time . For , consider the strategy with and
[TABLE]
Clearly, for every . Suppose, for some , that for every . For , we then have
[TABLE]
Therefore, by induction and the definition of expected payoff in an iterated game, . ∎
As a consequence of Theorem 1, we see that is a enforces a linear payoff relationship [1] whenever does. However, the converse need not hold; Fig. 5(b) gives an example in which ’s payoff is a function of ’s when uses but not when uses . Although this example illustrates an extreme case of when the payoff region collapses, perhaps the most interesting behavior is illustrated by Fig. 5(a),(c),(d). In these examples, we focus on the payoff regions that can be obtained against memory-one opponents. Using instead of can both bias payoffs in favor of and limit potential losses against a spiteful opponent.
For a memory-one strategy , we can ask how the region compares to . In other words, does the map transform the feasible region of a strategy when the opponents are also subjected to this map? Fig. 6 demonstrates that this map can significantly distort the distribution of payoffs within the feasible region.
3.4. Optimization through mutation
Suppose that uses a fixed reactive learning strategy, , for some . Starting from some random memory-one strategy, , the opponent might seek to optimize his or her payoff through a series of mutations. In other words, is subjected to the following process: First, sample a new strategy . If the payoff to for against exceeds that of against , switch to ; otherwise, retain . This step then repeats until has a sufficiently high payoff (or else has not changed strategies in some fixed number of steps). From Fig. 6, one expects this process to give different results from the same update scheme when plays the memory-one strategy instead of .
As expected, Fig. 7 shows that this optimization process behaves quite differently against as it does against . Whereas using in this example results in equitable outcomes, using gives a much higher payoff than , indicating extortionate behavior. One can also imagine other optimization procedures (not covered here), such as when is always sufficiently close to (i.e. local mutations). When uses , a path from the red point to the magenta point in Fig. 6 through random local sampling of typically requires to initially accept lower payoffs. If uses instead of , as in Fig. 6(b), this effect is amplified.
4. Discussion
Our primary focus has been on the feasible region generated by a fixed strategy. This approach to studying ’s strategy is inspired by the “zero-determinant” strategies of Press and Dyson [1], which enforce linear subsets of the feasible region. This perspective has also been expanded to cover so-called “partner” and “rival” strategies [2, 3, 4], which have proven extremely useful in understanding repeated games from an evolutionary perspective. The feasible region of a memory-one strategy, , is quite simple and can be characterized as the convex hull of at most points. Furthermore, these points are all straightforward to write down explicitly in terms of the payoff matrix and the entries of (see Eq. 6). The feasible region of a reactive learning strategy, in terms of its boundary and extreme points, is evidently more complicated in general.
Both memory-one and reactive learning strategies contain the set of all reactive strategies. For every memory-one strategy, , there exists a corresponding linear reactive learning strategy, , and this correspondence defines an injective map . In general, however, cannot be identified with its image, , unless is reactive. We make this claim formally using the geometry of a strategy within the feasible region, , which captures all possible payoff pairs against an opponent. For any memory-one strategy, we have . Therefore, reactive learning strategies generally allow a player to impose greater control over where payoffs fall within the feasible region than do traditional memory-one strategies. As illustrated in Fig. 5(a), this added control can prevent a greedy, self-payoff-maximizing opponent from obtaining more than when uses , even when such an opponent receives an unfair share of the payoffs when uses instead. The proof of the containment also extends to discounted games, where each payoff unit received rounds into the future is valued at units at present for some “discounting factor,” .
Another property of the map sending to is that it distorts the distribution of payoffs within the feasible region. Since can be identified with the space of linear reactive learning strategies under this map, it is natural to compare the region of possible payoffs when plays against memory-one strategies to the one obtained from when plays against linear reactive learning strategies. These distortions, as illustrated in Fig. 6, are particularly relevant when plays against an opponent who is using a process such as simulated annealing to optimize payoff. One can see from this example that if initially has a low payoff, then with localized strategy exploration they must be willing to accept lower payoffs before they find a strategy that improves their initial payoff. This concern is not relevant when can simply compute the best response to ’s strategy, but it is highly pertinent to evolutionary settings in which the opponent’s strategy is obtained through mutation and selection rather than “computation.”
Reactive learning strategies are also more intuitive than memory-one strategies in some ways. Rather than being a dictionary of mixed actions based on all possible observed outcomes, a reactive learning strategy is simply an algorithm for updating one’s tendency to choose a certain action. It therefore allows a player to alter their behavior (mixed action) over time in response to various stimuli (actions of the opponent). This strategic approach to iterated games is reminiscent of both the Bush-Mosteller model [19] and the weighted majority algorithm [24], although traditionally these models are not studied through the payoff regions they generate in iterated games. There are several interesting directions for future research in this area. For one, we have mainly considered the space of linear reactive learning strategies, but the space is much larger and could potentially exhibit complicated evolutionary dynamics. Furthermore, one could relax the condition that these strategies be reactive and allow them to use ’s realized action in addition to ’s mixed action. But even without these complications, we have seen that linear reactive learning strategies have quite interesting relationships to traditional memory-one strategies.
Appendix. Convergence of mixed actions
Suppose that and use strategies and , respectively. Let and be the initial distributions on for and , respectively. If these distributions are known at time , then, on average, the corresponding distributions at time are given by the system of equations,
[TABLE]
This system suggests a fixed-point analysis to determine whether the sequence converges.
Suppose that is a fixed point of this system, i.e.
[TABLE]
We consider this system for two types of linear reactive learning strategies: those coming from reactive strategies and those coming from general memory-one strategies under the map .
We first consider reactive strategies of the form , where (resp. ) is the probability a player uses after the opponent played (resp. ). Let and be fixed strategies for and . For these reactive strategies, the system Eq. A1 takes the form
[TABLE]
One can easily check that this dynamical system has a unique fixed point, which Hofbauer and Sigmund [23] refer to as the “asymptotic -level” of against , and which is given explicitly by
[TABLE]
Furthermore, we have the following, straightforward convergence result:
Proposition 2**.**
If , and if is given by Eq. A4, then
[TABLE]
for any initial condition, .
Proof.
For , consider the map
[TABLE]
For , we have
[TABLE]
It follows that , where . By the contraction mapping theorem, there is then a unique fixed point such that
[TABLE]
for any . It is straightforward to check that Eq. A4 is a fixed point of Eq. A3. ∎
In particular, if , then a straightforward calculation shows that is the stationary distribution of (Eq. 2).
Remark 4**.**
Proposition 2 need not hold if and are not strictly between [math] and . For example, when and both play TFT, is a simple involution with , which preserves distance.
Consider now the case of general memory-one strategies with for and for . For these strategies, the system defined by Eq. A1 has the form
[TABLE]
In the spirit of Proposition 2, for fixed , we could consider the map
[TABLE]
and analyze its fixed points. At this point, however, a couple of remarks are in order:
- (i)
need not be a contraction, even when and have entries strictly between [math] and . For example, with and ,
[TABLE]
We would conjecture that this map is an eventual contraction, in which case the convergence result of Proposition 2 still holds (although the explicit formulas for and differ from Eq. A4). 2. (ii)
a fixed point of , , even when it exists and is unique, generally does not have the property that , where is the stationary distribution of Eq. 2. Furthermore, the long-run mean-frequency distribution on can be distinct from both of these distributions, including when the opponent plays against and when they play against . An example of when these four distributions are pairwise distinct is easy to write down, e.g. and . All four distributions coincide when and are both reactive, but in general they can be distinct.
Acknowledgments
The authors are grateful to Krishnendu Chatterjee, Christian Hilbe, and Joshua Plotkin for many helpful conversations and for feedback on earlier versions of this work.
Funding statement
The authors gratefully acknowledge support from the Lifelong Learning Machines program from DARPA/MTO. Research was sponsored by the Army Research Laboratory (ARL) and was accomplished under Cooperative Agreement Number W911NF-18-2-0265. The views and conclusions contained in this document are those of the authors and should not be interpreted as representing the official policies, either expressed or implied, of the Army Research Laboratory or the U.S. Government. The U.S. Government is authorized to reproduce and distribute reprints for Government purposes notwithstanding any copyright notation herein.
The reference list from the paper itself. Each links out to its DOI / PubMed record.
- 1Press and Dyson [2012] W. H. Press and F. J. Dyson. Iterated prisoner’s dilemma contains strategies that dominate any evolutionary opponent. Proceedings of the National Academy of Sciences , 109(26):10409–10413, May 2012. doi: 10.1073/pnas.1206569109.
- 2Akin [2015] E. Akin. What You Gotta Know to Play Good in the Iterated Prisoner’s Dilemma. Games , 6(3):175–190, Jun 2015. doi: 10.3390/g 6030175.
- 3Hilbe et al. [2015] C. Hilbe, A. Traulsen, and K. Sigmund. Partners or rivals? Strategies for the iterated prisoner’s dilemma. Games and Economic Behavior , 92:41–52, Jul 2015. doi: 10.1016/j.geb.2015.05.005.
- 4Hilbe et al. [2018] C. Hilbe, K. Chatterjee, and M. A. Nowak. Partners and rivals in direct reciprocity. Nature Human Behaviour , 2018. doi: 10.1038/s 41562-018-0320-9.
- 5Nowak and Sigmund [1993] M. Nowak and K. Sigmund. A strategy of win-stay, lose-shift that outperforms tit-for-tat in the Prisoner’s Dilemma game. Nature , 364(6432):56–58, Jul 1993. doi: 10.1038/364056 a 0.
- 6Axelrod [1984] R. Axelrod. The Evolution of Cooperation . Basic Books, 1984.
- 7Lehrer [1988] E. Lehrer. Repeated games with stationary bounded recall strategies. Journal of Economic Theory , 46(1):130–144, Oct 1988. doi: 10.1016/0022-0531(88)90153-6.
- 8Hauert and Schuster [1997] C. Hauert and H. G. Schuster. Effects of increasing the number of players and memory size in the iterated prisoner’s dilemma: a numerical approach. Proceedings of the Royal Society of London B: Biological Sciences , 264(1381):513–519, 1997. doi: 10.1098/rspb.1997.0073.