Mean Field Equilibrium: Uniqueness, Existence, and Comparative Statics

TL;DR

This paper establishes conditions for the uniqueness, existence, and comparative statics of mean field equilibrium in large-player stochastic games, facilitating analysis where traditional methods are computationally infeasible.

Contribution

It provides the first known uniqueness conditions for MFE, generalizes existence results, and offers broad comparative statics applicable to economic and operational models.

Findings

Proved conditions ensuring MFE uniqueness.

Generalized MFE existence results.

Derived comparative statics for large-player models.

Abstract

The standard solution concept for stochastic games is Markov perfect equilibrium (MPE); however, its computation becomes intractable as the number of players increases. Instead, we consider mean field equilibrium (MFE) that has been popularized in the recent literature. MFE takes advantage of averaging effects in models with a large number of players. We make three main contributions. First, our main result provides conditions that ensure the uniqueness of an MFE. We believe this uniqueness result is the first of its nature in the class of models we study. Second, we generalize previous MFE existence results. Third, we provide general comparative statics results. We apply our results to dynamic oligopoly models and to heterogeneous agent macroeconomic models commonly used in previous work in economics and operations.

Equations

$$x_{i,t}=w(x_{i,t-1},a_{i,t-1},{\mathbf{x}}_{-\mathbf{i},\mathbf{t}-1},{\zeta }_{i,t}).$$

$$s_{-i,t}^{(m)}(y)=\frac{1}{m-1}\sum _{j\ne i}{1}_{\{x_{j,t}=y\}}$$

$$\pi (x,a,s)=P\left({\int }_X\bar{q}(y)s(dy)\right)\bar{q}(x)-da,$$

$$w(x,a,s,\zeta )=((1-\delta )x+k(a))\zeta ,$$

$$\underset{n\to \infty }{\lim }{\int }_{X}f(x)s_n(dx)={\int }_{X}f(x)s(dx).$$

$$V_{\sigma }(x,s)={\mathbb{E}}_{\sigma }\left(\sum _{t=1}^{\infty }{\beta }^{t-1}\pi (x(t),a(t),s)\right).$$

$$V(x,s)=\underset{\sigma }{\sup }{V}_{\sigma }(x,s).$$

$$V(x,s)=\underset{a\in \Gamma (x)}{\max }\pi (x,a,s)+\beta \sum _{j=1}^n{p}_{j}V(w(x,a,s,{\zeta }_j),s).$$

$$G(x,s)=\underset{a\in \Gamma (x)}{\mathrm{argmax}}\pi (x,a,s)+\beta \sum _{j=1}^n{p}_{j}V(w(x,a,s,{\zeta }_j),s).$$

$$Q_g(x,s,B)=\mathrm{Pr}(w(x,g(x,s),s,\zeta )\in B).$$

$$s(B)={\int }_X{Q}_g(x,s,B)s(dx).$$

$$\Phi s(B)={\int }_X{Q}_g(x,s,B)s(dx),$$

$$M_s\theta (B)={\int }_{X}Q(x,s,B)\theta (dx).$$

$${\int }_{X}f(x)s_1(dx)\ge {\int }_{X}f(x)s_2(dx),$$

$$M_{s_2}{\theta }_2(B)$$

$$\le {\int }_{X}Q(x,s_1,B){\theta }_2(dx)$$

$$\le {\int }_{X}Q(x,s_1,B){\theta }_1(dx)$$

$$=M_{s_1}{\theta }_1(B).$$

$${\int }_{X}f(y_1,y_2)Q((x_1,x_2),s,d(y_1,y_2))$$

$$x_{i,t}=((1-\delta )x_{i,t-1}+k(a_{i,t-1})){\zeta }_{i,t}$$

$$u(x,s)=P\left({\int }_X\bar{q}(y)s(dy)\right)\bar{q}(x).$$

$$u_{ijt}={\theta }_1\ln (x_{it}+1)+{\theta }_2\ln (Y-p_{it})+v_{ijt},$$

$$u(x,s)=\frac{\tilde{c}(x+1{)}^{{\theta }_1}}{{\int }_X(y+1{)}^{{\theta }_1}s(dy)}$$

$$x_{i,t}=(1-\delta )(x_{i,t-1}+a_{i,t-1}){\zeta }_{i,t}.$$

$$\pi (x,a,s)=r\frac{(x+a{)}^{{\gamma }_1}}{(\int (x^{\prime }+a^{\prime })s(d(x^{\prime },a^{\prime })){)}^{{\gamma }_2}}-a$$

$$x_{i,t}=\left(\min \left(\frac{x_{i,t-1,2}}{1+x_{i,t-1,2}}{x}_{i,t-1,1}+\frac{1}{1+x_{i,t-1,2}}(k(a)+{\zeta }_{i,t}),M_1\right),\min \left(x_{i,t-1,2}+1,M_2\right)\right),$$

$$\pi (x,a,s)=\frac{\nu (x_1,x_2)}{\int \nu (x_1,x_2)s(d(x_1,x_2))}-da$$

$$(x_{i,t,1},x_{i,t,2})=(a_{i,t-1},m(x_{i,t-1,2},{\zeta }_{i,t})),$$

$$Q(x_1,x_2,s,B_1\times B_2)=1_{B_1}(\tilde{g}(x_1,x_2,H(s))\sum _j{p}_j{1}_{B_2}(m(x_2,{\zeta }_j)),$$

$$\tilde{\pi }(x,a,H(s))=u\left((f^{\prime }(H(s))-\delta +1)x_1+(f(H(s))-f^{\prime }(H(s))H(s))x_2-a\right).$$

$$s\left(B\times D\right)={\int }_X{\int }_B{1}_D(g\left(y,s\right))Q\left(x,s,dy\right)s\left(dx,A\right)={\int }_X\overline{Q}(x,s,B\times D)s(dx,A),$$

$$\overline{Q}\left(x,s,B\times D\right)={\int }_B{1}_D(g\left(y,s\right))Q\left(x,s,dy\right).$$

Full text

Mean Field Equilibrium: Uniqueness, Existence, and Comparative Statics111The authors wish to thank Aaron Bodoh-Creed, Ramesh Johari, Bob Wilson, three anonymous referees, and the associate and area editors, as well as seminar participants at Stanford and several conferences for their valuable comments. The second author thanks Joseph and Laurie Lacob for the support during the 2017-2018 and 2018-2019 academic years as a Joseph and Laurie Lacob Faculty Scholar at Stanford Graduate School of Business.

Bar Light222Graduate School of Business, Stanford University, Stanford, CA 94305, USA. e-mail: barl@stanford.edu   and Gabriel Y. Weintraub333 Graduate School of Business, Stanford University, Stanford, CA 94305, USA. e-mail: gweintra@stanford.edu

Abstract:

The standard solution concept for stochastic games is Markov perfect equilibrium (MPE); however, its computation becomes intractable as the number of players increases. Instead, we consider mean field equilibrium (MFE) that has been popularized in the recent literature. MFE takes advantage of averaging effects in models with a large number of players. We make three main contributions. First, our main result provides conditions that ensure the uniqueness of an MFE. We believe this uniqueness result is the first of its nature in the class of models we study. Second, we generalize previous MFE existence results. Third, we provide general comparative statics results. We apply our results to dynamic oligopoly models and to heterogeneous agent macroeconomic models commonly used in previous work in economics and operations.

Keywords: Dynamic games; Mean field equilibrium; Uniqueness of equilibrium; Comparative statics; Dynamic oligopoly models; Heterogeneous agent macroeconomic models

**

1 Introduction

In this paper we consider a general class of stochastic games in which every player has an individual state that impacts payoffs. Historically, Markov perfect equilibrium (MPE) has been a standard solution concept for this type of stochastic games (Maskin and Tirole, 2001). However, in realistically-sized applications, MPE suffers from two drawbacks. First, because in MPE players keep track of the state of every competitor, the state space grows very quickly as the number of players grows, making the analysis and computation of MPE infeasible in many applications of practical interest. Second, as the number of players increases, it becomes difficult to believe that players can in fact track the exact state of the other players and optimize their strategies accordingly.

As an alternative, mean field equilibrium (MFE) has received extensive attention in the recent literature. In an MFE, each player optimizes her expected discounted payoff, assuming that the distribution of the other players’ states is fixed. Given the players’ strategy, the distribution of the players’ states is an invariant distribution of the stochastic process that governs the states’ dynamics. As a solution concept for stochastic games, MFE offers several advantages over MPE. First, because players only condition their strategies on their own state (the competitors’ state is assumed to be fixed), MFE is computationally tractable. Second, as several of the papers we cite below prove, due to averaging effects MFE provides accurate approximations of optimal behavior as the number of players grows. As a result, it provides an appealing behavioral model in games with many players.

MFE models have many applications in economics, operations research, and optimal control; e.g., studies of anonymous sequential games (Jovanovic and Rosenthal, 1988), continuous-time mean field models (Huang et al. (2006) and Lasry and Lions (2007)), dynamic user equilibrium (Friesz et al., 1993), auction theory (Iyer et al. (2014), Balseiro et al. (2015), and Bimpikis et al. (2018)), dynamic oligopoly models (Weintraub et al. (2008) and Adlakha et al. (2015)), heterogeneous agent macro models (Hopenhayn (1992) and Heathcote et al. (2009)), matching markets (Kanoria and Saban (2019) and Arnosti et al. (2020)), spatial competition (Yang et al., 2018), and evolutionary game theory (Tembine et al., 2009).

We provide three main contributions regarding MFE. First, we provide conditions that ensure the uniqueness of an MFE. This novel result is important because it implies sharp counterfactual predictions. Second, we generalize previous existence results to a general state space setting. Our existence result includes the case of a countable state space and a countable number of players, as well as the case of a continuous state space and a continuum of players. In addition, we provide novel comparative statics results for stochastic games that do not exhibit strategic complementarities.

We apply our results to well-known dynamic oligopoly models in which individual states represent the firms’ ability to compete in the market (Doraszelski and Pakes, 2007). MFE and the related concept of oblivious equilibrium have previously been used to analyze such models.444For example, Adlakha et al. (2015) use MFE, which they call stationary equilibrium. Adlakha et al. (2015) was motivated by Hopenhayn (1992) who introduced the term to study models with infinite numbers of firms. Weintraub et al. (2008) introduce oblivious equilibrium to study settings with finite numbers of firms. In the models we study, for each firm, being in a larger state is more profitable, while if competitors’ states are larger it is less profitable. This structure is quite natural in dynamic models of competition that have been studied in the operations research and economics literature, and we leverage it to prove our uniqueness result. We provide examples of dynamic investments models of quality, capacity, and advertising, as well as a dynamic reputation model of an online market. We also apply our results to commonly used heterogeneous agent macroeconomic models.

We now explain our contributions in more detail and compare them to previous work on MFE.

Uniqueness. We do not know of any general uniqueness result regarding MFE in discrete-time mean field equilibrium models.555Lasry and Lions (2007) prove the uniqueness of an MFE in a continuous time setting under a certain monotonicity condition (see also Carmona and Delarue (2018)). This monotonicity condition is different and does not hold in the applications studied in the present paper. Only a few papers have obtained uniqueness results in specific applications. Hopenhayn (1992) proves the uniqueness of an MFE in a specific dynamic competition model. Light (2020) proves the uniqueness of an MFE in a Bewley-Aiyagari model under specific conditions on the model’s primitives (see a related result in Hu and Shmaya (2019)). Our main theorem in this paper is a novel result that provides conditions ensuring the uniqueness of an MFE for broader classes of models. Informally, under mild additional technical conditions, we show that if the probability that a player reaches a higher state in the next period is decreasing in the other players’ states, and is increasing in the player’s own state in the current period, then the MFE is unique (see Theorem 1). Hence, the conditions reduce the difficulty of showing that a stochastic game has a unique MFE to proving properties of the players’ optimal strategies.

In many applications, one can show that these properties of the optimal strategies arise naturally. For example, in several dynamic models of competition in operations research and economics, a higher firm’s state (e.g., the quality of the firm’s product or the firm’s capacity) implies higher profitability, and the firm can make investments in each period in order to improve its state. In this setting, one can show that a firm invests less when its competitors’ states are higher; hence, competitors’ higher states induce a lower state for the firm in the next period. In contrast, if the firm’s own current state is higher, it induces a higher state in the next period. Another example is heterogeneous agent macro models where each agent solves a consumption-savings problem. The agents’ states correspond to their current savings level and current labor productivity. Under certain conditions it can be shown that an agent saves less when the other agents save more. On the other hand, the agents’ next period’s savings are increasing in their current savings.

We apply our uniqueness result to a general class of dynamic oligopoly models and heterogeneous agent macroeconomic models for which MFE has been used to perform counterfactual predictions implied by a policy or system change. In the past, in the absence of this result, previous work mostly focused on a particular MFE selected by a given algorithm, or on one with a specific structure. In the absence of uniqueness, the predictions often depend on the choice of the MFE, and therefore, uniqueness significantly sharpens such counterfactual analysis. We also show that the uniqueness results proved in Hopenhayn (1992) and Light (2020) can be obtained using our approach.

Existence. Prior literature has considered the existence of equilibria in stochastic games. Some prior work considered the existence of Markov perfect equilibria (MPE) (see Doraszelski and Satterthwaite (2010) and He and Sun (2017)). Adlakha et al. (2015) prove the existence of an MFE for the case of a countable and unbounded state space. Acemoglu and Jensen (2015) consider a closely related notion of equilibrium that is called stationary equilibrium and prove its existence for the case of a compact state space and a specific transition dynamic that is commonly used in economics (see Stokey and Lucas (1989)). Stationary equilibrium in the sense of Acemoglu and Jensen (2015) is an MFE where the players’ payoff functions depend on the other players’ states through an aggregator. Our existence result applies for a general compact state space, more general dependence on the payoff function, and more general transitions. In this sense, it is more closely related to the result of Adlakha and Johari (2013). Adlakha and Johari (2013) prove the existence of an MFE for the case of a compact state space in stochastic games with strategic complementarities using a lattice-theoretical approach. Instead, we do not assume strategic complementarities and our state space can be any compact separable metric space. For our existence result, we assume the standard continuity conditions on model primitives that are assumed in the papers mentioned above. In addition, we assume that the optimal stationary strategy of the players is single-valued.666In the dynamic oligopoly models and the heterogeneous agent macro models that we study in Sections 4 and 5, previous literature assumes that the players use pure strategies. Motivated by this fact, we focus on pure strategy MFE. In this case, if the optimal stationary strategy of the players is not single-valued then the MFE operator may not be convex-valued. Similar problems arise in proving the existence of a pure-strategy Nash equilibrium. Concavity conditions on the profit function and the transition function can be imposed in order to ensure that the optimal stationary strategy is indeed single-valued. The main technical difficulty in proving existence is to prove the weak continuity of the nonlinear MFE operator (see Theorem 3).

Comparative statics. While some papers contain certain specific results on how equilibria change with the parameters of the model (for example, see Hopenhayn (1992) and Aiyagari (1994)), only a few papers have obtained general comparative results in large dynamic economies (see Acemoglu and Jensen (2015) for a discussion of the difficulties associated with deriving such results). Three notable exceptions are Adlakha and Johari (2013), Acemoglu and Jensen (2015), and Acemoglu and Jensen (2018). Adlakha and Johari (2013) use the techniques for comparing equilibria developed in Milgrom and Roberts (1994) to derive general comparative statics results, and essentially rely on results about the monotonicity of fixed points. The direct application of these results requires that the MFE operator (see Equation (1)) be increasing. Our comparative statics results are different because they rely on the uniqueness of an MFE. In particular, the MFE operator is not increasing in our setting (see more details in Section 3). In this sense, our comparative static results are more similar to the results in Acemoglu and Jensen (2015); however, our model has more general dynamics that include, for example, investment decisions with random outcomes that are typically considered in dynamic oligopoly models (see Section 4). Our results are useful because they establish the directional changes of MFE when important model parameters, such as the discount factor and the investment cost, change.

2 The Model

In this section we define our general model of a stochastic game and define mean field equilibrium (MFE). The model and the definition of an MFE are similar to Adlakha and Johari (2013) and Adlakha et al. (2015).

2.1 Stochastic Game Model

In this section we describe our stochastic game model. Differently to standard stochastic games in the literature (see Shapley (1953)), in our model, every player has an individual state. Players are coupled through their payoffs and state transition dynamics. A stochastic game has the following elements:

Time. The game is played in discrete time. We index time periods by $t=1,2,\ldots\;\;\text{.}\;\;$

*Players. *There are $m$ players in the game. We use $i$ to denote a particular player.

States. The state of player $i$ at time $t$ is denoted by $x_{i,t}\in X$ where $X$ is a separable metric space. Typically, we assume that the state space $X$ is in $\mathbb{R}^{n}$ or that $X$ is countable. We denote the state of all players at time $t$ by $\bm{x}_{\mathbf{t}}$ and the state of all players except player $i$ at time $t$ by $\bm{x}_{-\bm{i},\bm{t}}$.

*Actions. *The action taken by player $i$ at time $t$ is denoted by $a_{i,t}\in A$ where $A\subseteq\mathbb{R}^{q}$. We use $\bm{a}_{\mathbf{t}}$ to denote the action of all players at time $t$. The set of feasible actions for a player in state $x$ is given by $\Gamma(x)\subseteq A$.

*States’ dynamics. *The state of a player evolves in a Markov fashion. Formally, let $h_{t}=\{\bm{x}_{0},\bm{a}_{0},\ldots,\bm{x}_{\bm{t}-1},\bm{a}_{\bm{t}-1}\}$ denote the history up to time $t$. Conditional on $h_{t}$, players’ states at time $t$ are independent of each other. This assumption implies that random shocks are idiosyncratic, ruling out aggregate random shocks that are common to all players. Player $i$’s state $x_{i,t}$ at time $t$ depends on the past history $h_{t}$ only through the state of player $i$ at time $t-1$, $x_{i,t-1}$; the states of other players at time $t-1$, $\bm{x}_{-\bm{i},\bm{t}-1}$; and the action taken by player $i$ at time $t-1$, $a_{i,t-1}$.

If player $i$’s state at time $t-1$ is $x_{i,t-1}$, the player takes an action $a_{i,t-1}$ at time $t-1$, the states of the other players at time $t-1$ are $\bm{x}_{-\bm{i},\bm{t}-1}$, and $\zeta_{i,t}$ is player $i$’s realized idiosyncratic random shock at time $t$, then player $i$’s next period’s state is given by

[TABLE]

We assume that $\zeta$ is a random variable that takes values $\zeta_{j}\in E$ with probability $p_{j}$ for $j=1,\ldots,n$. $w:X\times A\times X^{m-1}\times E\rightarrow X$ is the transition function.

Payoff. In a given time period, if the state of player $i$ is $x_{i}$, the state of the other players is $\bm{x}_{-\bm{i}}$, and the action taken by player $i$ is $a_{i}$, then the single-period payoff to player $i$ is $\pi(x_{i},a_{i},\bm{x}_{-\bm{i}})\in\mathbb{R}$. In Section 2.2 we extend our model to a model in which players are also coupled through actions, that is, the functions $w$ and $\pi$ can also depend on the rivals’ current actions.

Discount factor. The players discount their future payoff by a discount factor $0<\beta<1$. Thus, a player $i$’s infinite horizon payoff is given by: $\sum_{t=1}^{\infty}\beta^{t-1}\pi(x_{i,t},a_{i,t},\bm{x}_{-\bm{i},\bm{t}})$.

In many games, coupling between players is independent of the identity of the players. This notion of anonymity captures scenarios where the interaction between players is via aggregate information about the state (see Jovanovic and Rosenthal (1988)). Let $s_{-i,t}^{(m)}(y)$ denote the fraction of players excluding player $i$ that have their state as $y$ at time $t$. That is,

[TABLE]

where $1_{D}$ is the indicator function of the set $D$. We refer to $s_{-i,t}^{(m)}$ as the population state at time $t$ (from player $i$’s point of view).

Definition 1

*(Anonymous stochastic game). A stochastic game is called an *anonymous stochastic game if the payoff function $\pi(x_{i,t},a_{i,t},\bm{x}_{-\bm{i},\bm{t}})$ and the transition function $w(x_{i,t},a_{i,t},\bm{x}_{-\bm{i},\bm{t}},\zeta_{i,t+1})$ depend on $\bm{x}_{-i,t}$ only through $s_{-i,t}^{(m)}$. In an abuse of notation, we write $\pi(x_{i,t},a_{i,t},s_{-i,t}^{(m)})$ for the payoff to player $i$, and $w(x_{i,t},a_{i,t},s_{-i,t}^{(m)},\zeta_{i,t+1})$ for the transition function for player $i$.

For the remainder of the paper, we focus our attention on anonymous stochastic games. For ease of notation, we often drop the subscripts $i$ and $t$ and denote a generic transition function by $w(x,a,s,\zeta)$ and a generic payoff function by $\pi(x,a,s)$ where $s$ represents the population state of players other than the player under consideration. Anonymity requires that a player’s single-period payoff and transition function depend on the states of other players via their empirical distribution over the state space, and not on their specific identify. In anonymous stochastic games the functional form of the payoff function and transition function are the same, regardless of the number of players $m$.777Our results also generalize for models in which the primitives depend on the number of players $m$ like in the study of oblivious equilibria (Weintraub et al., 2008)). In that sense, we often interpret the profit function $\pi(x,a,s)$ as representing a limiting regime in which the number of players is infinite.

We now provide a simple model of capacity competition that illustrates some of the notation presented above. This is one of the dynamic competition models that we study in Section 4.1.

Example 1

Our example is based on the capacity competition models of Besanko and Doraszelski (2004) and Besanko et al. (2010). We consider an industry with homogeneous products, where each firm’s state variable determines its production capacity. If the firm’s state is $x$, then its capacity is $\bar{q}(x)$. In each period, each firm takes a costly action to improve its capacity in the next period. Further, in each period, firms compete in a capacity-constrained quantity setting game. The inverse demand function is given by $P(Q)$, where $Q$ represents the total industry output. For simplicity, we assume that the marginal costs of all the firms are equal to zero. Given the total quantity produced by its competitors $Q_{-i}$, the profit maximization problem for firm $i$ is given by $\underset{0\leq q_{i}\leq\bar{q}(x_{i})}{\max}P(q_{i}+Q_{-i})q_{i}$.

In general, one could solve for the equilibrium of the capacity-constrained static quantity game played by firms, and these static equilibrium actions would determine the single-period profits. However, we focus on the limiting regime with a large number of firms with out market power, that is, firms take $Q$ as fixed. In this case, each firm produces at full capacity and the limiting profit function is given by:

[TABLE]

where $a$ is the firm’s investment and $d$ is the unit investment cost (see also Ifrach and Weintraub (2016)). The next period’s state depends on the amount of investment, the current state, and a random shock. For example, assuming that the state depreciates at rate $\delta$, a possible transition function is given by:

[TABLE]

where $k$ is an increasing function that determines the impact of the firm’s investment and $\zeta$ represents uncertainty in the investment process.

Now, we let $\mathcal{P}(X)$ be the set of all possible population states on $X$, that is $\mathcal{P}(X)$ is the set of all probability measures on $X$. We endow $\mathcal{P}(X)$ with the weak topology. Since $\mathcal{P}(X)$ is metrizable, the weak topology on $\mathcal{P}(X)$ is determined by weak convergence (for details see Aliprantis and Border (2006)). We say that $s_{n}\in\mathcal{P}(X)$ converges weakly to $s\in\mathcal{P}(X)$ if for all bounded and continuous functions $f:X\rightarrow\mathbb{R}$ we have

[TABLE]

For the rest of the paper, we assume the following conditions on the primitives of the model:

Assumption 1

*(i) $\pi$ is bounded and (jointly) continuous. $w$ is continuous.888 Recall that we endow $\mathcal{P}(X)$ with the weak topology. *

(ii) $X$ is compact.

*(iii) The correspondence $\Gamma:X\rightarrow 2^{A}$ is compact-valued and continuous.999 By continuous we mean both upper hemicontinuous and lower hemicontinuous. *

2.2 Extensions To The Basic Model

We note two extensions that can be important in applications for which we can extend our results.

First, in our basic mean field model, we assume that the players are coupled through their states: both the transition function and the payoff function of each player depend on the states of all other players. We note that even in this setting, a player’s payoff function can depend on rivals’ actions as long as these actions do not affect the evolution of their own state nor the evolution of the population state. For instance, the players’ payoff functions can depend on the static pricing or quantity decisions of the other players. In Section 4.1 we study models in which the firms’ (static) actions affect other players’ current payoffs but do not affect the evolution of future states.

In certain models of interest such as learning-by-doing and dynamic advertising, however, players’ states are coupled through the dynamic actions, $a_{i,t}$. That is, the actions of other players, $\bm{a}_{-\bm{i},\bm{t}}$, affect a player’s transition function and payoff function. For these cases, we consider a model where the transition function and the payoff function of each player depend on both the states and the actions of all other players. The model is like our original model except that now the probability measure $s$ describes the joint distribution of players over actions and states and not only over states, that is, $s\in\mathcal{P}(X\times A)$. Thus, the transition function $w(x,a,s,\zeta)$ and the payoff function $\pi(x,a,s)$ depend on the joint distribution over state-action pairs $s\in\mathcal{P}(X\times A)$.

All the results in the paper can be extended to this setting where the population state is a measure on $\mathcal{P}(X\times A)$ (see Section A.1 in the Appendix for more details). The monotonicity conditions that are needed in order to prove the uniqueness of an MFE in the case that the population is a measure on $\mathcal{P}(X\times A)$ are similar to the conditions that are needed in the case that the population is a measure on $\mathcal{P}(X)$. In Section 4.2 we prove the uniqueness of an MFE for a dynamic advertising model where the players’ payoff functions depend on the other players’ actions (advertising expenditures), and thus, the population state is a measure on $\mathcal{P}(X\times A)$.

Our second extension relaxes the assumption on our base model that players are ex-ante homogeneous. To consider players that may be ex-ante heterogeneous with different model primitives, we extend our model to a setting in which each player has a fixed type through out the time horizon that is drawn from a finite set. Then, the payoff function and transition function can depend on this type. We show that all our results hold in this more general setting (see Section A.2 for more details). In particular, we show that if the conditions that we use in order to prove our results hold for every type, then the results are valid for the model with ex-ante heterogeneous players.

2.3 Mean Field Equilibrium

In Markov perfect equilibrium (MPE), players’ strategies are functions of the population state. However, MPE quickly becomes intractable as the number of players grows, because the number of possible population states becomes too large. Instead, in a game with a large number of players, we might expect that idiosyncratic fluctuations of players’ states “average out”, and hence the actual population state remains roughly constant over time. Because the effect of other players on a single player’s payoff and transition function is only via the population state, it is intuitive that, as the number of players increases, a single player’s effect on the outcome of the game is negligible. Based on this intuition, related schemes for approximating Markov perfect equilibrium (MPE) have been proposed in different application domains via a solution concept we call mean field equilibrium (MFE).

Informally, an MFE is a strategy for the players and a population state such that: (1) Each player optimizes her expected discounted payoff assuming that this population state is fixed; and (2) Given the players’ strategy, the fixed population state is an invariant distribution of the states’ dynamics. The interpretation is that a single player conjectures the population state to be $s$. Therefore, in determining her future expected payoff stream, a player considers a payoff function and a transition function evaluated at the fixed population state $s$. In MFE, the conjectured $s$ is the correct one given the strategies being played. MFE alleviates the complexity of MPE, because in the former the population state is fixed, while in the latter players keep track of the exact evolution of the population state. We refer the reader to the papers cited in Section 1 for a more detailed motivation and rigorous justifications for using MFE.

Let $X^{t}:=\underbrace{X\times\ldots\times X}_{t~{}\mathrm{t}\mathrm{i}\mathrm{m}\mathrm{e}\mathrm{s}}$. For a fixed population state, a nonrandomized pure strategy $\sigma$ is a sequence of (Borel) measurable functions $(\sigma_{1},\sigma_{2},\ldots,)$ such that $\sigma_{t}:X^{t}\rightarrow A$ and $\sigma_{t}(x_{1},\ldots,x_{t})\in\Gamma(x_{t})$ for all $t\in\mathbb{N}$. That is, a strategy $\sigma$ assigns a feasible action to every finite string of states. Note that a single player’s strategy depends only on her own history of states and does not depend on the population state. This strategy is called an oblivious strategy (see Weintraub et al. (2008) and Adlakha et al. (2015)).

For each initial state $x\in X$ and long run average population state $s\in\mathcal{P}(X)$, a strategy $\sigma$ induces a probability measure over the space $X^{\mathbb{N}}$, describing the evolution of a player’s state.101010The probability measure on $X^{\mathbb{N}}$ is uniquely defined (see for example Bertsekas and Shreve (1978)). We denote the expectation with respect to that probability measure by $\mathbb{E}_{\sigma}$, and the associated states-actions stochastic process by $\{x(t),a(t)\}_{t=1}^{\infty}$.

When a player uses a strategy $\sigma$, the population state is fixed at $s\in\mathcal{P}(X)$, and the initial state is $x\in X$, then the player’s expected present discounted value is

[TABLE]

Denote

[TABLE]

That is, $V(x,s)$ is the maximal expected payoff that the player can achieve when the initial state is $x$ and the population state is fixed at $s\in\mathcal{P}(X)$. We call $V$ the value function and a strategy $\sigma$ attaining it optimal.

Standard dynamic programming arguments (see Bertsekas and Shreve (1978)) show that the value function satisfies the Bellman equation:

[TABLE]

Under Assumption 1, there exists an optimal stationary Markov strategy (see Lemma 3 in the Appendix). Let $G(x,s)$ be the optimal stationary strategy correspondence, i.e.,

[TABLE]

Let $\mathcal{B}(X)$ be the Borel $\sigma$-algebra on $X$. For a strategy $g\in G$ and a fixed population state $s\in\mathcal{P}(X)$, the probability that player $i$’s next period’s state will lie in a set $B\in\mathcal{B}(X)$, given that her current state is $x\in X$ and she takes the action $a=g(x,s)$, is:

[TABLE]

Now suppose that the population state is $s$, and all players use a stationary strategy $g\in G$. Because of averaging effects, we expect that if the number of players is large, then the long run population state should in fact be an invariant distribution of the Markov kernel $Q_{g}$ on $X$ that describes the dynamics of an individual player.

We can now define an MFE. In an MFE, every player conjectures that $s$ is the fixed long run population state and plays according to a stationary strategy $g$. On the other hand, if every agent plays according to $g$ when the population state is $s$, then the long run population state of all players, $s$, should constitute an invariant distribution of $Q_{g}$.

Definition 2

A stationary strategy $g$ and a population state $s\in\mathcal{P}(X)$ constitute an MFE if the following two conditions hold:

1. Optimality: $g$ is optimal given $s$, i.e., $g(x,s)\in G(x,s)$.

2. Consistency: $s$ is an invariant distribution of $Q_{g}$. That is,

[TABLE]

for all $B\in\mathcal{B}(X)$, where we take Lebesgue integral with respect to the measure $s$.

Under Assumption 1 it can be shown that $G(x,s)$ is nonempty, compact-valued and upper hemicontinuous. The proof is a standard application of the maximum theorem. We provide the proof for completeness (see Lemma 3). In Theorem 3 we prove the existence of a population state that satisfies the consistency requirement in Definition 2.

3 Main Results

In this section we present our main results. In Section 3.1 we provide conditions that ensure the uniqueness of an MFE. In Section 3.2 we prove the existence of an MFE. In Section 3.3 we provide conditions that ensure unambiguous comparative statics results regarding MFE.

3.1 The Uniqueness of an MFE

In this section we present our uniqueness result.

We recall that a stationary strategy-population state pair $(g,s)$ is an MFE if and only if $g$ is optimal and $s$ is a fixed point of the operator $\Phi:\mathcal{P}(X)\rightarrow\mathcal{P}(X)$ defined by

[TABLE]

for all $B\in\mathcal{B}(X)$.

We prove uniqueness by showing that the operator $\Phi$ has a unique fixed point. In order to prove uniqueness we will assume that $G$ is single-valued. For the rest of the section we will assume that $g\in G$ is the unique selection from the optimal strategy correspondence $G$. In the next section we provide conditions that ensure that $G$ is indeed single-valued (see Lemma 1). $G$ being single-valued and Theorem 3 (see Section 3.2) imply that $\Phi$ has at least one fixed point. In Theorem 1 we will show that under certain conditions the operator $\Phi$ has at most one fixed point.

We omit the reference to $g$ in $Q_{g}(x,s,B)$, i.e., we write $Q(x,s,B)$ instead of $Q_{g}(x,s,B)$. Since the Markov kernel $Q$ depends on $s$, it is complicated to work directly with the operator $\Phi$. Thus, to prove the uniqueness of an MFE and to prove our comparative statics results, we introduce an auxiliary operator that is easier to work with. For each $s\in\mathcal{P}(X)$, define the operator $M_{s}:\mathcal{P}(X)\rightarrow\mathcal{P}(X)\;$by

[TABLE]

We introduce the following useful definition.

Definition 3

We say that $Q$ is $X$-ergodic if the following two conditions hold:

(i) For any $s\in\mathcal{P}(X)$, the operator $M_{s}$ has a unique fixed point $\mu_{s}$.

(ii) $M_{s}^{n}\theta$ converges weakly to $\mu_{s}$ for any probability measure $\theta\in\mathcal{P}(X)$.

Note that $s$ is an MFE if and only if $\mu_{s}=s$ is a fixed point of the operator $M_{s}$. $X$-ergodicity means that for every population state $s\in\mathcal{P}(X)$ the players’ long-run state is independent of the initial state. The $X$-ergodicity of $Q$ can be established using standard results from the theory of Markov chains in general state spaces (see Meyn and Tweedie (2012)). When $Q$ is increasing in $x$, which we assume in order to prove the uniqueness of an MFE (see Assumption 2), then the $X$-ergodicity of $Q$ can be established using results from the theory of monotone Markov chains. These results usually require a splitting condition (see Bhattacharya and Lee (1988) and Hopenhayn and Prescott (1992)) that typically holds in applications of interest. Specifically, in Sections 4 and 5 we show that $X$-ergodicity holds in important classes of dynamic models.

We now introduce other notation and definitions that are helpful in proving uniqueness. We assume that $X$ is endowed with a closed partial order $\geq$. In the important case $X=\mathbb{R}^{n}$, $x,y\in X$ we write $x\geq y$ if $x_{i}\geq y_{i}$ for each $i=1,..,n$. Let $S\subseteq X$. We say that a function $f:S\rightarrow\mathbb{R}$ is increasing if $f(y)\geq f(x)$ whenever $y\geq x$ and we say that $f$ is strictly increasing if $f(y)>f(x)$ whenever $y>x$.

For $s_{1},s_{2}\in\mathcal{P}(X)$ we say that $s_{1}$ stochastically dominates $s_{2}$ and we write $s_{1}\succeq_{SD}s_{2}$ if for every increasing function $f:X\rightarrow\mathbb{R}$ we have

[TABLE]

when the integrals exist. We say that $B\in\mathcal{B}(X)$ is an upper set if $x_{1}\in B$ and $x_{2}\geq x_{1}$ imply $x_{2}\in B$. Recall from Kamae et al. (1977) that $s_{1}\succeq_{SD}s_{2}$ if and only if for every upper set $B$ we have $s_{1}(B)\geq s_{2}(B)$.

In addition, for the rest of the section we will assume that there exists a binary relation $\succeq$ on $\mathcal{P}(X)$, such that $s_{2}\sim s_{1}$ (i.e., $s_{2}\succeq s_{1}$ and $s_{1}\succeq s_{2})$) implies $\pi(x,a,s_{1})=\pi(x,a,s_{2})$ for all $(x,a)\in X\times A$ and $w(x,a,s_{1},\zeta)=w(x,a,s_{2},\zeta)$ for all $(x,a,\zeta)\in X\times A\times E$.

Note that such binary relation always exists, for example one can take $s_{2}\sim s_{1}\Leftrightarrow s_{2}=s_{1}$. For our uniqueness result we will further require that the binary relation $\succeq$ on $\mathcal{P}(X)$ is complete, that is, for all $s_{1},s_{2}\in\mathcal{P}(X)$ we either have $s_{1}\succeq s_{2}$ or $s_{2}\succeq s_{1}$. In many applications (see Section 4 and Section 5) there exists a function $H:\mathcal{P}(X)\rightarrow\mathbb{R}$ such that $\tilde{\pi}(x,a,H(s))=\pi(x,a,s)$ and $\tilde{w}(x,a,H(s),\zeta)=w(x,a,s,\zeta)$, where $H$ is continuous and increasing with respect to the stochastic dominance order $\succeq_{SD}$. In this case, a natural complete order $\succeq$ on $\mathcal{P}(X)$ arises by defining $s_{1}\succeq s_{2}$ if and only if $H(s_{1})\geq H(s_{2})$. Below, we also discuss the case of a non-complete order. We say that $\succeq$ agrees with $\succeq_{SD}$ if for any $s_{1},s_{2}\in\mathcal{P}(X)$, $s_{1}\succeq_{SD}s_{2}$ implies $s_{1}\succeq s_{2}$.

We say that $Q$ is increasing in $x$ if for each $s\in\mathcal{P}(X)$, we have $Q(x_{2},s,\cdot)\succeq_{SD}Q(x_{1},s,\cdot)$ whenever $x_{2}\geq x_{1}$. In addition, we say that $Q$ is decreasing in $s$ if for each $x\in X$, we have $Q(x,s_{1},\cdot)\succeq_{SD}Q(x,s_{2},\cdot)$ whenever $s_{2}\succeq s_{1}$. We now state the main theorem of the paper. We show that if $Q$ is $X$-ergodic, $Q$ is increasing in $x$ and decreasing in $s$, and $\succeq$ is complete and agrees with $\succeq_{SD}$, then if an MFE exists, it is unique.

Intuitively, $Q$ decreasing in $s$ implies that the probability that a player will move to a higher state in the next period is decreasing in the current period’s population state. If there are two MFEs, $s_{2}$ and $s_{1}$, such that $s_{2}\succeq s_{1}$ (i.e., $s_{2}$ is “higher” than $s_{1}$), then the probability of moving to a higher state under $s_{2}$ is lower than under $s_{1}$, which is not consistent with $s_{2}\succeq s_{1}$, with the definition of an MFE, and the fact that $\succeq$ agrees with $\succeq_{SD}$.111111In some models, the condition that $Q$ is decreasing in $s$ follows from the fact that the policy function $g$ is decreasing in the population state $s$ (see Section 4). Xu and Hajek (2013) prove the uniqueness of an equilibrium in a supermarket mean field game under a similar monotonicity condition on the policy function. Their setting is different from ours because the players do not have individual states nor they dynamically optimize.

Assumption 2

(i) $Q$ is $X$-ergodic. $Q$ is increasing in $x$ and decreasing in $s$.

(ii) $\succeq$ agrees with $\succeq_{SD}$.

(iii) $G$ is single-valued.

Theorem 1

Suppose that Assumption 2 holds. If the binary relation $\succeq$ is complete, then if an MFE exists, it is unique.

Proof. Let $\theta_{1},\theta_{2}\in\mathcal{P}(X)$ and assume that $\theta_{1}\succeq_{SD}\theta_{2}$. Let $B$ be an upper set and let $s_{1},s_{2}$ be two MFEs such that $s_{2}\succeq s_{1}$. We have

[TABLE]

Thus, for any upper set $B$ we have $M_{s_{2}}\theta_{2}(B)\leq M_{s_{1}}\theta_{1}(B)$ which implies that $M_{s_{1}}\theta_{1}\succeq_{SD}M_{s_{2}}\theta_{2}$. The first inequality follows from the fact that $Q(x,s,B)$ is decreasing in $s$ for an upper set $B$ and all $x$. The second inequality follows from the fact that $\theta_{1}\succeq_{SD}\theta_{2}$ and $Q(x,s,B)$ is increasing in $x$ for an upper set $B$ and any $s$.

We conclude that $M_{s_{1}}^{n}\theta_{1}\succeq_{SD}M_{s_{2}}^{n}\theta_{2}$ for all $n\in\mathbb{N}$. $Q$ being $X$-ergodic implies that $M_{s_{i}}^{n}\theta_{i}$ converges weakly121212 Recall that $\mu_{s}$ is the unique fixed point of $M_{s}$ and that $s$ is an MFE if and only if $\mu_{s}=s$. to $\mu_{s_{i}}=s_{i}$. Since $\succeq_{SD}$ is closed under weak convergence (see Kamae et al. (1977)), we have $s_{1}\succeq_{SD}s_{2}$.

We conclude that if $s_{1}$ and $s_{2}$ are two MFEs such that $s_{2}\succeq s_{1}$, then $s_{1}\succeq_{SD}s_{2}$. Since $\succeq$ agrees with $\succeq_{SD}$, we have $s_{1}\succeq s_{2}$. That is, $s_{1}\sim s_{2}$, which implies that $\pi(x,a,s_{1})=\pi(x,a,s_{2})$ and $w(x,a,s_{1},\zeta)=w(x,a,s_{2},\zeta)$. Thus, under $s_{1}$ the players play according to the same strategy as under $s_{2}$ (i.e., $g(x,s_{1})=g(x,s_{2})$ for all $x\in X$). We conclude that $Q(x,s_{1},B)=Q(x,s_{2},B)$ for all $x\in X$ and $B\in\mathcal{B}(X)$. $X$-ergodicity of $Q$ implies that $M_{s_{1}}$ and $M_{s_{2}}$ have a unique fixed point. Thus, $\mu_{s_{1}}=\mu_{s_{2}}$, i.e., $s_{1}=s_{2}$. Similarly, we can show that $s_{1}\succeq s_{2}$ implies that $s_{1}=s_{2}$.

Since $\succeq$ is complete if $s_{1}$ and $s_{2}$ are two MFEs we have $s_{2}\succeq s_{1}$ or $s_{1}\succeq s_{2}$. Thus, we proved that if $s_{1}$ and $s_{2}$ are two MFEs then $s_{1}=s_{2}$. We conclude that if an MFE exists, it is unique.

The assumptions on $Q$ in Theorem 1 involve assumptions on the optimal strategy $g$. Thus, these assumptions are not over the primitives of the model. In Section 4 we introduce conditions on the primitives of dynamic oligopoly models that guarantee the uniqueness of an MFE. In particular, we show that the monotonicity conditions over $Q$ arise naturally in important classes of these models. In Section 5 we apply our result to prove the uniqueness of an MFE in heterogeneous agent macro models.

In some applications the assumption that the binary relation $\succeq$ is complete is restrictive. In the case that $\succeq$ is not complete and Assumption 2 holds, the following Corollary shows that the MFEs are not comparable by the binary relation $\succeq$. This Corollary can be used to derive properties on the MFE when there are multiple MFEs. For example, suppose that there exist two functions $H_{i}:\mathcal{P}(X)\rightarrow\mathbb{R}$, $i=1,2$ such that $\tilde{\pi}(x,a,H_{1}(s))=\pi(x,a,s)$ and $\tilde{w}(x,a,H_{2}(s),\zeta)=w(x,a,s,\zeta)$, where $H_{i}$ is continuous and increasing with respect to the stochastic dominance order $\succeq_{SD}$. We can define an order $\succeq$ on $\mathcal{P}(X)$ by defining $s_{1}\succeq s_{2}$ if $H_{1}(s_{1})\geq H_{1}(s_{2})$ and $H_{2}(s_{1})\geq H_{2}(s_{2})$. Clearly, this may not be a complete order. The following Corollary provides conditions that imply that if $s_{1}$ and $s_{2}$ are two MFEs, then it cannot be the case that $H_{1}(s_{1})>H_{1}(s_{2})$ and $H_{2}(s_{1})>H_{2}(s_{2})$. We write $s_{1}\succ s_{2}$ if $s_{1}\succeq s_{2}$ and $s_{2}\nsucceq s_{1}$.

Corollary 1

Suppose that Assumption 2 holds. If $s_{1}$ and $s_{2}$ are two MFEs then $s_{1}\nsucc s_{2}$ and $s_{2}\nsucc s_{1}$.

Proof. Suppose, in contradiction, that $s_{2}\succ s_{1}$. The argument in the proof of Theorem 1 implies that $s_{1}\succeq_{SD}s_{2}$. Since $\succeq$ agrees with $\succeq_{SD}$, we have $s_{1}\succeq s_{2}$, which is a contradiction. We conclude that $s_{2}\nsucc s_{1}$. Similarly, we can show that $s_{1}\nsucc s_{2}$.

When the state space $X$ is given by the product space $X=X_{1}\times X_{2}$ where $X_{1}$ and $X_{2}$ are separable metric spaces, a modification of our uniqueness result can be applied to prove the uniqueness of an MFE under slightly different conditions than the conditions of Assumption 2.

Assumption 2 requires that $Q$ be increasing in $x$ on $X$. However, when $X=X_{1}\times X_{2}$, and $X_{i}$ is endowed with the closed partial order $\geq_{i}$, it is enough to assume that $Q$ is increasing in $x_{i}$ on $X_{i}$ for some $i=1,2$ to prove that the MFE is unique. We say that $Q$ is increasing in $x_{1}$ if for all functions $f:X_{1}\times X_{2}\rightarrow\mathbb{R}$ that are increasing in $x_{1}$ on $X_{1}$, for all $s\in\mathcal{P}(X)$, and for all $x_{2}\in X_{2}$, the function

[TABLE]

is increasing in $x_{1}$. Similarly, $Q$ is decreasing in $s$ with respect to $x_{1}$ if for all functions $f:X_{1}\times X_{2}\rightarrow\mathbb{R}$ that are increasing in $x_{1}$on $X_{1}$ and for all $x\in X$ the function in (2) is decreasing in $s$. In Sections 4.3 and 5 we show the usefulness of Theorem 2. We establish the uniqueness of an MFE for dynamic reputation models and heterogeneous agent macro models by proving that $Q$ is increasing in $x_{i}$ for some $i=1,2$. In these models it is not necessarily true that $Q$ is increasing in $x$ on $X$, so Theorem 1 cannot be applied directly. The Appendix contains the proofs not presented in the main text.

Theorem 2

Suppose that $X=X_{1}\times X_{2}$. Suppose that Assumption 2 holds, apart from the condition that $Q$ is increasing in $x$ and decreasing in $s$. Suppose that $Q$ is increasing in $x_{i}$ and decreasing in $s$ with respect to $x_{i}$ for some $i=1,2$. If the binary relation $\succeq$ is complete, then if an MFE exists, it is unique.

3.2 The Existence of an MFE

In this section we study the existence of an MFE. We show that if $G$ is single-valued, then the operator $\Phi$ defined in Equation (1) has a fixed point and thus, there exists an MFE.

Theorem 3

Assume that $G\;$is single-valued. There exists a mean field equilibrium.

Note that we do not impose Assumption 2 for this result. Also note that $X$ can be any compact separable metric space in the proof of Theorem 3, so the existence result holds for the important cases of finite state spaces, countable state spaces, and $X\subseteq\mathbb{R}^{n}$. In addition, the proof of existence does not depend on the number of players in the game; the number of players in the game can be finite, countable or uncountable. Finally, we note that we do not require $X$-ergodicity (see Definition 3) to show existence; instead we use compactness and continuity (see Assumption 1). The main challenge to prove existence is to prove the weak continuity of the nonlinear MFE operator. To do so, we leverage a generalized version of the bounded convergence theorem by Serfozo (1982).

We now provide conditions over the model primitives that guarantee that $G$ is single-valued when $X$ is a convex set in $\mathbb{R}^{n}$. Similar conditions have been used in dynamic oligopoly models.131313For similar results in a countable state space setting see Adlakha et al. (2015) and Doraszelski and Satterthwaite (2010)).

Assumption 3

Suppose that $X\subseteq\mathbb{R}^{n}$ and is convex.

(i) Assume that $\pi(x,a,s)$ is concave in $(x,a)$, strictly concave in $a$ and increasing in $x$ for each $s\in\mathcal{P}(X)$.

(ii) Assume that $w$ is increasing in $x$ and concave in $(x,a)$ for each $\zeta\in E$.

(iii) $\Gamma(x)$ is convex-valued and increasing in the sense that $x_{2}\geq x_{1}$ implies $\Gamma(x_{2})\supseteq\Gamma(x_{1})$.

The following Lemma shows that the preceding conditions on the primitives of the model ensure that $G$ is single-valued.

Lemma 1

Suppose that Assumption 3 holds. Then $G$ is single-valued.

The previous results can be summarized by the following Corollary that imposes conditions over the primitives of the model which guarantee the existence of an MFE.

Corollary 2

Suppose that Assumption 3 holds. Then, there exists an MFE.

3.3 Comparative Statics

In this section we derive comparative statics results. Let $(I,\succeq_{I})$ be a partially ordered set that influences the players’ optimal decisions. We denote a generic element in $I$ by $e$. For example, $e$ can be the discount factor, a parameter that influences the players’ payoff functions, or a parameter that influences the players’ transition dynamics. Throughout this section we slightly abuse notation and when the parameter $e$ influences the players’ optimal decisions we add it as a parameter. For instance, we write $Q(x,s,e,\cdot)$ instead of $Q(x,s,\cdot)$. We say that $Q$ is increasing in $e$ if $Q(x,s,e_{2},\cdot)\succeq_{SD}Q(x,s,e_{1},\cdot)$ for all $x$, $s$, and all $e_{2},e_{1}\in I$ such that $e_{2}\succeq_{I}e_{1}$. We prove that under the assumptions of Theorem 1, if $Q$ is increasing in $e$ then $e_{2}\succeq_{I}e_{1}$ implies that the unique MFE under $e_{2}$ is higher than the unique MFE under $e_{1}$ with respect to $\succeq$.

Adlakha and Johari (2013) derive comparative statics results for MFE in the case that $Q$ is increasing in $s$, $x$ and $e$. They prove that $e_{2}\succeq_{I}e_{1}$ implies $s(e_{2})\succeq_{SD}s(e_{1})$ where $s(e)$ is the maximal MFE with respect to $\succeq_{SD}$ under $e$. Adlakha and Johari (2013) use the techniques to compare equilibria developed in Milgrom and Roberts (1994) (see also Topkis (2011)). We note that under the assumptions of Theorem 1, $Q$ is increasing in $x$ but decreasing in $s$. Thus, the results in Adlakha and Johari (2013) do not apply to our setting. However, with the help of the uniqueness of an MFE, we derive a general comparative statics result.

Theorem 4

Let $(I,\succeq_{I})$ be a partial order. Assume that $Q$ is increasing in $e$ on $I$. Then, under the assumptions of Theorem 1, the unique MFE $s(e)$ is increasing in the following sense: $e_{2}\succeq_{I}e_{1}$ implies $s(e_{2})\succeq s(e_{1})$.

The same result can be shown with a similar argument under the assumptions of Theorem 2. We omit the details for sake of brevity. We note that our comparative statics result is with respect to the order $\succeq$ and not with respect to the usual stochastic dominance order. The machinery mentioned in the paragraph above is not directly applicable in our models, and without it we believe that comparative statics results with respect to the usual stochastic dominance order are much harder to obtain. We discuss the usefulness of our comparative static result with respect to the order $\succeq$ in the context of dynamic oligopoly models below.

4 Dynamic Oligopoly Models

In this section we study various dynamic models of competition or dynamic oligopoly models that capture a wide range of phenomena in economics and operations research.141414Even though we study models with potentially large numbers of firms, we keep the name dynamic oligopoly to be consistent with previous literature in which MFE or its variants have been used to approximate oligopolistic behavior (for example, see Qi (2013), Adlakha et al. (2015), and Onishi (2016)). We leverage our results to provide conditions under which a broad class of dynamic oligopoly models admit a unique MFE. We also provide comparative statics results.

More specifically, we show that under concavity assumptions and a natural substitutability condition, the MFE is unique. The substitutability condition requires that the firms’ profit function has decreasing differences in each firm’s own state and the states of the other firms. This condition implies that the marginal profit of a firm (with respect to its own state) is decreasing in the other firms’ states. It arises naturally in many dynamic oligopoly models. In Section 4.1 we consider well studied capacity competition and quality ladder models. In Section 4.2 we consider a dynamic advertising model. In Section 4.3 we introduce a dynamic reputation model of an online market. In all of these models, it holds that the firms’ actions are higher when their own state is higher and the firms’ actions are lower when the competitors’ states (or the competitors’ actions) are higher. These are essentially the conditions that imply the uniqueness of an MFE for dynamic oligopoly models.

4.1 Capacity Competition and Quality Ladder Models

In this section we consider dynamic capacity competition models and dynamic quality ladder models which have received significant attention in the recent operations research and economics literature. In these models, firms’ states correspond to a variable that affects their profits. For example, the state can be the firm’s capacity or the quality of the firm’s product. Per-period profits are based on a static competition game that depends on the heterogeneous firms’ state variables. Firms take actions in order to improve their individual state over time.

We now describe the models we consider.

States. The state of firm $i$ at time $t$ is denoted by $x_{i,t}\in X$ where $X\subseteq\mathbb{R}_{+}$ and is convex.

Actions. At each time $t$, firm $i$ invests $a_{i,t}\in A=[0,\bar{a}]$ to improve its state. The investment changes the firm’s state in a stochastic fashion.

States’ dynamics. A firm’s state evolves in a Markov fashion. Let $0<\delta<1$ be the depreciation rate. If firm $i$’s state at time $t-1$ is $x_{i,t-1}$, the firm takes an action $a_{i,t-1}$ at time $t-1$, and $\zeta_{i,t}$ is firm $i$’s realized idiosyncratic random shock at time $t$, then firm $i$’s state in the next period is given by:

[TABLE]

where $k:A\rightarrow\mathbb{R}$ is typically an increasing function that determines the impact of investment $a$. We assume that $\zeta$ takes positive values $0<\zeta_{1}<\ldots<\zeta_{n}$, where $\zeta_{1}<1$, $\zeta_{n}>1$, $p_{1},p_{n}>0$. That is, there exists a positive probability for a bad shock $\zeta_{1}$ and a positive probability for a good shock $\zeta_{n}$. In each period, the firm’s state is naturally depreciating at rate $\delta$, but the firm can make investments in order to improve it. Further, the outcome of depreciation and investment is subject to an idiosyncratic random shock ($\zeta$) that, for example, could capture uncertainty in R&D or a marketing campaign. Related dynamics have been used in previous literature. Further, our uniqueness result for capacity competition and quality ladder models holds under other states’ dynamics. For example, we could also assume additive dynamics $x_{i,t}=(1-\delta)x_{i,t-1}+k(a_{i,t-1})+\zeta_{i,t}$.151515For our results to hold we need to impose some constraints on these additive dynamics so that the state space remains compact. We can also assume an exogenous bound on the state as in Section 4.3. We believe that our results also hold if we drop the assumption that $X$ is compact, under some additional conditions over model primitives that ensure some form of “decreasing returns to larger states” (see Adlakha et al. (2015)). We make the following assumption over the dynamics that we discuss later before Theorem 5.

Assumption 4

(i) $k(a)$ is strictly concave, continuously differentiable, strictly increasing and $k(0)>0$.161616 The differentiability assumptions can be relaxed. We assume differentiability of $u$ and $k$ in order to simplify the proof of Theorem 5.

(ii) $(1-\delta)\zeta_{n}<1$.

Payoff. The cost of a unit of investment is $d>0$.171717The investment cost could be a convex function, but linearity simplifies the comparative static results in the parameter $d$. We assume there is a single-period profit function $u(x,s)$ derived from a static game. When a firm invests $a\in A$, the firm’s state is $x\in X$, and the population state is $s\in\mathcal{P}(X)$, then the firm’s single-period payoff function is given by $\pi(x,a,s)=u(x,s)-da$.

We assume that there exists a complete and transitive binary relation $\succeq$ on $\mathcal{P}(X)$ such that $s_{1}\sim s_{2}$ implies that $u(x,s_{1})=u(x,s_{2})$ for all $s_{1},s_{2}\in\mathcal{P}(X)$ and $x\in X$. Furthermore, we assume that $\succeq$ agrees with $\succeq_{SD}$ (cf. Section 3.1).

To prove the uniqueness of an MFE for capacity competition and quality ladder models, we introduce the following conditions on the primitives of the model. It is simple to verify that both of the dynamic oligopoly models introduced in the examples below satisfy these assumptions. We believe the conditions are quite natural, and thus other commonly used dynamic oligopoly models may satisfy them as well.

Recall that a function $f(x,s)$ is said to have decreasing differences in $(x,s)$ on $X\times S$ if for all $x_{2}\geq x_{1}$ and $s_{2}\succeq s_{1}$ we have $f(x_{2},s_{2})-f(x_{1},s_{2})\leq f(x_{2},s_{1})-f(x_{1},s_{1}).$ $f$ is said to have increasing differences if $-f$ has decreasing differences.

Assumption 5

$u(x,s)$* is jointly continuous. Further, it is concave and continuously differentiable in $x$, for each $s\in\mathcal{P}(X)$. In addition, $u(x,s)$ has decreasing differences in $(x,s)$.*

We now provide two classic examples of profit functions $u(x,s)$ that are commonly used in the literature. For these examples, we explicitly define the binary relation $\succeq$.

The first one is the capacity competition model described in Example 1. Recall that if the firm’s state is $x$, then its capacity is $\bar{q}(x)$. We assume that $\bar{q}$ is an increasing, continuously differentiable, concave, and bounded function. We also assume that the inverse demand function $P(\cdot)$ is decreasing and continuous. In this model,

[TABLE]

For the capacity competition model, we define $s_{2}\succeq s_{1}$ if and only if $\int\bar{q}(y)s_{2}(dy)\geq\int\bar{q}(y)s_{1}(dy)$. Since $\bar{q}$ is an increasing function, $\succeq$ agrees with $\succeq_{SD}$. It can be verified that $u$ satisfies the conditions of Assumption 5.

Our second example is a classic quality ladder model, where individual states represent the quality of a firm’s product (see, e.g., Pakes and McGuire (1994) and Ericson and Pakes (1995)). Consider a price competition under a logit demand system. There are $N$ consumers in the market. The utility of consumer $j$ from consuming the good produced by firm $i$ at period $t$ is given by

[TABLE]

where $\theta_{1}<1,\theta_{2}>0$, $p_{it}$ is the price of the good produced by firm $i$, $Y$ is the consumer’s income, $x_{it}$ is the quality of the good produced by firm $i$, and $\{v_{ijt}\}_{i,j,t}$ are i.i.d Gumbel random variables that represent unobserved characteristics for each consumer-good pair.

There are $m$ firms in the market and the marginal production cost is constant and the same across firms. There is a unique Nash equilibrium in pure strategies of the pricing game (see Caplin and Nalebuff (1991)). These equilibrium static prices determine the single-period profits. Now, the limiting profit function that we focus on can be obtained from the asymptotic regime in which the number of consumers $N$ and the number of firms $m$ grow to infinity at the same rate. The limiting profit function corresponds to a logit model of monopolistic competition given by:

[TABLE]

(see Besanko et al. (1990)). $\tilde{c}$ is a constant that depends on the limiting equilibrium price, the marginal production cost, the consumer’s income, and $\theta_{2}$. For the quality ladder model, we define $s_{2}\succeq s_{1}$ if and only if $\int(y+1)^{\theta_{1}}s_{2}(dx)\geq\int(y+1)^{\theta_{1}}s_{1}(dy)$. It is easy to see that $\succeq$ agrees with $\succeq_{SD}$. It can also be verified that $u$ satisfies the conditions of Assumption 5.

The proof of our uniqueness result for the capacity competition and quality ladder models consists of showing that Assumptions 4 and 5 imply Assumptions 1 and 2, and that $\succeq$ is a complete order. These are the conditions we use to show the existence of a unique MFE in Sections 3.1 and 3.2.

Specifically, similarly to Lemma 1, one can show that the concavity assumptions in Assumptions 4 and 5 imply that $G$ is single-valued. The assumption that $k(0)>0$ (see condition (i) in Assumption 4) is used to prevent the pathological case that the Dirac measure on the point [math] is an invariant distribution of $M_{s}$ which could violate $X$-ergodicity (see Section 3.1). In addition, condition (ii) in Assumption 4 is used to control the growth of firms, so that one can show that the state space remains compact. We believe our results hold with a milder version of this assumption. With this, the only remaining assumption that we need to show in order to prove the uniqueness of an MFE for our capacity competition and quality ladder models is Assumption 2(i). For this, we use the fact that the profit function has decreasing differences in the state $x$ and the population state $s$. This implies that firms invest less when the population state is higher (see Lemma 4). We use this fact to show the desired monotonicity of $Q$.

Our main result for dynamic capacity competition and dynamic quality ladder models is the following:

Theorem 5

Suppose that Assumptions 4 and 5 hold. Then there exists a unique MFE for the capacity competition and quality ladder models.

Under Assumptions 4 and 5 we can also derive comparative statics results for our capacity competition and quality ladder models. In particular, we show that an increase in the cost of a unit of investment decreases the unique MFE population state. Note that an increase in the investment cost decreases firms incentives to invest. However, a lower population state incentivizes the firms to invest more. As a consequence, our model does not have the properties of a supermodular game (e.g., Topkis (1979)). Despite this, relying on the uniqueness of an MFE and on Theorem 4 we are able to show that in fact the unique MFE decreases when the cost of a unit of investment increases.

We also derive comparative statics results regarding a change in a parameter that influences the profit function and a change in the discount factor. We show that if there is a parameter $c$ such that the marginal profit of the firms is decreasing in that parameter, then the unique MFE decreases in the parameter $c$. For example, in the quality ladder model, as the marginal cost of production goes up, the unique MFE decreases. In the capacity competition model, as the potential market size increases, the MFE increases. In addition, we show that an increase in the discount factor increases the unique MFE.

We note that all of our comparative statics results are with respect to the order $\preceq$ and not with respect to the usual stochastic dominance order as one would typically obtain using supermodularity arguments (e.g., Adlakha and Johari (2013)). We believe that these results provide helpful information because the order $\preceq$ relates to the single-period profit function, and therefore, MFE can be ordered in terms of firms’ payoffs. Further, $\preceq$ typically orders a variable of economic interest, such as the average capacity level in the capacity competition model or the average quality level in the quality ladder model.

Theorem 6

Suppose that Assumptions 4 and 5 hold. We denote by $s(e)$ the unique MFE when the parameter that influences the firms’ decisions is $e$.

(i) If the cost of a unit of investment increases, then the unique MFE decreases, i.e., $d_{2}\leq d_{1}$ implies $s(d_{2})\succeq s(d_{1})$.

(ii) Let $c\in I\subseteq\mathbb{R}$ be a parameter that influences the firms’ profit function. If the profit function $u(x,s,c)$ has decreasing differences in $(x,c)$, then the unique MFE decreases in $c$, i.e., $c_{1}\geq c_{2}$ implies $s(c_{2})\succeq s(c_{1})$.

(iii) Assume that $u(x,s)$ is increasing in $x$. If the discount factor $\beta$ increases, then the unique MFE $s(\beta)$ increases, i.e., $\beta_{2}\geq\beta_{1}$ implies $s(\beta_{2})\succeq s(\beta_{1})$.

4.2 Dynamic Advertising Competition Models

In this section we consider dynamic advertising competition models. In these models, firms’ states correspond to customer goodwill or market share. In each period, the firms decide on their advertising expenditures $a$. The probability that the next period’s customer goodwill is higher increases when the firms spend more on advertising. The firms’ payoff functions depend on their own spending on advertising, on their own state, on the other firms’ states, and on the other firms’ spending on advertising. Thus, a firm’s payoff function depends on the other firms’ dynamic actions (in Sections 2.2 and A.1 we extend the model and the results presented in Sections 2 and 3 to the case in which each player’s payoff function depends on the other players’ actions). Variants of dynamic models with this structure have been studied in the operations research literature in contexts other than advertising (for example, see Hall and Porteus (2000)). We now describe our specific model.181818Our model is a mean field version of the dynamic advertising model presented in Heyman and Sobel (2004) and in Section 4.3 in Olsen and Parker (2014)).

States. The state of firm $i$ at time $t$ is denoted by $x_{i,t}\in X$ where $X=\mathbb{R}_{+}$. The state of a firm $x_{i,t}\in X$ represents the customer goodwill.

Actions. At each time $t$, firm $i$ chooses an amount of money to spend on advertising $a_{i,t}\in A=[1,\bar{a}]$ where $\bar{a}>1$.

States’ dynamics. When the firm spends more on advertising, the customer goodwill increases. The customer goodwill depreciates over time at rate $0<\delta<1$. If firm $i$’s state at time $t-1$ is $x_{i,t-1}$, the firm takes an action $a_{i,t-1}$ at time $t-1$, and $\zeta_{i,t}$ is firm $i$’s realized idiosyncratic random shock at time $t$, then firm $i$’s state in the next period is given by

[TABLE]

We assume that $\zeta$ takes positive values $0<\zeta_{1}<\ldots<\zeta_{n}$. To ensure compactness we also assume that $(1-\delta)\zeta_{n}<1$ (see Section 4.1). We slightly modify the transition dynamics from Section 4.1 to remain consistent with the models used in the papers that motivate this section.

Payoff. When a firm chooses to spend $a\in A$ on advertising, the firm’s state is $x\in X$, and the population action-state profile is $s\in\mathcal{P}(X\times A)$, then the firm’s single-period payoff function is given by

[TABLE]

where $\frac{(x+a)^{\gamma_{1}}}{(\int(x^{\prime}+a^{\prime})s(d(x^{\prime},a^{\prime})))^{\gamma_{2}}}$ is the expected demand, $r>0$ is the price, and $0<\gamma_{1}<1$, $0<\gamma_{2}<1$ are parameters. The expected demand is increasing in the firm’s current advertising expenditure and in the firm’s current state, and is decreasing in the other firms’ advertising expenditures and the other firms’ states.

We define a complete binary relation $\succeq$ on $\mathcal{P}(X\times A)$, by $s_{1}\succeq s_{2}$ if and only if $(\int(x^{\prime}+a^{\prime})s_{1}(d(x^{\prime},a^{\prime})))^{\gamma_{2}}\geq(\int(x^{\prime}+a^{\prime})s_{2}(d(x^{\prime},a^{\prime})))^{\gamma_{2}}$. Clearly, $\succeq$ agrees with $\succeq_{SD}$ (see Section 3.1). We can also derive comparative statics results for the dynamic advertising model. For example, using similar arguments to the arguments in Section 4.1 we can show that when the discount factor $\beta$ increases, then the unique MFE increases in the following sense: if $\beta_{2}>\beta_{1}$, then $s(\beta_{2})\succeq s(\beta_{1})$ where $s(\beta)$ is the unique MFE under discount factor $\beta$. We also show that the unique MFE increases when the market price $r$ increases.

Theorem 7

(i) The dynamic advertising competition model has a unique MFE.

(ii) Let $s(\beta)$ be the unique MFE under the discount factor $\beta$. Then $\beta_{2}>\beta_{1}$ implies $s(\beta_{2})\succeq s(\beta_{1})$.

(iii) Let $s(r)$ be the unique MFE under the price $r$. Then $r_{2}>r_{1}$ implies $s(r_{2})\succeq s(r_{1})$.

4.3 A Dynamic Reputation Model

In this section we consider a dynamic reputation model. Motivated by the proliferation of online markets, reputation models and the design of reputation systems have recently been widely studied in the operations and management science literature.191919For example, see Dellarocas (2003), Aperjis and Johari (2010), Bolton et al. (2013), Papanastasiou et al. (2017), and Besbes and Scarsini (2018). These systems can mitigate the mistrust between buyers and sellers participating in the marketplace (see Tadelis (2016)). Further, online markets typically consist of many small sellers, and therefore, assuming an MFE limit is natural.

We study a dynamic reputation model in which sellers improve their reputation level over time. The state of each seller consists of the average review given to her in the past history and the total number of reviews she has received.202020Typically, review systems report simple averages; the number of reviews may also be relevant as it may signal more sales and more experience from a seller. In each period, each seller receives a review from buyers.212121 This assumption is made only for simplicity. We can also assume that reviews arrive according to a Poisson process. A seller’s ranking is a simple average of her past reviews. Sellers invest in their products in order to improve their reviews over time. For example, Airbnb hosts can invest in cleaning their apartments, and sellers on eBay can invest in their packaging. Higher investments increase the probability of receiving a good review. Sellers’ payoffs depend on their rankings and on the number of reviews they receive as well as on the other sellers’ rankings and number of reviews. Each seller’s payoff function increases in her ranking and in her number of reviews and decreases in the other sellers’ rankings and number of reviews. This can capture, for example, the fact that a seller with a higher ranking can charge a higher price or garner more demand.

The dynamic reputation model we consider in this section assumes that sellers arrive and depart over time. We make this modeling choice because of its realistic appeal, and to ensure that the number of reviews does not tend to infinity. Because we study a stationary setting, we assume that the sellers’ rates of arrival and departure balance, so that the market size remains constant over time (in expectation). After each review, a seller departs the market and never returns with probability $1-\beta$ where $0<\beta<1$. For each seller $i$ that departs, a new seller immediately arrives. We assign the new seller the same label $i$, and a [math] ranking, and [math] reviews. Under this assumption, it is straightforward to show that the seller’s decision problem is the same stationary, infinite horizon, expected discounted reward maximization problem that we introduced in Section 2, where the discount factor is the probability of remaining in the market.222222For example, Iyer et al. (2014) provide a similar regenerative model of arrivals and departures.

We now describe the dynamic reputation model in more detail.

States. The state of seller $i$ at time $t$ is denoted by $x_{i,t}=(x_{i,t,1},x_{i,t,2})\in X_{1}\times X_{2}=X$. $x_{i,t,1}$ represents seller $i$’s average numerical review rating up to time $t$. We call $x_{i,t,1}$ seller $i$’s ranking at period $t$. $x_{i,t,2}$ represents the number of reviews seller $i$ has received up to period $t$.

Actions. At each time $t$, seller $i$ chooses an action $a_{i,t}\in A=[0,\bar{a}]$ in order to improve her ranking. The action changes the seller’s state in a stochastic fashion.

States’ dynamics. If seller $i$’s state at time $t-1$ is $x_{i,t-1}$, the seller takes an action $a_{i,t-1}$ at time $t-1$, and $\zeta_{i,t}$ is seller $i$’s realized idiosyncratic random shock at time $t$, then seller $i$’s state in the next period is given by:

[TABLE]

where $k:A\rightarrow\mathbb{R}$ is a strictly increasing and strictly concave function that determines the impact of the seller’s investment on the next period’s review. The next period’s numerical review, $k(a)+\zeta$, is assumed to be non-negative.232323In order to simplify the analysis and preserve Assumption 1, we assume that the numerical value of a review $k(a)+\zeta$ can be any non-negative number and not a discrete number. In a model where $k(a)+\zeta$ is discrete our results still hold as long as the optimal strategy is single-valued. $M_{1}>0$ is the upper bound on the sellers’ ranking and $M_{2}>0$ is the upper bound on the sellers’ number of reviews. The latter are useful to keep the state space compact. The first term in the dynamics represents the simple average of the numerical reviews received so far, while the second term represents the total number of reviews. Similarly to the previous models, the random shocks represent uncertainty in the review process.

Payoff. The cost of a unit of investment is $d>0$. When the seller’s ranking is $x_{1}$, the seller’s number of reviews is $x_{2}$, the seller chooses an action $a\in A$, and the population state is $s\in\mathcal{P}(X)$, then the seller’s single-period payoff is given by

[TABLE]

where $\nu$ is increasing in $x_{1}$ and $x_{2}$, concave, continuously differentiable in $x_{1}$, and positive. The functional form resembles the logit model studied in Section 4.1.

The cost of a unit of investment can be seen as a lever that a platform may impact by design. In particular, a platform can reduce the cost of a unit of investment for the sellers by introducing tools to improve the buyers’ experience of using the sellers’ products. For example, an e-commerce platform could help facilitating logistics for its sellers, and a rental sharing platform could help its hosts connecting cleaning services.

We define a complete and transitive binary relation $\succeq$ on $\mathcal{P}(X)$ by $s_{1}\succeq s_{2}$ if and only if $\int\nu(x_{1},x_{2})s_{1}(d(x_{1},x_{2}))\geq\int\nu(x_{1},x_{2})s_{2}(d(x_{1},x_{2}))$. It is easy to see that $\succeq$ agrees with $\succeq_{SD}$ (see Section 3.1).

We use Theorem 2 to prove that the dynamic reputation model admits a unique MFE.242424For this model we are able to show the monotonicity of the kernel $Q$ with respect to $x_{1}$ but not with respect to $x_{2}$. We also show that when the platform reduces the cost of a unit of investment then the MFE increases.

Theorem 8

(i) The dynamic reputation model has a unique MFE.

(ii) Let $s(d)$ be the unique MFE under the unit of investment cost $d$. Then $d_{2}\geq d_{1}$ implies $s(d_{2})\preceq s(d_{1})$.

5 Heterogeneous Agent Macroeconomic Models

In this section we consider heterogeneous agent macro models. In these models, there is a continuum of agents facing idiosyncratic risks only (and no aggregate risks). The heterogeneous agents make decisions given certain market prices (in Aiyagari (1994), for example, the market prices are the interest rate and the wage rate). The market prices are determined by the aggregate decisions of all the agents in the market. We consider a setting similar to the one presented in Acemoglu and Jensen (2015). We note that this setting encompasses many important models in the economics literature. Examples include Bewley-Aiyagari models (see Bewley (1986), and Aiyagari (1994)), and models of industry equilibrium (see Hopenhayn (1992)). While Acemoglu and Jensen (2015) derive important existence and comparative statics results for these models, to the best of our knowledge there are no general uniqueness results. In this Section we show that if the agents’ strategy is decreasing in the aggregator (in the sense of Acemoglu and Jensen (2015)), there exists a unique equilibrium.

We now describe our specific model.

States. The state of player $i$ at time $t$ is denoted by $x_{i,t}=(x_{i,t,1},x_{i,t,2})\in X_{1}\times X_{2}=X$ where $X_{1}\subseteq\mathbb{R}$ and $X_{2}\subseteq\mathbb{R}^{n-1}$. For example, in Bewley models $x_{i,t,1}$ typically represents agent $i$’s savings at period $t$ and $x_{2}$ represents agent $i$’s income or labor productivity at period $t$ (in this case $n=2$).

Actions. At each time $t$, player $i$ chooses an action $a_{i,t}\in\Gamma(x_{i,t})\subset\mathbb{R}$.

States’ dynamics. The state of a player evolves in a Markovian fashion. If player $i$’s state at time $t-1$ is $x_{i,t-1}$, player $i$ takes an action $a_{i,t-1}$ at time $t-1$, and $\zeta_{i,t}$ is player $i$’s realized idiosyncratic random shock at time $t$, then player $i$’s state in the next period is given by

[TABLE]

where $m:X_{2}\times E\rightarrow X_{2}$ is a continuous function. For example, in Bewley models, in each period agents choose how much to save for future consumption and how much to consume in the current period. The agents’ labor productivity evolves exogenously and the labor productivity function $m$ determines the next period’s labor productivity given the current labor productivity. So if an agent chooses to save $a$, $\zeta$ is the realized random shock, and her current labor productivity is $x_{2}$, then the agent’s next period state (savings-labor productivity pair) is given by $(a,m(x_{2},\zeta))$.

Payoff. As in Acemoglu and Jensen (2015), we assume that the payoff function depends on the population state through an aggregator. That is, if the population state is $s$, then the aggregator is given by $H(s)$ where $H:\mathcal{P}(X)\rightarrow\mathbb{R}$ is a continuous function. If the aggregator is $H(s)$, the player’s state is $x\in X$, and the player takes an action $a\in\Gamma(x)$, then the player’s single-period payoff function is given by $\tilde{\pi}(x,a,H(s))$.

We define a complete and transitive binary relation $\succeq$ on $\mathcal{P}(X)$ by $s_{1}\succeq s_{2}$ if and only if $H(s_{1})\geq H(s_{2})$. We assume that $\succeq$ agrees with $\succeq_{SD}$. This assumption holds in most of the heterogeneous agent macro models, where $H$ is usually assumed to be increasing with respect to first order stochastic dominance (see Acemoglu and Jensen (2015)).

Note that under the states’ dynamics defined above, and assuming that $g(x,s)=\tilde{g}(x,H(s))$ is the optimal stationary strategy, the transition kernel $Q$ is given by

[TABLE]

where $B_{1}\times B_{2}\in\mathcal{B}(X_{1}\times X_{2})$.

We show that the model has a unique MFE if the optimal strategy is decreasing in the aggregator, i.e., if $H(s_{2})\geq H(s_{1})$ implies $\tilde{g}(x_{1},x_{2},H(s_{2}))\leq\tilde{g}(x_{1},x_{2},H(s_{1}))$, $Q$ is $X$-ergodic, and $\tilde{g}$ is increasing in $x_{1}$. We note that we cannot apply Theorem 1 to this model, since in most applications the optimal stationary strategy $\tilde{g}$ is not increasing in $x_{2}$, and thus $Q$ may not be increasing in $x_{2}$. However, in most applications (for example, all the applications discussed in Acemoglu and Jensen (2015)) $\tilde{g}$ is increasing in $x_{1}$. Thus, we can use Theorem 2 to show that the heterogeneous agent macro model has a unique MFE under the conditions stated above.252525 Note that an MFE is usually called a stationary equilibrium in the economics literature (e.g., Acemoglu and Jensen (2015)).

Corollary 3

Assume that $G$ is single-valued, $Q$ is $X$-ergodic, and $\tilde{g}$ is increasing in $x_{1}$ and decreasing in the aggregator. Then the heterogeneous agent macro model has a unique MFE.

In most applications, the payoff function $\tilde{\pi}$ has increasing differences in $(x_{1},a)$ which ensures that $\tilde{g}$ is increasing in $x_{1}$. The condition that $Q$ is $X$-ergodic also usually holds in applications. For example, Aiyagari (1994) proves that $Q$ is $X$-ergodic in his model. Thus, in many applications, in order to ensure uniqueness, one only needs to check that $\tilde{g}$ is decreasing in the aggregator. In the next section we illustrate this in a Bewley-type model introduced in Aiyagari (1994).

A Bewley-Aiyagari Model. Bewley models are widely studied and used in the modern macroeconomics literature (for a survey see Heathcote et al. (2009)). As previously mentioned, in Bewley models agents receive a state-dependent income in each period and they solve an infinite horizon consumption-savings problem; that is, the agents must decide how much to save and how much to consume in each period. The agents can transfer assets from one period to another only by investing in a risk-free bond, and have some borrowing limit. Aiyagari (1994) extends the Bewley model to a general equilibrium model with production. We now describe the model of Aiyagari (1994) in the setting of a mean field game.

In a Bewley-Aiyagari model, $x_{1}$ represents the agents’ savings and $x_{2}$ represents the agents’ labor productivity. $m(x_{2},\zeta)$ represents the labor productivity function. That is, if the current labor productivity is $x_{2}$ then the next period’s labor productivity is given by $m(x_{2},\zeta_{j})$ with probability $p_{j}$. If the agents’ labor productivity is $x_{2}$ then their income is given by $wx_{2}$ where $w>0$ is the wage rate. The agents’ savings rate of return is $R>0$.

In each period $t$, the agents choose their next period’s savings level $a\in\Gamma(x_{1},x_{2})$ where $\Gamma(x_{1},x_{2})=[-\underline{b},\min\{Rx_{1}+wx_{2},\bar{b}\}]$, and consume $c=Rx_{1}+wx_{2}-a$. That is, the agents’ savings are lower than their cash-on-hand $Rx_{1}+wx_{2}$ and higher than the borrowing constraint $\underline{b}\geq 0$. $\bar{b}$ is an upper bound on savings that ensures compactness.

The wage rate and the interest rate are determined in general equilibrium. There is a representative firm with a production function $F(K,N)$ that is homogeneous of degree one. $N$ is the labor supply and $K$ is the capital. We assume that $F$ is twice continuously differentiable, strictly concave, and strictly increasing. The first order conditions of the firm’s maximization problem yield262626The firm’s maximization problem is given by $\max_{K,N}F(K,N)-(R-1+\delta)K-wN$. For more details see, for example, Acemoglu and Jensen (2015) and Light (2020). $F_{k}(K,N)=R+\delta-1$ and $F_{N}(K,N)=w$ where $\delta>0$ is the depreciation rate and $F_{i}(K,N)$ denotes the partial derivative of $F$ with respect to $i=K,N$. A standard argument272727Since $F$ is homogeneous of degree one we have $F(K,1)=KF_{K}(K,1)+F_{N}(K,1)$. Using the first order conditions we have $f(K)=Kf^{\prime}(K)+w$. shows that $R=f^{\prime}(K)-\delta+1$ and $w=f(K)-f^{\prime}(K)K$ where $F(K,1)=f(K)$.

In equilibrium we have $\int_{X}x_{1}s(d(x_{1},x_{2}))=K$ where $s$ is an invariant savings-labor productivities distribution. That is, the aggregate supply of savings equals the total capital. We define $H(s)=\int_{X}x_{1}s(d(x_{1},x_{2}))$. It is easy to see that $\succeq$ agrees with $\succeq_{SD}$ (see Section 3.1).

The agents’ utility from consumption is given by a utility function $u$ which is assumed to be strictly concave and strictly increasing. If the agents choose to save $a$ then their consumption in the current period is $Rx_{1}+wx_{2}-a$. Thus, using the equilibrium conditions $R=f^{\prime}(H(s))-\delta+1$ and $w=f(H(s))-f^{\prime}(H(s))H(s)$, in a Bewley-Aiyagari model the payoff function $\tilde{\pi}$ is given by

[TABLE]

It is easy to establish that $G$ is single-valued and that Assumption 1 holds. Thus, the existence of an equilibrium in a Bewley-Aiyagari model follows from Theorem 3.282828Some of the previous existence results rely on the $X$-ergodicity of $Q$ (e,g., Acikgoz (2018)) or on monotonicity arguments (e.g., Acemoglu and Jensen (2015)). The proof presented in this paper shows that these conditions are not needed in order to establish the existence of an equilibrium.

Under mild technical conditions on the utility function (for example, if $u$ is bounded or if $u$ belongs to the constant relative risk aversion class), the $X$-ergodicity of $Q$ can be proven in a similar manner to Acikgoz (2018). It can be established also that the next period’s savings are increasing in the current period’s savings, i.e., $\tilde{g}$ is increasing in $x_{1}$. Thus, to prove the uniqueness of an MFE in a Bewley-Aiyagari model, one needs to prove that $\tilde{g}$ is decreasing in the aggregator $H(s)$. In a recent paper, Light (2020) proves the uniqueness of an MFE for the special case that the agents’ utility function is in the CRRA (constant relative risk aversion) class with a relative risk aversion coefficient that is less than or equal to one, and the production function’s elasticity of substitution is bounded below by $1$. Under these assumptions, we can use the results in Light (2020) to show that $\tilde{g}$ is decreasing in the aggregator $H(s)$. Then, we can use Corollary 3 to prove the uniqueness of an MFE. As a note for future research, our results suggest that the result in Light (2020) could be generalized, weakening the conditions on the relative risk aversion and on the production function. With this, we believe our approach could be used to show uniqueness for a broader class of heterogeneous agent macro models. Finally, we note that the uniqueness result in Hopenhayn (1992) can be obtained from Corollary 3 also. For the sake of brevity we omit the details here.

6 Conclusions

This paper studies the existence and uniqueness of an MFE in stochastic games with a general state space. We provide conditions that ensure the uniqueness of an MFE. We also prove that there exists an MFE under continuity and concavity conditions on the primitives of the model. We show that a general class of dynamic oligopoly models satisfies these conditions, and thus, these models have a unique MFE. Furthermore, we prove the existence of a unique MFE in heterogeneous agent macro models. We also derive general comparative statics results regarding the MFE and apply them to dynamic oligopoly models.

We believe that our results can be applied to other models in operations research and economics. For example, in order to analyze market design problems in online platforms, like in the reputation model we studied, it is natural to assume a large-scale MFE limit. Typical questions of interest in these contexts involve the market’s response to platforms’ market design choices. Hence, knowing that this response is unique and that one can predict its directional changes could significantly strengthen the analysis of these platforms.

We believe our results can be extended to prove the uniqueness of an invariant distribution for a general Markov chain where the next period’s state depends on the previous state and on the previous state’s distribution. These Markov chains can capture other interesting applications in operations research, such as strategic queueing systems. We leave this direction for future work.

Appendix A Appendix: Extensions

In this section we extend the model presented in Section 2. In Section A.1 we study a model where the players are coupled through actions and in Section A.2 we study a model where the players are ex-ante heterogeneous.

A.1 Coupling Through Actions

In this section we consider a model where the transition function and the payoff function of each player depend on both the states and the actions of all other players. The model is the same as the original model in Section 2 except that now the probability measure $s$ describes the joint distribution of players over actions and states and not only over states, that is, $s\in\mathcal{P}(X\times A)$. Thus, the transition function $w(x,a,s,\zeta)$ and the payoff function $\pi(x,a,s)$ depend on the joint distribution over state-action pairs $s\in\mathcal{P}(X\times A)$. We refer to $s\in\mathcal{P}(X\times A)$ as the population action-state profile and to the marginal distribution of the population action-state profile over $X$ as the population state (i.e., the population state’s distribution is described by the probability measure $s\left(\cdot,A\right)$).

An MFE is defined similarly to the definition in Section 2. In an MFE, every player conjectures that $s$ is the fixed long run population action-state profile, and plays according to a stationary strategy $g$. If every player plays according to the strategy $g$ when the population action-state profile is $s$, then $s$ constitutes an invariant distribution.

Given the stationary strategy $g$, $s\in\mathcal{P}(X\times A)$ is an invariant distribution if

[TABLE]

for all $B\times D\in\mathcal{B}(X\times A)$ where $Q(x,s,B)=\Pr(w(x,s,g(x,s),\zeta)\in B)$ and292929 Note that $\overline{Q}$ is a Markov kernel on $X\times A.$

[TABLE]

To see that Equation (3) holds, first assume that $X$ and $A$ are discrete sets. The joint probability mass function of a stationary distribution $s\left(y,a\right)$ is given by

[TABLE]

where $\overline{s}(a|y)$ is the probability of playing the action $a\in A$ given that the state is $y\in X$. Since the players use the pure strategy $g$ we have $\overline{s}(a|y)=1_{\{a\}}(g(y,s))$. Thus,

[TABLE]

In addition, since $s$ is invariant, the marginal distribution $s\left(\cdot,A\right)$ must satisfy $s(y,A)=\sum_{x\in X}s(x,A)Q(x,s,y)$. Thus,

[TABLE]

Similarly, Equation (3) holds in the general state space.

If $A$ is compact then $X\times A$ is compact, and thus, $\mathcal{P}\left(X\times A\right)$ is compact in the weak topology. Similar arguments to the arguments in the proof of Theorem 3 show that the operator $\overline{\Phi}:\mathcal{P}(X)\rightarrow\mathcal{P}(X)$ defined by

[TABLE]

is continuous (see more details in the proof of Theorem 9). Thus, as in the proof of Theorem 3, we can apply Schauder-Tychonoff’s fixed point theorem to prove that $\overline{\Phi}$ has a fixed point.

The uniqueness result holds under the same conditions as the conditions in Theorem 1 except that the assumptions on the Markov kernel $Q$ in Assumption 2 part (i) are assumed on the Markov kernel $\overline{Q}$. The proof of Theorem 9 part (i) is essentially the same as the proof of Theorem 1. Similarly, Theorem 9 part (iii) holds when the assumptions on the Markov kernel $Q$ are assumed on the Markov kernel $\overline{Q}$.

We summarize the discussion in the following Theorem.

Theorem 9

Consider the model described in this section. Suppose that the action set $A$ is compact.

(i) Under the assumptions of Theorem 1 where $Q$ is replaced by $\overline{Q}$ the MFE is unique.

(ii) Under the assumptions of Theorem 3 there exists an MFE.

(iii) Let $(I,\succeq_{I})$ be a partially ordered set. Assume that $\overline{Q}$ is increasing in $e$ on $I$.Then, under the assumptions of part (i), the unique MFE $s(e)$ is increasing in the following sense: $e_{2}\succeq_{I}e_{1}$ implies $s(e_{2})\succeq s(e_{1})$.303030 Recall that we say that $\overline{Q}$ is increasing in $e$ if $\overline{Q}(x,s,e_{2},\cdot)\succeq_{SD}\overline{Q}(x,s,e_{1},\cdot)$ for all $x$, $s$, and all $e_{2},e_{1}\in I$ such that $e_{2}\succeq_{I}e_{1}$. Note that the orders $\succeq_{SD}$ and $\succeq$ are on measures over state-action pairs.

The assumptions on $\overline{Q}$ that are needed in order to guarantee the uniqueness of an MFE can be verified in a similar manner to the assumptions on $Q$. In particular, in some models it is enough to show that the policy function $g(x,s)$ is increasing in the state $x$ and decreasing in the population action-state profile state $s$ which is a natural property in many dynamic oligopoly models (see Section 4). In Section 4.2 we prove that the policy function $g(x,s)$ is increasing in $x$ and decreasing in $s$ in a dynamic advertising model where each player’s payoff function depends on the other players’ actions, and we use Theorem 9 to prove that the model has a unique MFE.

A.2 Ex-ante Heterogeneity

In this section we study a mean field model with ex-ante heterogeneous players. We assume that the players are heterogeneous in their payoff functions and in their transition functions. Assume that before the time horizon, each player has a type $\theta\in\Theta$, where $\Theta$ is a finite partially ordered set. Each player’s type is fixed throughout the horizon. Let $\Upsilon$ be the probability mass function over the type space; $\Upsilon(\theta)$ is the mass of players whose type is $\theta\in\Theta$, which is common knowledge. Adding the argument $\theta\in\Theta$ to the functions defined in Section 2, we can modify the definitions of Section 2 to include the ex-ante heterogeneity of the players. In particular, we denote by $w(x,a,s,\zeta,\theta)$ the transition function of type $\theta\in\Theta$ and by $\pi(x,a,s,\theta)$ the payoff function of type $\theta\in\Theta$.

Let $X_{h}=X\times\Theta$ be an extended state space for the mean field model with ex-ante heterogeneous players. If a player’s extended state is $x_{h}=(x,\theta)\in X_{h}$ then the player’s state is $x$ and the player’s type is $\theta$. Let $s_{h}$ be the population state over the extended state space, i.e., $s_{h}\in\mathcal{P}(X\times\Theta)$.

For a probability measure $s_{h}\in\mathcal{P}(X\times\Theta)$, define a probability measure $S\left(s_{h}\right)\in\mathcal{P}(X)$ by

[TABLE]

for all $B\in\mathcal{B}(X)$. That is, $S(s_{h})$ is the marginal distribution of $s_{h}$ that describes the population state.

For the model with ex-ante heterogeneous players we define the payoff function $\pi_{h}\left(x_{h},a,s_{h}\right)=\pi(x,a,S\left(s_{h}\right),\theta)$. Note that we consider a model where each player’s payoff function depends on the other players’ states (the population state) and not on the other players’ types. This seems reasonable in most applications, as types usually represent ex-ante heterogeneity in the payoff functions, discount factors, etc. We now define the transition function.

For a fixed extended population state $s_{h}\in\mathcal{P}(X\times\Theta)$ and a strategy $g(x,S\left(s_{h}\right),\theta)$, the probability that player $i$’s next period’s state will lie in a set $B\times D\in\mathcal{B}(X)\times 2^{\Theta}$, given that her current state is $x_{h}=(x,\theta)\in X_{h}$, her type is $\theta$, and she takes the action $a=g(x,S\left(s_{h}\right),\theta)$, is:

[TABLE]

These definitions map the payoff function and transition function in the model with ex-ante heterogeneous players to the model with ex-ante homogeneous players that we considered in Section 2. Thus, all the results in this paper hold also in the case of ex-ante heterogeneity where the assumptions that we made on $\pi$, $w$ and $Q$ are now assumed on $\pi_{h}$, $w_{h}$ and $Q_{h}$. Thus, all our results can easily be extended to the case of ex-ante heterogeneous players. Note that in this model, players of different types may play different MFE strategies. We now provide more details.

Similarly to Section 2, in an MFE every player plays according to the strategy $g$ when the extended population state is $s_{h}$ and $s_{h}$ constitutes an invariant distribution given the strategy $g$. That is, $s_{h}$ satisfies

[TABLE]

for all $B\times D\in\mathcal{B}(X)\times 2^{\Theta}$.

The following theorem follows immediately from the results in the main text when $Q$ is replaced by $Q_{h}$. Note that $X_{h}=X\times\Theta$ is a product space so we can use Theorem 2 instead of Theorem 1 to prove the uniqueness of an MFE.

Theorem 10

Consider the model described in this section.

(i) Under the assumptions of Theorem 2 (with the state space $X\times\Theta$) where $Q$ is replaced by $Q_{h}$, the MFE is unique.

(ii) Under the assumptions of Theorem 3 there exists an MFE.

(iii) Let $(I,\succeq_{I})$ be a partially ordered set. Assume that $Q_{h}$ is increasing in $e$ on $I$. Then, under the assumptions of part (i), the unique MFE $s_{h}(e)$ is increasing in the following sense: $e_{2}\succeq_{I}e_{1}$ implies $s_{h}(e_{2})\succeq s_{h}(e_{1})$.

We define the $X$-transition function of a type $\theta$ player by

[TABLE]

for all $B\in\mathcal{B}(X)$. As discussed in Section 3.1, the key assumption that implies the uniqueness of an MFE is related to the transition function’s monotonicity properties. In particular, the assumption is that the transition function is increasing in the players’ own states and decreasing in the extended population state. In the case of ex-ante heterogeneity, the next Lemma shows that if the transition function of each player $Q_{\theta}$ is increasing in $x$ and decreasing in $s_{h}$ for every type $\theta$ then $Q_{h}$ is increasing in $x$ and decreasing in $s_{h}$ with respect to $x$. This fact is useful for applications when we want to verify the monotonicity conditions needed in Theorem 10 part (i) that imply the uniqueness of an MFE.

Lemma 2

Assume that $Q_{\theta}$ is increasing in $x$ and decreasing in $s_{h}$ for every type $\theta$. Then $Q_{h}$ is increasing in $x$ and decreasing in $s_{h}$ with respect to $x$.

Appendix B Appendix: Proofs

B.1 Uniqueness: Proof of Theorem 2

Proof of Theorem 2. Assume without loss of generality that $Q$ is increasing in $x_{1}$ and decreasing in $s$ with respect to $x_{1}$.

For $s_{1},s_{2}\in\mathcal{P}(X)$ we write $s_{1}\succeq_{SD,X_{1}}s_{2}$ if for all functions $f:X_{1}\times X_{2}\rightarrow\mathbb{R}$ that are increasing in the first argument (i.e., $x_{1}^{\prime}\geq x_{1}$ implies that $f(x_{1}^{\prime},x_{2})\geq f(x_{1},x_{2})$ for all $x_{2}\in X$) we have

[TABLE]

We note that if $\succeq$ agrees with $\succeq_{SD}$, then $\succeq$ agrees with $\succeq_{SD,X_{1}}$ (recall that $s_{2}\succeq_{SD}s_{1}$ if the last inequality holds for every increasing function $f:X_{1}\times X_{2}\rightarrow\mathbb{R}$).

Let $f:X_{1}\times X_{2}\rightarrow\mathbb{R}$ be increasing in the first argument, $\theta_{1},\theta_{2}\in\mathcal{P}(X)$ and assume that $\theta_{1}\succeq_{SD,X_{1}}\theta_{2}$. Let $s_{1},s_{2}$ be two MFEs such that $s_{2}\succeq s_{1}$. We have

[TABLE]

Thus, $M_{s_{1}}\theta_{1}\succeq_{SD,X_{1}}M_{s_{2}}\theta_{2}$. The first inequality follows from the facts that $f$ is increasing in the first argument, $Q$ is increasing in $x_{1}$, and $\theta_{1}\succeq_{SD,X_{1}}\theta_{2}$. The second inequality follows from the fact that $Q$ is decreasing in $s$ with respect to $x_{1}$.

We conclude that $M_{s_{1}}^{n}\theta_{1}\succeq_{SD,X_{1}}M_{s_{2}}^{n}\theta_{2}$ for all $n\in\mathbb{N}$. $Q$ being $X$-ergodic implies that $M_{s_{i}}^{n}\theta_{i}$ converges weakly to $\mu_{s_{i}}=s_{i}$. Since $\succeq_{SD,X_{1}}$ is a closed order, we have $s_{1}\succeq_{SD,X_{1}}s_{2}$ which implies that $s_{1}\succeq s_{2}$. The rest of the proof is the same as the proof of Theorem 1.

B.2 Existence: Proofs of Theorem 3 and Lemma 1

We first introduce preliminary notation and results.

Let $B(X\times\mathcal{P}(X))$ be the space of all bounded functions on $X\times\mathcal{P}(X)$. Define the operator $T:B(X\times\mathcal{P}(X))\rightarrow B(X\times\mathcal{P}(X))$ by

[TABLE]

where

[TABLE]

The operator $T$ is called the Bellman operator.

Lemma 3

The optimal strategy correspondence $G(x,s)$ is non-empty, compact-valued and upper hemicontinuous.

Proof. Assume that $f\in B(X\times\mathcal{P}(X))$ is (jointly) continuous. Then for each $\zeta\in E$, $f(w(x,a,s,\zeta),s)$ is continuous as the composition of continuous functions. Thus, $h(x,a,s,f)$ is continuous as the sum of continuous functions. Since $\Gamma(x)$ is continuous, the maximum theorem (see Theorem 17.31 in Aliprantis and Border (2006)) implies that $Tf(x,s)$ is jointly continuous.

We conclude that for all $n=1,2,3\ldots$, $T^{n}f$ is continuous. Under Assumption 1, standard dynamic programming arguments (see Bertsekas and Shreve (1978)) show that $T^{n}f$ converges to $V$ uniformly. Since the set of continuous functions is closed under uniform convergence, $V$ is continuous. Thus, $h(x,a,s,V)$ is continuous. From the maximum theorem, $G(x,s)$ is non-empty, compact-valued and upper hemicontinuous.

We say that $k_{n}:X\rightarrow\mathbb{R}$ converges continuously to $k$ if $k_{n}(x_{n})\rightarrow k(x)$ whenever $x_{n}\rightarrow x$. The following Proposition is a special case of Theorem 3.3 in Serfozo (1982).

Proposition 1

Assume that $k_{n}:X\rightarrow\mathbb{R}$ is a uniformly bounded sequence of functions. If $k_{n}:X\rightarrow\mathbb{R}$ converges continuously to $k$ and $s_{n}$ converges weakly to $s$ then

[TABLE]

In order to establish the existence of an MFE, we will use the following Proposition (see Corollary 17.56 in Aliprantis and Border (2006)).

Proposition 2

(Schauder-Tychonoff) Let $K$ be a nonempty, compact, convex subset of a locally convex Hausdorff space, and let $f:K\rightarrow K$ be a continuous function. Then the set of fixed points of $f$ is compact and nonempty.

Proof of Theorem 3. Let $g(x,s)=G(x,s)$ be the unique optimal stationary strategy. From Lemma 3, $g$ is continuous.

Consider the operator $\Phi:\mathcal{P}(X)\rightarrow\mathcal{P}(X)$ defined by

[TABLE]

If $s$ is a fixed point of $\Phi$ then $s$ is an MFE. Since $X$ is compact $\mathcal{P}(X)$ is compact (i.e., compact in the weak topology, see Aliprantis and Border (2006)). Clearly $\mathcal{P}(X)$ is convex. $\mathcal{P}(X)$ endowed with the weak topology is a locally convex Hausdorff space. If $\Phi$ is continuous, we can apply Schauder-Tychonoff’s fixed point theorem to prove that $\Phi$ has a fixed point. We now show that $\Phi$ is continuous.

First, note that for every bounded and measurable function $f:X\rightarrow\mathbb{R}$ and for every $s\in\mathcal{P}(X)$ we have

[TABLE]

To see this, first assume that $f=1_{B}$ where $1_{B}$ is the indicator function of $B\in\mathcal{B}(X)$. Then

[TABLE]

A standard argument shows that (4) holds for every bounded and measurable function $f$.

Assume that $s_{n}$ converges weakly to $s$. Let $f:X\rightarrow\mathbb{R}$ be a continuous and bounded function. Since $w$ is jointly continuous and $g$ is continuous(see Lemma 3), we have

[TABLE]

whenever $x_{n}\rightarrow x$. Let $k_{n}(x):=\sum_{j=1}^{n}p_{j}f(w(x,g(x,s_{n}),s_{n},\zeta_{j})$ and

$k(x):=\sum_{j=1}^{n}p_{j}f(w(x,g(x,s),s,\zeta_{j})$. Then $k_{n}$ converges continuously to $k$, i.e., $k_{n}(x_{n})\rightarrow k(x)$ whenever $x_{n}\rightarrow x$. Since $f$ is bounded, the sequence $k_{n}$ is uniformly bounded. Using Proposition 1 and equality (4), we have

[TABLE]

Thus, $\Phi s_{n}$ converges weakly to $\Phi s$. We conclude that $\Phi$ is continuous. Thus, by the Schauder-Tychonoff’s fixed point theorem, $\Phi$ has a fixed point.

Proof of Lemma 1. Assume that $f\in B(X\times\mathcal{P}(X))$ is concave and increasing in $x$. Since the composition of a concave and increasing function with a concave function is a concave function, the function $f(w(x,a,s,\zeta),s)$ is concave in $(x,a)$ for all $s$ and $\zeta$. Since $w$ and $f$ are increasing in $x$ then $f(w(x,a,s,\zeta),s)$ is increasing in $x$ for all $a$, $s$ and $\zeta$. Thus, $h(x,a,s,f)$ is concave in $(x,a)\;$and increasing in $x$ as the sum of concave and increasing functions. A standard argument shows that $Tf$ is increasing in $x$. Proposition 2.3.6 in Bertsekas et al. (2003) and the fact that $\Gamma(x)$ is convex-valued imply that $Tf(x,s)=\underset{a\in\Gamma(x)}{\max}h(x,a,s,f)$ is concave in $x$.

We conclude that for all $n=1,2,3\ldots$, $T^{n}f$ is concave and increasing in $x$. Standard dynamic programming arguments (see Bertsekas and Shreve (1978)) show that $T^{n}f$ converges to $V$ uniformly. Since the set of concave and increasing functions is closed under uniform convergence, $V$ is concave and increasing in $x$.

Since $\pi$ is strictly concave in $a$, $h(x,a,s,V)$ is strictly concave in $a$. This implies that $G(x,s)=\operatorname*{argmax}_{a\in\Gamma(x)}h(x,a,s,V)$ is single-valued which proves the Lemma.

B.3 Comparative statics: Proof of Theorem 4

Proof of Theorem 4. Under the assumptions of Theorem 1, the operator $M_{s}:\mathcal{P}(X)\times I\rightarrow\mathcal{P}(X)$ defined by

[TABLE]

has a unique fixed point $\mu_{s,e}$ for each $s\in\mathcal{P}(X)$ and $e\in I$.

Fix $s\in\mathcal{P}(X)$. Let $\theta_{2}\succeq_{SD}\theta_{1}$ and $e_{2}\succeq_{I}e_{1}$ and let $B$ be an upper set. We have

[TABLE]

Thus, $M_{s}(\theta_{2},e_{2})\succeq_{SD}M_{s}(\theta_{1},e_{1})$. The first inequality holds because $\theta_{2}\succeq_{SD}\theta_{1}$ and $Q$ is increasing in $x$ when $B$ is an upper set. The second inequality follows from the fact that $Q\;$is increasing in $e$ when $B$ is an upper set.

We conclude that $M_{s}$ is an increasing function from $\mathcal{P}(X)\times I$ into $\mathcal{P}(X)$ when $\mathcal{P}(X)$ is endowed with $\succeq_{SD}$. Thus, $M_{s}^{n}(\theta_{2},e_{2})\succeq_{SD}M_{s}^{n}(\theta_{1},e_{1})$ for all $n\in\mathbb{N}$. $Q$ being $X$-ergodic implies that $M_{s}^{n}(\theta_{i},e_{i})$ converges weakly to $\mu_{s,e_{i}}$. Since $\succeq_{SD}$ is closed under weak convergence (see Kamae et al. (1977)), we have $\mu_{s,e_{2}}\succeq_{SD}\mu_{s,e_{1}}$.

Now assume that $e_{2}\succeq_{I}e_{1}$ and let $s(e_{2}),s(e_{1})$ be the corresponding MFEs. Assume in contradiction that $s(e_{2})\prec s(e_{1})$. From the same argument as in Theorem 1 we can conclude that $\mu_{s(e_{2}),e}\succeq_{SD}\mu_{s(e_{1}),e}$ for each $e\in I$. Note that $s(e)$ is an MFE if and only if $s(e)=\mu_{s(e),e}$. We have

[TABLE]

Transitivity of $\succeq_{SD}$ implies $s(e_{2})\succeq_{SD}s(e_{1})$. But since $\succeq_{SD}$ agrees with $\succeq$, $s(e_{2})\succeq_{SD}s(e_{1})$ implies $s(e_{2})\succeq s(e_{1})$ which is a contradiction. We conclude that $s(e_{2})\succeq s(e_{1})$.

B.4 Dynamic Oligopoly Models: Proofs of Theorems 5, 6, 7, and 8

Proof of Theorem 5. The idea of the proof is to show that the conditions of Theorem 1 and Theorem 3 hold. In Lemma 4 we prove that the optimal stationary investment strategy is single-valued. In Lemma 5 we prove that $Q$ is increasing in $x$ and decreasing in $s$. In Lemma 7 we prove that the state space can be chosen to be compact. That is, there exists a compact set $\bar{X}=[0,\bar{x}]$ such that $Q(x,s,\bar{X})=1$ whenever $x\in\bar{X}$ and all $s\in\mathcal{P}(X)$. This means that if a firm’s initial state is in $\bar{X}$, then the firm’s state will remain in $\bar{X}$ in the next period with probability $1$. In Lemma 8 we prove that $Q$ is $\bar{X}$-ergodic. Thus, all conditions from Theorem 1 and Theorem 3 hold and we conclude that the model has a unique MFE.

We first introduce some notations.

Let $B(X\times\mathcal{P}(X))$ be the space of all bounded functions on $X\times\mathcal{P}(X)$. For $f\in B(X\times\mathcal{P}(X))$ define

[TABLE]

For the rest of the paper we say that $f\in B(X\times\mathcal{P}(X))$ is differentiable if it is differentiable in the first argument. Similarly, we write $u_{x}(x,s)$ to denote the derivative of $u$ with respect to $x$.

For the proof of the theorem, it will be convenient to change the decision variable in the Bellman equation. Define

[TABLE]

and note that we can write $a=k^{-1}(z-(1-\delta)x)$, which is well defined because $k$ is strictly increasing. The resulting Bellman operator is given by

[TABLE]

where $\Gamma(x)=[(1-\delta)x+k(0),(1-\delta)x+k(\bar{a})]$ and

[TABLE]

where $\pi(x,z,s)=u(x,s)-dk^{-1}(z-(1-\delta)x)$.

Let $\mu_{f}(x,s)=\operatorname{argmax}_{z\in\Gamma(x)}J(x,z,s,f)$ and $\mu(x,s)=\operatorname{argmax}_{z\in\Gamma(x)}J(x,z,s,V)$. Note that $\mu(x,s)=(1-\delta)x+k(g(x,s))$ where $g$ is the optimal stationary investment strategy. With this change of variables, we can use the envelope theorem (see Benveniste and Scheinkman (1979)). Since $u$ and $k$ are continuously differentiable, then $J(x,z,s,f)$ is continuously differentiable in $x$. The envelope theorem implies that $Kf$ is differentiable and

[TABLE]

Lemma 4

$\mu(x,s)$* is single-valued, increasing in $x$ and decreasing in $s$.*

Proof. The main step of the proof is to show that if $f\in B(X\times\mathcal{P}(X))$ has decreasing differences then $Kf\in B(X\times\mathcal{P}(X))$ has decreasing differences. This implies that the value function $V$ has decreasing differences. An application of a Theorem by Topkis implies that $\mu(x,s)$ is increasing in $x$ and decreasing in $s$. Single-valuedness of $\mu$ follows from the concavity of the value function. We provide the details below.

Assume that $f\in B(X\times\mathcal{P}(X))$ is concave in $x$, differentiable, and has decreasing differences. The function $f(z\zeta,s)$ is concave and increasing in $z$ for all $s$ and $\zeta$. Since $k$ is strictly concave and strictly increasing, $k^{-1}$ is strictly convex and strictly increasing. This implies that $-k^{-1}(z-(1-\delta)x)$ is concave in $(x,z)$. Thus, $J(x,z,s,f)$ is concave in $(x,z)$ as the sum of concave functions. Proposition 2.3.6 in Bertsekas et al. (2003) and the fact that $\Gamma(x)$ is convex-valued imply that $Kf(x,s)$ is concave in $x$.

Since $f$ has decreasing differences, then $f(z\zeta,s)$ has decreasing differences in $(z,s)$ for all $\zeta$. Thus, $J$ has decreasing differences in $(z,s)$ as the sum of functions with decreasing differences. From Theorem 6.1 in Topkis (1978), $\mu_{f}(x,s)$ is decreasing in $s$ for every $x$.

Let $x_{2}\geq x_{1}$, $z_{2}\geq z_{1}$, $y^{\prime}=z_{1}-(1-\delta)x_{2}$, $y=z_{1}-(1-\delta)x_{1}$ and $t=z_{2}-z_{1}$. Note that $y\geq y^{\prime}$. Convexity of $k^{-1}$ implies that for $y\geq y^{\prime}$ and $t\geq 0$, we have $k^{-1}(y+t)-k^{-1}(y)\geq k^{-1}(y^{\prime}+t)-k^{-1}(y^{\prime})$. That is, $k^{-1}(z-(1-\delta)x)$ has decreasing differences in $(x,z)$. Thus, $\pi(x,z,s)=u(x,s)-k^{-1}(z-(1-\delta)x)$ has increasing differences in $(x,z)$.

Let $s_{2}\succeq s_{1}$. For every $x\in X$ we have

[TABLE]

The first and last equality follow from the envelope theorem. The first inequality follows since $\pi$ has decreasing differences in $(x,s)$. The second inequality follows from the facts that $\pi$ has increasing differences in $(x,z)$ and $\mu_{f}(x,s_{1})\geq\mu_{f}(x,s_{2})$. Thus, $Kf$ has decreasing differences.

Define $f^{n}=K^{n}f:=K(K^{n-1}f)$ for $n=1,2,\ldots$ where $K^{0}f:=f$. By iterating the previous argument we conclude that $f_{x}^{n}(x,s)$ is decreasing in $s$ and $f^{n}(x,s)$ is concave in $x$ for every $n\in\mathbb{N}$.

Standard dynamic programming arguments (see Bertsekas and Shreve (1978)) show that $f^{n}$ converges uniformly to $V$. Since the set of concave functions is closed under uniform convergence, $V$ is concave in $x$. The envelope theorem implies that $f_{x}^{n}(x,s)=\pi_{x}(x,\mu_{f^{n}}(x,s),s)$ for every $n\in\mathbb{N}$. Since $J(x,z,s,f^{n})$ is strictly concave in $z$ when $f^{n}$ is concave, $\mu_{f^{n}}$ is single-valued. Theorem 3.8 and Theorem 9.9 in Stokey and Lucas (1989) show that $\mu_{f^{n}}\rightarrow\mu$. Thus, $f_{x}^{n}(x,s)=\pi_{x}(x,\mu_{f^{n}}(x,s),s)\rightarrow\pi_{x}(x,\mu(x,s),s)=V_{x}(x,s)$. Using (5), we conclude that $V_{x}(x,s)$ is decreasing in $s$; hence, $V$ has decreasing differences. The same argument as above shows that $J(x,z,s,V)$ has decreasing differences in $(z,s)$ and increasing differences in $(x,z)$. Since $J(x,z,s,V)$ is strictly concave in $z$, then $\mu$ is single-valued. It is easy to see that $\Gamma(x)$ is ascending in the sense of Topkis (1978) (i.e., for $x_{2}\geq x_{1}$ if $z\in\Gamma(x_{2})$ and $z^{\prime}\in\Gamma(x_{1})$ then $\max\{z,z^{\prime}\}\in\Gamma(x_{2})$ and $\min\{z,z^{\prime})\in\Gamma(x_{1})$). Theorem 6.1 in Topkis (1978) implies that $\mu(x,s)$ is increasing in $x$ and decreasing in $s$ which proves the Lemma.

Lemma 5

$Q$* is increasing in $x$ for each $s\in S$ and decreasing in $s$ for each $x\in X$.*

Proof. For each $s\in\mathcal{P}(X)$, $x_{2}\geq x_{1}$ and any upper set $B$ we have

[TABLE]

where the inequality follows since $\mu$ is increasing in $x$. Thus, $Q(x_{2},s,\cdot)\succeq_{SD}Q(x_{1},s,\cdot)$.

Similarly since $\mu(x,s)$ is decreasing in $s$, $Q$ is decreasing in $s$ for each $x\in X$.

We prove the following useful auxiliary lemma.

Lemma 6

(i) $\mu(x,s)$ is strictly increasing in $x$.

(ii) For all $s\in\mathcal{P}(X)$, $\mu$ is Lipschitz-continuous in the first argument with a Lipschitz constant $1$. That is,

[TABLE]

for all $x_{2},x_{1}$ and $s\in\mathcal{P}(X)$.

Proof. (i) Fix $s\in\mathcal{P}(X)$. Assume in contradiction that $x_{2}>x_{1}$ and $\mu(x_{1},s)=\mu(x_{2},s)$. First note that $\mu(x_{1},s)=\mu(x_{2},s)\geq(1-\delta)x_{2}+k(0)>(1-\delta)x_{1}+k(0):=\min\Gamma(x_{1})$. Thus, $\min\Gamma(x_{1})<\mu(x_{1},s)\leq\max\Gamma(x_{1})<\max\Gamma(x_{2})$. We have

[TABLE]

which contradicts the optimality of $\mu(x_{2},s)$, since $\mu(x_{2},s)<\max\Gamma(x_{2})$. The first inequality follows from the first order condition (recall that $\min\Gamma(x_{1})<\mu(x_{1},s)$). The second inequality follows from the fact that $k^{-1}$ is strictly convex, which implies that $(k^{-1})^{\prime}$ is strictly increasing. Thus, $\mu$ is strictly increasing in $x$.

(ii) Fix $s\in\mathcal{P}(X)$. Let $x_{2}>x_{1}$. If $\mu(x_{1},s)=\max\Gamma(x_{1})=(1-\delta)x_{1}+k(\bar{a})$, then

[TABLE]

So we can assume that $\mu(x_{1},s)<\max\Gamma(x_{1})$. Assume in contradiction that $\mu(x_{2},s)-\mu(x_{1},s)>x_{2}-x_{1}$. Then $\mu(x_{2},s)-(1-\delta)x_{2}>\mu(x_{1},s)-(1-\delta)x_{1}$. We have

[TABLE]

The first inequality follows from the first order condition. The second inequality follows from the facts that $(k^{-1})$ is strictly convex and $V$ is concave (see the proof of Lemma 4). The last inequality implies that $\mu(x_{2},s)=\min\Gamma(x_{2})=(1-\delta)x_{2}+k(0)$. But $\mu(x_{1},s)\geq\min\Gamma(x_{1})$ implies

[TABLE]

which is a contradiction. We conclude that $\mu$ is Lipschitz-continuous in the first argument with a Lipschitz constant $1$.

Lemma 7

The state space can be chosen to be compact: There exists a compact set $\bar{X}=[0,\bar{x}]$ such that $Q(x,s,\bar{X})=1$ whenever $x\in\bar{X}$ and all $s\in\mathcal{P}(X)$.

Proof. Fix $s\in\mathcal{P}(X)$. Since $\max\Gamma(x)=(1-\delta)x+k(\bar{a})$, for all $x>0$, we have

[TABLE]

The last inequality and the fact that $(1-\delta)\zeta_{n}<1$ imply that there exists $\bar{x}$ (that does not depend on $s$) such that $\mu(x,s)\zeta_{n}<x$ for all $x\geq\bar{x}$.

Let $\bar{X}=[0,\bar{x}]$. For all $s\in\mathcal{P}(X)$ and $\zeta\in E$, if $x\in\bar{X}$ we have

[TABLE]

That is, $\mu(x,s)\zeta\in\bar{X}$. Thus, $Q(x,s,\bar{X})=\Pr(\mu(x,s)\zeta\in\bar{X})=1$ whenever $x\in\bar{X}$.

Lemma 8

$Q$* is $\bar{X}$-ergodic.*

Proof. Fix $s\in\mathcal{P}(X)$. Define the sequences $x_{k+1}=\mu(x_{k},s)\zeta_{n}$ and $y_{k+1}=\mu(y_{k},s)\zeta_{1}$ where $x_{1}=0$ and $y_{1}=\bar{x}$. Note that $\{x_{n}\}_{n=1}^{\infty}$ is strictly increasing, i.e., $x_{k+1}>x_{k}$ for all $k$. To see this, first note that $x_{2}=\mu(x_{1},s)\zeta_{n}\geq k(0)\zeta_{n}>0=x_{1}$. Now if $x_{k}>x_{k-1}$, then $\mu$ being strictly increasing in $x$ (see Lemma 6 part (i)) implies that $x_{k+1}=\mu(x_{k},s)\zeta_{n}>\mu(x_{k-1},s)\zeta_{n}=x_{k}$. Let $C_{s}=\min\{x\in\mathbb{R}_{+}:\mu(x,s)\zeta_{n}=x\}$. From the facts that $\mu(0,s)\zeta_{n}\geq k(0)\zeta_{n}>0$, $\mu(\bar{x},s)\zeta_{n}<\bar{x}$ (see Lemma 7), and $\mu$ is continuous (see Lemma 3), by Brouwer fixed point theorem $C_{s}$ is well defined. Similarly, the sequence $\{y_{n}\}_{n=1}^{\infty}$ is strictly decreasing and therefore converges to a limit $C_{s}^{\ast}$.

We claim that $C_{s}>C_{s}^{\ast}$. To see this, first note that Lemma 7 implies that the function $f_{s}$, defined by $f_{s}(x,\zeta)=\mu(x,s)\zeta$, is from $\bar{X}\times E$ into $\bar{X}$. Note that $f_{s}$ is increasing in both arguments and that $\bar{X}$ is a complete lattice. Thus, Corollary 2.5.2 in Topkis (2011) implies that the greatest and least fixed points of $f_{s}$ are increasing in $\zeta$. Lemma 6 part (ii) and $\zeta_{1}<1$ imply that $f_{s}(x,\zeta_{1})=\mu(x,s)\zeta_{1}$ is a contraction mapping from $\bar{X}$ to itself. Thus, $f_{s}(x,\zeta_{1})$ has a unique fixed point which equals the limit of the sequence $\{y_{n}\}_{n=1}^{\infty}$, $C_{s}^{\ast}$. Since the least fixed point of $f_{s}$ is increasing in $\zeta$ we conclude that $C_{s}\geq C_{s}^{\ast}$. Since $\mu$ is increasing and positive we have $C_{s}=\mu(C_{s},s)\zeta_{n}>\mu(C_{s},s)\zeta_{1}\geq\mu(C_{s}^{\ast},s)\zeta_{1}=C_{s}^{\ast}$

Let $x^{\ast}=(C_{s}+C_{s}^{\ast})/2$. Since $x_{k}\uparrow C_{s}$ and $y_{k}\downarrow C_{s}^{\ast}$, there exists a finite $N_{1}$ such that $x_{k}>x^{\ast}$ for all $k\geq N_{1}$, and similarly, there exists a finite $N_{2}$ such that $y_{k}<x^{\ast}$ for all $k\geq N_{2}$. Let $m=\max\{N_{1},N_{2}\}$. Thus, after $m$ periods there exists a positive probability ($\zeta_{1}^{m}$) to move from the state $\bar{x}$ to the set $[0,x^{\ast}]$, and a positive probability to move from the state [math] to the set $[x^{\ast},\bar{x}]$. That is, we found $x^{\ast}\in[0,\bar{x}]$ and $m>0$ such that $Q^{m}(\bar{x},s,[0,x^{\ast}])>0$ and $Q^{m}(0,s,[x^{\ast},\bar{x}])>0$. Since $\bar{X}$ is compact and $Q$ is increasing in $x$, then $Q$ is $\bar{X}$-ergodic (see Theorem 2 in Hopenhayn and Prescott (1992) or Theorem 2.1 in Bhattacharya and Lee (1988)).

Now, we prove Theorem 6. The main idea behind the proof is to show that the optimal stationary strategy $g$ is increasing or decreasing in the relevant parameter using a lattice-theoretical approach and then to conclude that the conditions of Theorem 4 hold.

Let $(I,\succeq_{I})$ be a partial order set that influences the firms’ decisions. We denote a generic element in $I\;$ by $e$. For instance, $e$ can be the discount factor or the cost of a unit of investment. Throughout the proof of Theorem 6 we allow an additional argument in the functions that we consider. For instance, the value function $V$ is denoted by:

[TABLE]

Likewise, the optimal stationary strategy is denoted by $g(x,s,e)$, and $u(x,s,e)$ is the one-period profit function. Here, we come back to the original formulation over actions $a$.

Proof of Theorem 6. i) Assume that $f\in B(X\times\mathcal{P}(X)\times I)$ is concave in the first argument and has decreasing differences in $(x,d)$ where $I\subseteq\mathbb{R}_{+}$ is the set of all possible unit investment costs endowed with the natural order, $d_{2}\geq d_{1}$.

Fix $s\in\mathcal{P}(X)$. Note that $da$ has increasing differences in $(a,d)$. Thus, $u(x,s)-da$ has decreasing differences in $(a,d)$, $(x,a)$ and $(x,d)$. Since $f$ has decreasing differences and $k$ is increasing, the function $f(((1-\delta)x+k(a))\zeta,s,d)$ has decreasing differences in $(a,d)$ and $(x,d)$ for every $\zeta\in E$. Since $f$ is concave in the first argument and $k$ is increasing, it can be shown that the function $f(((1-\delta)x+k(a))\zeta,s,d)$ has decreasing differences in $(x,a)$ for every $\zeta\in E$. Thus, the function

[TABLE]

has decreasing differences in $(x,a)$, $(x,d)$ and $(a,d)$ as the sum of functions with decreasing differences.

A similar argument to Lemma 1 in Hopenhayn and Prescott (1992) or Lemma 2 in Lovejoy (1987) implies that if $h(x,a,s,d,f)$ has decreasing differences in $(x,a)$, $(x,d)$ and $(a,d)$, then $Tf(x,s,d)=\underset{a\in[0,\bar{a}]}{\max}h(x,a,s,d,f)$ has decreasing differences in $(x,d)$. The proof of Lemma 4 implies that $Tf$ is concave in $x$. We conclude that for all $n=1,2,3...$, $T^{n}f$ is concave in $x$ and has decreasing differences. Standard dynamic programming arguments (see Bertsekas and Shreve (1978)) show that $T^{n}f$ converges to $V$ uniformly. Since the set of functions with decreasing differences is closed under uniform convergence, $V$ has decreasing differences in $(x,d)$. From the same argument as above, $h(x,a,s,d,V)$ has decreasing differences in $(a,d)$. Theorem 6.1 in Topkis (1978) implies that $g(x,s,d)$ is decreasing in $d$.

Define the order $\succeq_{I}$ by $d_{2}\succeq_{I}d_{1}$ if and only if $d_{1}\geq d_{2}$. Thus $d_{2}\succeq_{I}d_{1}$ implies that

[TABLE]

for all $x,s$ and every upper set $B$, because $g(x,s,d)$ is decreasing in $d$. That is, $Q(x,s,d_{2},\cdot)\succeq_{SD}Q(x,s,d_{1},\cdot)$ for all $x,s$ and $d_{2},d_{1}\in I$ such that $d_{2}\succeq_{I}d_{1}$. From Theorem 4 and Theorem 5 we conclude that $d_{2}\succeq_{I}d_{1}$ implies $s(d_{2})\succeq s(d_{1})$, i.e., $d_{2}\leq d_{1}$ implies $s(d_{2})\succeq s(d_{1})$.

(ii) The proof of part (ii) is the same as the proof of part (i) and is therefore omitted.

(iii) Assume that $f\in B(X\times\mathcal{P}(X)\times I)$ is increasing in the first argument and has decreasing differences in $(x,\beta)$ where $I=(0,1)$ is the set of all possible discount factors endowed with the reverse order; $\beta_{2}\succeq_{I}\beta_{1}$ if and only if $\beta_{1}\geq\beta_{2}$. A standard argument shows that $Tf$ is increasing in the first argument. We will only show that $h(x,a,s,\beta,f)$ has decreasing differences in $(a,\beta)$ and $(x,\beta)$; the rest of the proof is the same as the proof of part (i). Fix $s$, $x$ and let $\beta_{2}\succeq_{I}\beta_{1}$ (i.e., $\beta_{1}\geq\beta_{2}$), and $a_{2}\geq a_{1}$. Decreasing differences of $f$ and the fact that $k$ is increasing imply that $f(((1-\delta)x+k(a_{2}))\zeta,s,\beta)-f(((1-\delta)+k(a_{1}))\zeta,s,\beta)$ is decreasing in $\beta$ for all $\zeta\in E$. Since $\beta_{1}\geq\beta_{2}$, $f$ and $k$ are increasing, we have

[TABLE]

Thus $h(x,a,s,\beta,f)$ has decreasing differences in $(a,\beta)$. A similar argument shows that $h(x,a,s,\beta,f)$ has decreasing differences in $(x,\beta)$.

Proof of Theorem 7. (i) The proof of the Theorem is similar to the proof of Theorem 5. The idea of the proof is to show that the conditions of Theorem 9 hold. We now show that $\overline{Q}$ is increasing in $x$ and decreasing in $s$ (see Section A.1 for the definition of $\overline{Q}$).

We use the same change of variables and notation as in the proof of Theorem 5. Define

[TABLE]

and note that $a=(1-\delta)^{-1}z-x$. The resulting Bellman operator is given by

[TABLE]

where $\Gamma(x)=[(1-\delta)(x+1),(1-\delta)(x+\overline{a})]$,

[TABLE]

and

[TABLE]

Let $\mu(x,s)=\operatorname{argmax}_{z\in\Gamma(x)}J(x,z,s,V)$. Since $\pi$ is concave in $(x,z)$, Lemma 4 implies that the policy function $\mu(x,s)$ is single-valued.

It is immediate that $\pi$ has increasing differences in $(x,z)$, and decreasing differences in $(z,s)$ and $(x,s)$. Here $s_{2}\succeq s_{1}$ if and only if

[TABLE]

From Lemma 4, we can show that $\mu$ is increasing in $x$ and decreasing in $s$.

Thus, for each $s\in\mathcal{P}(X\times A)$, $x_{2}\geq x_{1}$ and any upper set $B\times D\in\mathcal{B}(X\times A)$ we have

[TABLE]

The equalities follow from the proof of Theorem 9. The inequality follows because $\mu$ is increasing in $x$. Thus, $\overline{Q}(x_{2},s,\cdot)\succeq_{SD}\overline{Q}(x_{1},s,\cdot)$, i.e., $\overline{Q}$ is increasing in $x$.

Similarly, because $\mu(x,s)$ is decreasing in $s$, we can show that $\overline{Q}$ is decreasing in $s$ for each $x\in X$.

We conclude that $\overline{Q}$ is decreasing in $s$ and increasing in $x$. Compactness of the state space $X$ and $X$-ergodicity of $\overline{Q}$ can be established using similar arguments to the arguments in Theorem 5. Thus, all the conditions of Theorem 9 parts (i) and (ii) hold. We conclude that the dynamic advertising model has a unique MFE.

The proofs of parts (ii) and (iii) are similar to the proof of Theorem 6 and are therefore omitted.

Proof of Theorem 8. (i) First note that the state space $X=[0,M_{1}]\times[0,M_{2}]$ is compact. We now show that $Q$ is increasing in $x_{1}$ and decreasing in $s$ with respect to $x_{1}$.

For the proof of the theorem, it will be convenient to change the decision variable in the Bellman equation. Define

[TABLE]

and note that we can write $a=k^{-1}(z(1+x_{2})-x_{2}x_{1})$, which is well defined because $k$ is strictly increasing. The resulting Bellman operator is given by

[TABLE]

where $\Gamma(x_{1},x_{2})=[\frac{x_{2}}{1+x_{2}}x_{1}+\frac{1}{1+x_{2}}k(0),\frac{x_{2}}{1+x_{2}}x_{1}+\frac{1}{1+x_{2}}k(\bar{a})]$,

[TABLE]

and

[TABLE]

Let $\mu(x_{1},x_{2},s)=\operatorname{argmax}_{z\in\Gamma(x_{1},x_{2})}J(x_{1},x_{2},z,s,V)$. From the arguments as the arguments in Lemma 4, the optimal stationary strategy $\mu(x_{1},x_{2},s)$ is single-valued.

Let $x^{\prime}_{1}\leq x_{1}$ and $s_{2}\succeq s_{1}$. Because $\nu$ is increasing, we have

[TABLE]

Thus, $\pi$ has decreasing differences in $(x_{1},s)$. In addition, $\pi$ has decreasing differences in $(z,s)$ and increasing differences in $(x_{1},z)$ (see the proof of Lemma 4). From Lemma 4, we can show that $\mu$ is increasing in $x_{1}$ and decreasing in $s$.

Recall that in every period, with probability $1-\beta$, each seller departs the market and a new seller with state $(0,0)$ immediately arrives to the market. With probability $\beta$, each seller stays in the market and moves to a new state according to the dynamics described in Section 4.3. Thus, we have

[TABLE]

where $\delta_{\{c\}}$ is the Dirac measure on the point $c\in\mathbb{R}^{2}$. Let $f:X_{1}\times X_{2}\rightarrow\mathbb{R}$ be increasing in the first argument. Assume that $x^{\prime}_{1}\leq x_{1}$. We have

[TABLE]

The inequality follows from the facts that $\mu$ is increasing in $x_{1}$, and $f$ is increasing in the first argument.

We conclude that $Q$ is increasing in $x_{1}$. Similarly, because $\mu$ is decreasing in $s$, we can prove that $Q$ is decreasing in $s$ with respect to $x_{1}$. We now show that $Q$ is $X$-ergodic.

The Markov chain $Q$ is said to satisfy the Doeblin condition if there exists a positive integer $n_{0}$, $\epsilon>0$ and a probability measure $\upsilon$ on $X$ such that $Q^{n_{0}}(x,s,B)\geq\epsilon\upsilon(B)$ for all $x\in X$ and all measurable $B$. From the definition of $Q$, we have $Q(x,s,B)\geq(1-\beta)\delta_{\{(0,0)\}}(B)$ for every measurable $B$, so $Q$ satisfies the Doeblin condition. Thus, $Q$ is $X$-ergodic (see Theorem 8 in Roberts and Rosenthal (2004)).

Thus, all the conditions of Theorem 2 and Theorem 3 are satisfied. We conclude that the dynamic reputation model has a unique MFE.

(ii) The proof of part (ii) is similar to the proof of Theorem 6 and is therefore omitted.

B.5 Heterogeneous Agent Macro Models: Proof of Corollary 3

Proof of Corollary 3. From Theorem 2 we only need to show that $Q$ is increasing in $x_{1}$ and decreasing in $s$ in order to prove Corollary 3.

Let $f:X_{1}\times X_{2}\rightarrow\mathbb{R}$ be increasing in the first argument. Assume that $x^{\prime}_{1}\leq x_{1}$. We have

[TABLE]

The inequality follows from the facts that $\tilde{g}$ is increasing in $x_{1}$ and $f$ is increasing in the first argument. In a similar manner, because $\tilde{g}$ is decreasing in the aggregator, we can show that $Q$ is decreasing in $s$ with respect to $x_{1}$.

We conclude that $Q$ is increasing in $x_{1}$ and decreasing in $s$.

B.6 Extensions: Proofs of Theorem 9 and Lemma 2

Proof of Theorem 9. The proofs of part (i) and of part (iii) are the same as the proofs of Theorem 1 and of Theorem 4. To prove part (ii) we need to show that the operator $\overline{\Phi}:\mathcal{P}(X)\rightarrow\mathcal{P}(X)$ defined by

[TABLE]

is continuous (the rest of the proof is the same as the proof of Theorem 3). The continuity of $\overline{\Phi}$ follows from a similar argument to the argument in the proof of Theorem 3. We provide the proof for completeness.

Note that for every bounded and measurable function $f:X\times A\rightarrow\mathbb{R}$ and for every $s\in\mathcal{P}(X\times A)$ we have

[TABLE]

To see this, first assume that $f=1_{B\times D}$ for some measurable set $B\times D\in\mathcal{B}(X\times A)$. We have

[TABLE]

A standard argument shows that Equation (7) holds for every bounded and measurable function $f$.

Assume that $s_{n}$ converges weakly to $s$. Thus, the marginal distribution $s_{n}(\cdot,A)$ converges weakly to $s(\cdot,A)$. Let $f:X\times A\rightarrow\mathbb{R}$ be a continuous and bounded function. Because $w$ and $g$ are continuous, we have

[TABLE]

whenever $x_{n}\rightarrow x$.

Let

[TABLE]

and

[TABLE]

Then $k_{n}$ converges continuously to $k$, i.e., $k_{n}(x_{n})\rightarrow k(x)$ whenever $x_{n}\rightarrow x$. Since $f$ is bounded, the sequence $k_{n}$ is uniformly bounded. Using Proposition 1 yields

[TABLE]

Thus, $\Phi s_{n}$ converges weakly to $\Phi s$. We conclude that $\Phi$ is continuous.

Proof of Lemma 2. Let $f:X\times\Theta\rightarrow R$ be increasing in in the first. The fact that $Q_{\theta}$ is increasing in $x$ implies that the function

[TABLE]

is increasing in $x$ for every type $\theta$ and every extended population state $s_{h}$. That is, $Q_{h}$ is increasing in $x$. Similarly, $Q_{h}$ is decreasing in $s_{h}$ with respect to $x$ when $Q_{\theta}$ is decreasing in $s_{h}$.