On Laws of Large Numbers for Systems with Mean-Field Interactions and Markovian Switching

TL;DR

This paper establishes laws of large numbers for mean-field systems with Markovian switching, where the empirical measure limit is a random measure influenced by the switching process's history.

Contribution

It introduces a novel approach to characterize the limit as a conditional distribution, addressing challenges posed by the coupling and randomness in switching diffusions.

Findings

Law of large numbers for mean-field switching diffusions established.

Limit of empirical measures is a random measure dependent on switching history.

Characterization of the limit via stochastic McKean-Vlasov equations with Markovian switching.

Abstract

Focusing on stochastic systems arising in mean-field models, the systems under consideration belong to the class of switching diffusions, in which continuous dynamics and discrete events coexist and interact. The discrete events are modeled by a continuous-time Markov chain. Different from the usual switching diffusions, the systems include mean-field interactions. Our effort is devoted to obtaining laws of large numbers for the underlying systems. One of the distinct features of the paper is the limit of the empirical measures is not deterministic but a random measure depending on the history of the Markovian switching process. A main difficulty is that the standard martingale approach cannot be used to characterize the limit because of the coupling due to the random switching process. In this paper, in contrast to the classical approach, the limit is characterized as the conditional distribution (given the history of the switching process) of the solution to a stochastic McKean-Vlasov differential equation with Markovian switching.

Equations

$$\begin{array}[]{rl}dx_{i}(t)&\!\!\!\displaystyle=b\bigg{(}x_{i}(t),{1\over N}\sum_{j=1}^{N}\delta_{x_{j}(t)},\alpha(t_{-}),u_{i}(t)\bigg{)}dt+\sigma\bigg{(}x_{i}(t),{1\over N}\sum_{j=1}^{N}\delta_{x_{j}(t)},\alpha(t_{-})\bigg{)}dw_{i}(t)\\ x_{i}(0)&\!\!\!\displaystyle=x_{i},\ i\leq N,\ \alpha(0)=\alpha.\end{array}$$

$$J_{i}(x_{j},u_{j}(\cdot):j\leq N)=\mathbb{E}_{\{x_{j}:j\leq N\}}\int^{T}_{0}R\bigg{(}x_{i}(t),{1\over N}\sum_{j=1}^{N}\delta_{x_{j}(t)},u_{i}(t)\bigg{)}dt,$$

$$dx_{i}(t)=b\bigg{(}x_{i}(t),{1\over N}\sum_{j=1}^{N}\delta_{x_{j}(t)},\alpha(t_{-})\bigg{)}dt+\sigma\bigg{(}x_{i}(t),{1\over N}\sum_{j=1}^{N}\delta_{x_{j}(t)},\alpha(t_{-})\bigg{)}dw_{i}(t),$$

$$\mathbb{P}\Big{(}\alpha(t+\Delta t)=j_{0}\Big{|}\alpha(t)=i_{0},\ 0\leq s\leq t\Big{)}=q_{i_{0}j_{0}}\Delta t+o(\Delta t),$$

$$\|\mu-\eta\|_{BL}=\sup\Bigg{\{}\big{|}\big{\langle}\mu,f\big{\rangle}-\big{\langle}\eta,f\big{\rangle}\big{|}:\|f\|\leq 1,\sup_{x\neq y\in\mathbb{R}^{d}}{|f(x)-f(y)|\over|x-y|}\leq 1\Bigg{\}},$$

$$d\big{(}(\mu,i_{0}),(\eta,j_{0})\big{)}=\big{\|}\mu-\eta\big{\|}_{BL}+d_{\mathbb{S}}\big{(}i_{0},j_{0}\big{)},\quad\forall\,\mu,\eta\in\mathscr{M}_{1},i_{0},j_{0}\in\mathbb{S}.$$

$$\mathcal{F}^{N,\alpha}_{t}=\sigma\big{\{}w_{i}(s),\alpha(s):0\leq s\leq t,1\leq i\leq N\big{\}}.$$

$$\mathscr{L}(\varsigma)\hbox{ its distribution and }\eta_{t}=\mathscr{L}\big{(}\varsigma\big{|}\mathcal{F}^{\alpha}_{t_{-}}\big{)}\hbox{ its conditional distribution given }\mathcal{F}^{\alpha}_{t_{-}}$$

$$\mathbb{E}\big{(}f(\varsigma)\big{|}\mathcal{F}^{\alpha}_{t_{-}}\big{)}=\int_{\mathbb{R}^{d}}f(x)\eta_{t}(dx)\hbox{ for any }f\in C_{b}(\mathbb{R}^{d}).$$

$$\Big{|}b\big{(}x,\mu,i_{0}\big{)}-b\big{(}y,\eta,i_{0}\big{)}\Big{|}+\Big{|}\sigma\big{(}x,\mu,i_{0}\big{)}-\sigma\big{(}y,\eta,i_{0}\big{)}\Big{|}\leq L\Big{(}\big{|}x-y\big{|}+\big{\|}\mu-\eta\big{\|}_{BL}\Big{)},$$

$$\Big{|}b\big{(}x,\mu,i_{0}\big{)}\Big{|}\leq C\Big{(}1+\big{|}x\big{|}+\big{\langle}\mu,\varphi\big{\rangle}\Big{)},\quad(x,\mu,i_{0})\in\mathbb{R}^{d}\times\mathscr{M}_{1}\times\mathbb{S},$$

$${\mu }_N(t,A)=\frac{1}{N}\sum _{j=1}^N{\delta }_{x_j(t)}(A).$$

$$\sup_{N\in\mathbb{N}}\mathbb{E}\big{\langle}\mu_{N}(0),\psi\big{\rangle}<\infty,\quad\mathscr{L}\big{(}\mu_{N}(0)\big{)}\Rightarrow\delta_{\mu_{0}}\text{ in }\mathcal{P}\big{(}\mathscr{M}_{1},\|\cdot\|_{BL}\big{)},$$

$$\big{(}\mu_{\alpha}(t),\alpha(t)\big{)}=\big{(}\mathscr{L}\big{(}y(t)\big{|}\mathcal{F}^{\alpha}_{t_{-}}\big{)},\alpha(t)\big{)},\quad 0\leq t\leq T,$$

$$dy(t)=b\Big{(}y(t),\mathscr{L}\big{(}y(t)\big{|}\mathcal{F}^{\alpha}_{t_{-}}\big{)},\alpha(t_{-})\Big{)}dt+\sigma\Big{(}y(t),\mathscr{L}\big{(}y(t)\big{|}\mathcal{F}^{\alpha}_{t_{-}}\big{)},\alpha(t_{-})\Big{)}d\tilde{w}(t),\quad\mathscr{L}\big{(}y(0)\big{)}=\mu_{0},$$

$$d{x}_i(t)$$

$$x_i(0)$$

$$\big{|}\underline{b}_{i}\big{(}t,\underline{x},i_{0}\big{)}-\underline{b}_{i}\big{(}t,\underline{y},i_{0}\big{)}\big{|}+\big{|}\underline{\sigma}_{i}\big{(}t,\underline{x},i_{0}\big{)}-\underline{\sigma}_{i}\big{(}t,\underline{y},i_{0}\big{)}\big{|}\leq K\big{|}\underline{x}-\underline{y}\big{|},\\$$

$$\big{|}\underline{b}_{i}\big{(}t,\underline{x},i_{0}\big{)}\big{|}+\big{|}\underline{\sigma}_{i}\big{(}t,\underline{x},i_{0}\big{)}\big{|}\leq K\big{(}1+\big{|}\underline{x}\big{|}\big{)},$$

$$\displaystyle\mathcal{L}_{N}V\big{(}t,\underline{x},i_{0}\big{)}$$

$$\displaystyle\quad+\sum_{j_{0}\in\mathcal{M}}q_{i_{0}j_{0}}\Big{(}V\big{(}t,\underline{x},j_{0}\big{)}-V\big{(}t,\underline{x},i_{0}\big{)}\Big{)},$$

$$\big{[}M_{i_{0}j_{0}}\big{]}(t)=\sum_{0\leq s\leq t}{1\!\!1}\big{(}\alpha(s_{-})=i_{0}\big{)}{1\!\!1}\big{(}\alpha(s)=j_{0}\big{)},\quad\quad\big{\langle}M_{i_{0}j_{0}}\big{\rangle}(t)=\int_{0}^{t}q_{i_{0}j_{0}}{1\!\!1}\big{(}\alpha(s_{-})=i_{0}\big{)}ds,$$

$$M_{i_{0}j_{0}}(t)=\big{[}M_{i_{0}j_{0}}\big{]}(t)-\big{\langle}M_{i_{0}j_{0}}\big{\rangle}(t)$$

$$\big{[}w_{i},w_{j}\big{]}=0\text{ when }i\neq j,\quad\big{[}M_{i_{0}j_{0}},w_{j}\big{]}=0,\quad\big{[}M_{i_{0}j_{0}},M_{p_{0}q_{0}}\big{]}=0\text{ when }(i_{0},j_{0})\neq(p_{0},q_{0}).$$

$$\begin{array}[]{ll}V\big{(}t,\underline{x}(t),\alpha(t)\big{)}&\!\!\displaystyle=V\big{(}0,\underline{x}(0),\alpha(0)\big{)}+\int_{0}^{t}\bigg{(}{\partial\over\partial s}+\mathcal{L}_{N}\bigg{)}V\big{(}s,\underline{x}(s),\alpha(s_{-})\big{)}ds\\ &\displaystyle\quad+\sum_{i=1}^{N}\int_{0}^{t}\Big{\langle}\nabla_{x_{i}}V\big{(}s,\underline{x}(s),\alpha(s_{-})\big{)},\underline{\sigma}_{i}\big{(}s,\underline{x}(s),\alpha(s_{-})\big{)}dw_{i}(s)\Big{\rangle}\\ &\displaystyle\quad+\sum_{i_{0}\neq j_{0}}\int_{0}^{t}\Big{(}V\big{(}s,\underline{x}(s),j_{0}\big{)}-V\big{(}s,\underline{x}(s),i_{0}\big{)}\Big{)}dM_{i_{0}j_{0}}(s).\end{array}$$

$$dx_{i}(t)=b\Big{(}x_{i}(t),\mu_{N}(t),\alpha(t_{-})\Big{)}dt+\sigma\Big{(}x_{i}(t),\mu_{N}(t),\alpha(t_{-})\Big{)}dw_{i}(t),\quad 1\leq i\leq N,$$

$${\mu }_N(t)={\delta }_{\underline{x}(t)}=\frac{1}{N}\sum _{j=1}^N{\delta }_{x_j(t)}\in {\mathcal{M}}_1.$$

$$\underline{b}_{i}\big{(}t,\underline{x},i_{0}\big{)}=b\big{(}x_{i},\delta_{\underline{x}},i_{0}\big{)},\quad\underline{\sigma}_{i}\big{(}t,\underline{x},i_{0}\big{)}=\sigma\big{(}x_{i},\delta_{\underline{x}},i_{0}\big{)},$$

$$\big{\|}\delta_{\underline{x}}-\delta_{\underline{y}}\big{\|}_{BL}\leq{C\over N}\big{|}\underline{x}-\underline{y}\big{|},\quad\forall\,\,\underline{x},\underline{y}\in(\mathbb{R}^{d})^{N}.$$

$$\sup_{0\leq t\leq T}\mathbb{E}\bigg{(}\big{\langle}\mu_{N}(t),\psi\big{\rangle}+1\bigg{)}^{p}\leq C,$$

$$e^{-C(t-s)}\Big{(}\big{\langle}\mu_{N}(s),\psi\big{\rangle}+1\Big{)}^{p}\leq\mathbb{E}\Big{[}\Big{(}\big{\langle}\mu_{N}(t),\psi\big{\rangle}+1\Big{)}^{p}\Big{|}{\mathcal{F}}^{N,\alpha}_{s}\Big{]}\leq e^{C(t-s)}\Big{(}\big{\langle}\mu_{N}(s),\psi\big{\rangle}+1\Big{)}^{p}.$$

Full text

Key Words. Mean-field model, Markovian switching process, law of large number, McKean-Vlasov equation.

Mathematics Subject Classification. 60J25, 60J27, 60J60, 93E20.

Running title. LLN for Systems with Mean-Field Interactions and Markovian Switching

1 Introduction

This work focuses on laws of large numbers for a class of stochastic systems involving mean-field interactions and random switching. It is motivated by two lines of recent advances in the study of stochastic systems and applications. One of them is the emerging interests in mean-field models, and the other is the use of regime switching in stochastic systems.

Originated from statistical physics, mean-field models describe stochastic systems containing a large number of particles having weak interactions. To overcome the complexity of interactions due to the large scale of system, all interactions with each particle are replaced by a single averaged interaction (naturally represented by an empirical measure associated to system). One of the first mathematical treatments was the influential work of Dawson [7], which rigorously justified the replacement of a large number of “bodies” by a representative “body” in many-body problems. Also obtained in that paper was phase transition properties. The subsequent work [9] delineates some limit theory for jump mean-field models. Although originally appeared in physics, mean-field models have arisen in many different application areas, including communication networks, mathematical finance, chemical and biological systems, and social sciences. For an extensive list of references to such applications, see [2, 4, 5, 30]. Recently, renewed interest has been shown using mean-field models in game theory, which is originated independently in the work of Huang, Caines, and Malhamé [16, 17], and Lasry and Lions [21]. The mean-field interaction has been used to model the weak interaction between players in large population games and the limiting results are used to construct computable decentralized strategies, leading to substantial progress in the development of mean-field game theory. The book by Bensoussan, Frehse, and Yam [4] provided an illuminating presentation and discussion of certain aspects of mean-field games and mean-field type controls, describing their similarities and differences together with a unified approach for treating them. A more analytic approach can be found in the book by Kolokoltsov [19]. It has been seen that mean-field models and mean-field games enjoy a wide range of applications and potential applications in economics, social networks, cyber physical systems, and other branches of sciences and engineering; see [4, 15, 16, 17, 26, 32, 33] and references therein.

Along another line, the so-called hybrid systems have gained increasing popularity due to their ability to handle numerous real-world applications in which discrete and continuous dynamics coexist and interact. One class of such hybrid systems is switching diffusions. Take for instance, applications in control systems and optimization. One of the most widely used control engineering models in the literature is the so-called linear quadratic Gaussian regulator problem; see [12] for a traditional model. For many new applications in networked systems, it has been found that in addition to the random noise represented by Brownian type of disturbances, there is a source of randomness owing to the presence of random environment that displays pure jump behavior and that can be modeled by a continuous-time Markov chain. As a result, one has a controlled switching diffusion instead of controlled diffusion as in the traditional setup; see [37] for some recent results on switching diffusions and applications. For a wide variety of applications, we mention the work on flexible manufacturing systems [29], approximation to invariant measures [3], controlled piecewise deterministic Markov processes [6], population dynamics [23], business cycle models in random environment [31], stochastic approximation [35] with applications to wireless communication such as spreading code optimization and adaptation in CDMA, Markowitz mean-variance portfolio selection [38], and Lotka-Volterra models in ecology [39], among others. Furthermore, there has been new effort in treating switching diffusions in conjunction with mean-field interactions [34].

Motivated by the aforementioned two aspects, the focus of the current paper lies in the intersection of the mean-field models and the switching diffusion models. We concentrate on large-scale systems with weak interactions in a random environment represented by switching diffusions in which the Markov chains delineate random environment changes not represented by the usual diffusions. Recently, some related works have been considered in [32, 33] for studying mean-field games and social optimality. In this work, we investigate functional laws of large numbers for such systems.

Why should we be so concerned about laws of large numbers and why should such an effort be necessary? Not only is the study interesting from a mathematical point of view, but also it is crucial from a practical consideration. Treating large-scale systems, a main effort is to reduce the computation complexity. Laws of large numbers provide us with an effective machinery to overcome the difficulties. As a motivational example, consider a mean-field game problem with $N$ players for a large number $N$. Let $x_{i}(t)\in{\mathbb{R}}^{d}$, $1\leq i\leq N$, be the state of player $i$ that satisfies the following equation

[TABLE]

where $b(\cdot,\cdot,\cdot,\cdot)$ and $\sigma(\cdot,\cdot,\cdot)$ are appropriate functions, $w_{i}(\cdot)$, $1\leq i\leq N$, are independent ${\mathbb{R}}^{d}$-valued Brownian motions, $\alpha(\cdot)$ is a continuous-time Markov chain independent of the Brownian motions $w_{i}(\cdot)$, $\delta_{x}(\cdot)$ denotes the Dirac measure centered at $x$ for each $x\in{\mathbb{R}}^{d}$, and $u_{i}(\cdot)$, $1\leq i\leq N$, is the control of player $i$ taking values in a compact subset of another Euclidean space ${\mathbb{R}}^{d_{1}}$. Player $i$, $1\leq i\leq N$, wishes to minimize its own cost

[TABLE]

where $R(\cdot,\cdot,\cdot)$ is a running cost function and the expectation is taken with $x_{j}(0)=x_{j}$. To obtain low complexity strategies, consistent mean-field approximations provide a powerful approach. Consequently, each player only needs to know its own state information and the aggregate effect of the overall population, which may be pre-computed off-line. A crucial step of this approach is to approximate the instantaneous measure ${1\over N}\sum_{j=1}^{N}\delta_{x_{j}(t)}$ of the processes under consideration by a stationary measure as $N\to\infty$. In order to take such a step, one needs to demonstrate that the system indeed possesses such a limit measure. The law of large numbers of the corresponding systems provides the existence of this limit and helps to characterize it. With the motivation for finding optimal strategies for mean-field models with $N$ players and Markovian switching, this work establishes the laws of large numbers for such systems and paves a way for solving the underlying problem.

Regarding law of large numbers, it is worth mentioning that since the pioneering works of Kac [18] and McKean [25], many important results have been obtained for investigating the time evolution of stochastic systems with long range weak interactions. Many variants of such systems have also been examined. For example, in [20], limit theorems were established for a model in which there is a common space noise process that influences the dynamics of each particle. Law of large numbers in a setting where particle evolution depends on independent jumps and switching processes were studied in [1, 27] and [14], respectively. In [8], law of large numbers was studied for a model where the noises are correlated.

One of the novel features of this paper is the limit of the empirical measures is not deterministic but a random measure that depends on the history of the Markovian switching process. In addition, the stochastic McKean-Vlasov equation in the limit is driven by martingales associate with the Markov switching process. As a consequence, there is a main difficulty to characterize the limit using the martingale problem formulation as in [8, 13, 14]. To overcome this difficulty, we use a new approach. Different from the classical work, we characterize the limit as the unique solution to a stochastic McKean-Vlasov equation with Markovian switching, which is represented by the conditional distribution of the solution to a McKean-Vlasov stochastic differential equation with a Markovian switching given the history of the switching process. In contrast, for the problem treated in [14], each particle possess its own switching process and the limit is represented as the distribution of solution to a McKean-Vlasov stochastic differential equation. We note that in [20], Kurtz and Xiong treated interacting particles. In their paper, there is a common space-time Gaussian white noise. They obtained law of large numbers with the conditional distribution in the limit. In their case, the martingale problem approach cannot be used either. Nevertheless, their model contains infinitely many exchangeable particles. Thus ergodic theory can be applied to the system, whereas in our case, we no longer have infinitely many exchangeable particles thus we cannot carry out the study by directly applying the existing ergodic theory.

In networked systems, the discrete component, namely, the random switching process often has a rather large state space. The transition among the states are not of the same speed. Some of them vary rapidly, whereas the others evolve slowly. As illustrated in [29] (see also [36]), the applications demand the consideration of the so-called nearly decomposable structures. Here nearly decomposable is understood in the sense that the switching among different subspaces are still possible although they appear relatively infrequently. Consequently, the large state space is naturally divisible into a number of subspaces so that the transitions in each subspace take place at a fast pace; the transitions from one subspace to another occur slowly. Such a situation leads to the modeling using two-time scales as in [36] by introducing a small parameter $\varepsilon>0$ into the systems. In this paper, we will also investigate this case. The goal is still to get laws of large numbers. However, in lieu of one parameter $N$, we have two parameters $N$ and $\varepsilon$. The limit is taken to be as $\varepsilon\to 0$, $N\to\infty$, and $(1/\varepsilon)\wedge N\to\infty$.

The rest of the paper is arranged as follows. Section 2 presents the formulation of the problem that we wish to study. Section 3 collects a number of preliminary results of interacting particle systems with Markovian switching. Section 4 demonstrates the law of large numbers for the systems. Section 5 examines systems in which the random switching displays two-time-scale behavior. Finally, an appendix containing the proofs of some technical lemmas is placed at the end of the paper. We remark that this paper is devoted to convergence in the form of law of large numbers. The rates of convergence is an interesting topic for future research. In the literature, some of such attempts can be found in [19] using an analytic approach and [17] using martingale-type estimates. For problems under the setting of this paper, because of the conditional distributions usage, careful thoughts and considerations are needed to treat the rate of convergence issue.

2 Formulation

We consider a mean-field system of $N$ particles (with $N$ being a large number), described by the following system of stochastic differential equations

[TABLE]

for $i=1,2,\ldots,N$, where $\delta_{x}(\cdot)$ denotes the Dirac measure centered at $x$ with $x\in\mathbb{R}^{d}$, $w_{1}(\cdot)$, $w_{2}(\cdot)$, $\ldots$, $w_{N}(\cdot)$ are $N$ independent $d$-dimensional standard Brownian motions, and $\alpha(\cdot)$ is a Markov chain taking values in a finite state space $\mathbb{S}=\{1,2,\ldots,m_{0}\}$ with a generator Q=\big{(}q_{i_{0}j_{0}}\big{)}_{i_{0},j_{0}\in\mathbb{S}} satisfying the following properties: $q_{i_{0}j_{0}}\geq 0$ for $i_{0}\neq j_{0}\in\mathbb{S}$ and $q_{i_{0}i_{0}}=-\sum_{j_{0}\neq i_{0}}q_{i_{0}j_{0}}$ for each $i_{0}\in{\mathbb{S}}$.

Throughout this paper, we assume that the Brownian motions $w_{i}(\cdot)$, $1\leq i\leq N$, and the Markov chain $\alpha(\cdot)$ are independent and defined on a common complete probability space $(\Omega,\mathcal{F},\mathbb{P})$. Note that the transition rule of the Markov chain $\alpha(t)$ satisfies

[TABLE]

for any pair $i_{0},j_{0}\in\mathbb{S}$. It is clear that $x_{i}(t)$ depends on $N$ in accordance with (2.1), but to simplify the notation, we omit the index $N$ in $x_{i}(t)$ in what follows.

Notation. Let $C_{b}(\mathbb{R}^{d})$ denote the space of bounded and continuous functions on $\mathbb{R}^{d}$ equipped with the usual supremum norm $\|\cdot\|$, $C^{k}_{b}(\mathbb{R}^{d})$ the space of all functions in $C_{b}(\mathbb{R}^{d})$ whose partial derivatives up to order $k$ are bounded and continuous, and $C^{k}_{c}(\mathbb{R}^{d})$ the space of functions whose partial derivatives up to order $k$ are continuous with compact support. Denote by $\mathscr{M}_{1}$ the space of all probability measures on $\mathbb{R}^{d}$. For $f\in C_{b}(\mathbb{R}^{d})$ and $\mu\in\mathscr{M}_{1}$, define $\langle\mu,f\rangle=\int_{\mathbb{R}^{d}}f(x)\mu(dx)$. We shall use the total variation metric $\|\cdot\|_{TV}$ and the bounded Lipschitz metric $\|\cdot\|_{BL}$ on $\mathscr{M}_{1}$ given as

[TABLE]

for $\mu,\eta\in\mathscr{M}_{1}$. It follows from [10] that $(\mathscr{M}_{1},\|\cdot\|_{BL})$ is a separable and complete metric space, which is topologically equivalent to the space of all probability measures on $\mathbb{R}^{d}$ equipped with the weak topology. Endow $\mathbb{S}$ with a metric $d_{\mathbb{S}}$ satisfying $d_{\mathbb{S}}(i_{0},i_{0})=0$ and $d_{\mathbb{S}}(i_{0},j_{0})=1$ if $i_{0}\neq j_{0}$ for $i_{0},j_{0}\in\mathbb{S}$. Define the following metric $d$ on the product space $\mathscr{M}_{1}\times\mathbb{S}$,

[TABLE]

For a metric space $E$, let $\mathcal{B}(E)$ be the Borel $\sigma$-field on $E$ and $\mathcal{P}(E)$ denote the space of all probability measures on \big{(}E,\mathcal{B}(E)\big{)} equipped with the weak convergence topology. Let $C([0,T],E)$ denote the space of all continuous functions $h:[0,T]\to E$ equipped with the supremum metric and $D([0,T],E)$ the space of all càdlàg functions $h:[0,T]\to E$ equipped with the usual Skorohod topology. Denote by $D_{f}([0,T],\mathbb{S})$ the subspace of $D([0,T],\mathbb{S})$ which contains all processes with finite jumps. Since $\mathbb{S}$ is a discrete set, $D_{f}([0,T],\mathbb{S})$ is a closed subset of $D([0,T],\mathbb{S})$. For a given $\mu\in\mathscr{M}_{1}$ and functions $f(\cdot,\cdot,\cdot)$ and $g(\cdot,\cdot)$ satisfying $f(\cdot,\cdot,i_{0})\in C_{b}(\mathbb{R}\times\mathbb{R}^{d})$ and $g(\cdot,i_{0})\in C_{b}(\mathbb{R}^{d})$ for each $i_{0}\in\mathbb{S}$, we define $\langle\mu,f(t,\cdot,i_{0})\rangle=\int_{\mathbb{R}^{d}}f(t,x,i_{0})\mu(dx)$ and $\langle\mu,g(\cdot,i_{0})\rangle=\int_{\mathbb{R}^{d}}g(x,i_{0})\mu(dx)$.

Let $\mathcal{B}(\mathbb{R}^{d})$ denote the usual Borel $\sigma$-field on $\mathbb{R}^{d}$. For any vector $x\in\mathbb{R}^{d}$ or matrix $A\in\mathbb{R}^{d\times d}$, $|x|$ and $|A|$ denote their usual norms in $\mathbb{R}^{d}$ and $\mathbb{R}^{d\times d}$, respectively, and $x^{\prime}$ and $A^{\prime}$ denote their transposes. In addition, the inner product of two vectors $x,y$ is denoted by $(x,y)$. In what follows, we frequently use two particular functions $\varphi(\cdot),\psi(\cdot):\mathbb{R}^{d}\to\mathbb{R}$ defined by $\varphi(x)=|x|$ and $\psi(x)=|x|^{2}$, respectively. For $t>0$, denote \mathcal{F}^{\alpha}_{t_{-}}=\sigma\big{\{}\alpha(s):0\leq s<t\big{\}} and

[TABLE]

For a random variable $\varsigma$ on \big{(}\Omega,\mathcal{F},\mathbb{P}\big{)}, we denote by

[TABLE]

in the sense that

[TABLE]

We make the following assumptions.

Assumption A.

• (A1)

For each $i_{0}\in\mathbb{S}$, $b(\cdot,\cdot,i_{0}):\mathbb{R}^{d}\times\mathscr{M}_{1}\to\mathbb{R}^{d}$ and $\sigma(\cdot,\cdot,i_{0}):\mathbb{R}^{d}\times\mathscr{M}_{1}\to\mathbb{R}^{d\times d}$ are Lipschitz continuous in that, there is a constant $L$ such that

[TABLE]

for all $x,y\in\mathbb{R}^{d}$ and $\mu,\eta\in\mathscr{M}_{1}$.

• (A2)

The $\mathbb{R}^{d}$-valued function $b(\cdot,\cdot,\cdot)$ satisfies

[TABLE]

for some constant $C$ and $\varphi:\mathbb{R}^{d}\to\mathbb{R}$, $\varphi(x)=|x|$ and the matrix-valued function $\sigma(\cdot,\cdot,\cdot)$ is bounded.

Note that in the above and throughout the paper, for notational simplicity, the same notion $|\cdot|$ is used to denote different norms in $\mathbb{R}^{d}$, or $(\mathbb{R}^{d})^{N}$, or $\mathbb{R}^{d\times d}$. It should, however, be clear from the context which norm is currently used.

It will be shown in the next section that under Assumption (A), for each fixed $N\geq 1$, system (2.1) has a unique solution \big{(}x_{1}(t),x_{2}(t),\ldots,x_{N}(t)\big{)}. For $N\geq 1$, $0\leq t\leq T$, and $A\in\mathcal{B}(\mathbb{R}^{d})$, define

[TABLE]

Then $\mu_{N}(t,\cdot)$ is a measured-valued process, taking value on the space $\mathscr{M}_{1}$ of probability measures on $\mathbb{R}^{d}$. We denote by $\mathscr{P}_{N}$ the induced probability measure of \big{(}\mu_{N}(\cdot),\alpha(\cdot)\big{)} on D\big{(}[0,T],\mathscr{M}_{1}\times\mathbb{S}\big{)}. It can be shown that $\mathscr{P}_{N}$ concentrates on the set C\big{(}[0,T],\mathscr{M}_{1}\big{)}\times D_{f}\big{(}[0,T],\mathbb{S}\big{)}, a closed subspace of D\big{(}[0,T],\mathscr{M}_{1}\times\mathbb{S}\big{)}. Using the notation mentioned thus far, in particular, (2.3) and (2.3), we proceed to derive the following main result. The proof is provided in Section 4, and some preliminary results are given in the next section as preparation.

Theorem 2.1**.**

Assume (A1), (A2), and

[TABLE]

where $\psi:\mathbb{R}^{d}\to\mathbb{R}$ with $\psi(x)=|x|^{2}$. Then \big{(}\mu_{N}(\cdot),\alpha(\cdot)\big{)} converges weakly to a process \big{(}\mu_{\alpha}(\cdot),\alpha(\cdot)\big{)}, where

[TABLE]

and $y(t)$, $0\leq t\leq T$, is the unique solution of the following stochastic differential equation

[TABLE]

where $\tilde{w}(\cdot)$ is a standard Brownian motion independent of $\alpha(\cdot)$.

As mentioned in the introduction, motivated by applications in networked systems where the random switching process often has a large state space and the transition among the states are not at the same speed, we also treat mean-field systems that capture different transition rates (slow and fast) of the switching process by using two-time scale approach. A parameter $\varepsilon$ will be used to depict the difference of transition speeds. It can be shown that the law of large numbers also holds true for this case under some mild conditions similar to those in Theorem 2.1. For clarity of presentation, the formulation of this case will be given in Section 5.

3 Preliminaries

In this section, we provide some preliminary results on weakly interacting systems with Markovian switching. For convenience, we first consider the general switching systems consisting of $N$-particles in ${\mathbb{R}}^{d}$ without the weak interaction assumption. These systems can be formulated as switching diffusion processes in the larger space $({\mathbb{R}}^{d})^{N}$. Weakly interacting systems of $N$-particles is then presented in Section 3.2 as a special case.

3.1 General $N$-Particle System with Markovian Switching

Let $x_{0,i},0\leq i\leq N$ be $\mathbb{R}^{d}$-valued random variables defined on $(\Omega,\mathcal{F},P)$ that are independent of $w_{i}(\cdot),1\leq i\leq N,$ and $\alpha(\cdot)$. Assume that $\mathbb{E}|x_{0,i}|^{2}<\infty$ for $1\leq i\leq N$. Consider the following stochastic differential equations with Markovian switching

[TABLE]

where \underline{x}(t)=\underline{x}_{N}(t)=\big{(}x_{1}(t),x_{2}(t),\ldots,x_{N}(t)\big{)}\in\big{(}\mathbb{R}^{d}\big{)}^{N}, \underline{b}_{i}(\cdot,\cdot,\cdot):\mathbb{R}\times\big{(}\mathbb{R}^{d}\big{)}^{N}\times\mathbb{S}\to\mathbb{R}^{d}, \underline{\sigma}_{i}(\cdot,\cdot,\cdot):\mathbb{R}\times\big{(}\mathbb{R}^{d}\big{)}^{N}\times\mathbb{S}\to\mathbb{R}^{d\times d} are vector-valued functions, $w_{1}(\cdot),w_{2}(\cdot),\ldots,w_{N}(\cdot)$ are $\mathbb{R}^{d}$-valued independent standard Brownian motions, and $\alpha(\cdot)$ is a Markov chain with the state space $\mathbb{S}$ and generator Q=\big{(}q_{i_{0},j_{0}}\big{)}_{i_{0},j_{0}\in\mathbb{S}} given as in the previous section. Assume that for $1\leq i\leq N$, $i_{0}\in\mathbb{S}$ and $0\leq t\leq T$, \underline{b}_{i}\big{(}t,\cdot,i_{0}\big{)} and \underline{\sigma}_{i}\big{(}t,\cdot,i_{0}\big{)} satisfy the following Lipschitz and linear growth conditions

[TABLE]

[TABLE]

for any $\underline{x},\underline{y}\in(\mathbb{R}^{d})^{N}$, where $K$ is a positive constant. It follows from Theorem 3.3.13 [24] that under (3.2) and (3.3), the system (3.1) has a unique solution.

By virtue of [24, 37], for a function $V(\cdot,\cdot,\cdot):[0,T]\times(\mathbb{R}^{d})^{N}\times\mathbb{S}\to\mathbb{R}$ such that for each $i_{0}\in\mathcal{M}$, V(\cdot,\cdot,i_{0})\in C^{1,2}\big{(}[0,T]\times(\mathbb{R}^{d})^{N}\big{)}, the generator of the general system of $N$ particles is defined by

[TABLE]

for \big{(}t,\underline{x},i_{0}\big{)}\in[0,T]\times(\mathbb{R}^{d})^{N}\times\mathbb{S}, where $\nabla_{x_{i}}$ denotes the gradient with respect to $x_{i}$, and \underline{a}_{i}\big{(}t,\underline{x},i_{0}\big{)}=\underline{\sigma}_{i}\big{(}t,\underline{x},i_{0}\big{)}\underline{\sigma}^{\prime}_{i}\big{(}t,\underline{x},i_{0}\big{)}\in\mathbb{R}^{d\times d} for each $1\leq i\leq N$.

Associated with each pair $(i_{0},j_{0})\in{\mathbb{S}}\times{\mathbb{S}}$, $i_{0}\neq j_{0}$, the states of the Markov chain $\alpha(\cdot)$, define

[TABLE]

where ${1\!\!1}$ denotes the usual zero-one indicator function. It follows from Lemma IV.21.12 [28] that the process $M_{i_{0}j_{0}}(t)$, $0\leq t\leq T$, defined by

[TABLE]

is a purely discontinuous and square integrable martingale with respect to $\mathcal{F}^{N,\alpha}_{t}$, which is null at the origin. The processes $[M_{i_{0}j_{0}}](t)$ and $\langle M_{i_{0}j_{0}}\rangle(t)$ are respectively its optional and predictable quadratic variations. For convenience, we define M_{i_{0}i_{0}}(t)=\big{[}M_{i_{0}i_{0}}\big{]}(t)=\big{\langle}M_{i_{0}i_{0}}\big{\rangle}(t)=0 for each $i_{0}\in\mathbb{S}$. From the definition of optional quadratic covariations (see Section 1.8 in [22]) we have the following orthogonality relation :

[TABLE]

For any function $V(\cdot,\cdot,\cdot):[0,T]\times(\mathbb{R}^{d})^{N}\times\mathbb{S}\to\mathbb{R}$ such that V(\cdot,\cdot,i_{0})\in C^{1,2}\big{(}[0,T]\times(\mathbb{R}^{d})^{N}\big{)} for each $i_{0}\in\mathcal{M}$, we have the following Itô formula

[TABLE]

It can be seen that we have two martingales. One of them is due to the Brownian motion, whereas the other is resulted from the jump process.

3.2 $N$-Particle Mean-Field Model with Markovian Switching

For \underline{x}=\big{(}x_{1},x_{2},\ldots,x_{N}\big{)}\in(\mathbb{R}^{d})^{N}, denote the associated empirical probability measure $\delta_{\underline{x}}$ on $\mathscr{M}_{1}$ by $\delta_{\underline{x}}={1\over N}\sum_{j=1}^{N}\delta_{x_{j}}$. Consider the system of $N$ particles \underline{x}(t)=\big{(}x_{1}(t),x_{2}(t),\ldots,x_{N}(t)\big{)} described by the mean-field model with Markovian switching

[TABLE]

where

[TABLE]

It is clear that this is a special case of the $N$-particle system given by (3.1) with

[TABLE]

for \big{(}t,\underline{x},i_{0}\big{)}\in[0,T]\times(\mathbb{R}^{d})^{N}\times\mathbb{S}.

Note that $|\underline{x}|^{2}=\sum_{i=1}^{N}|x_{i}|^{2}$ implies \big{\langle}\delta_{\underline{x}},\varphi\big{\rangle}={1\over N}\sum_{i=1}^{N}|x_{i}|\leq{1\over\sqrt{N}}|\underline{x}| for any $\underline{x}\in(\mathbb{R}^{d})^{N}$ and that

[TABLE]

Under Assumption (A), for $b(\cdot,\cdot,\cdot)$ and $\sigma(\cdot,\cdot,\cdot)$, one can easily prove that the functions $\underline{b}_{i}$ and $\underline{\sigma}_{i}$, $1\leq i\leq N$, defined above satisfy the Lipschitz and linear growth conditions (3.2) and (3.3). This implies that system (2.1) has a unique solution. The following lemma reveals the moment boundedness of the system. In order to keep the continuity of the presentation, its proof is relegated to the Appendix.

Lemma 3.1**.**

Assume (A1), (A2), and that \sup_{N\geq 1}\mathbb{E}\big{\langle}\mu_{N}(0),\psi\big{\rangle}<\infty where $\psi(x)=|x|^{2}$ for $x\in\mathbb{R}^{d}$. Then for positive numbers $T$ and $p$, $p\leq 1$, there is a constant $C$ independent of $N$ such that

[TABLE]

and for $0\leq s\leq t\leq T$,

[TABLE]

For f(\cdot,i_{0})\in C^{2}_{c}\big{(}\mathbb{R}^{d}\big{)}, $i_{0}\in\mathbb{S}$, and \big{(}x,\mu,i_{0}\big{)}\in\mathbb{R}^{d}\times\mathscr{M}_{1}\times\mathbb{S} denote the operator

[TABLE]

where

[TABLE]

Let $F(\cdot,\cdot,\cdot):\mathbb{R}\times(\mathbb{R}^{d})^{N}\times\mathbb{S}\to\mathbb{R}$ be a function such that

[TABLE]

for some functions f(\cdot,\cdot,i_{0})\in C^{1,2}\big{(}[0,T]\times\mathbb{R}^{d}\big{)}, $i_{0}\in\mathbb{S}$. For $\underline{b}_{i}$ and $\underline{\sigma}_{i}$ defined as in (3.9), and $\mathcal{L}_{N}$ and $\mathcal{L}$ defined as in (3.4) and (3.12), respectively, we have

[TABLE]

For each element \varsigma\in D_{f}\big{(}[0,T],\mathbb{S}\big{)} that represents a sample path of the switching process $\alpha(t)$, $0\leq t\leq T$, we denote the corresponding sample path of the associated martingale by a similar way to (3.5) and (3.6) as follow

[TABLE]

where

[TABLE]

Since the sample paths of $\alpha(\cdot)$ are in D_{f}\big{(}[0,T],\mathbb{S}\big{)}, for simplicity, we define M_{i_{0}j_{0}}^{\varsigma}(t)=\big{[}M_{i_{0}j_{0}}^{\varsigma}\big{]}(t)=\big{\langle}M_{i_{0}j_{0}}^{\varsigma}\big{\rangle}(t)=0 for \varsigma\in D\big{(}[0,T],\mathbb{S}\big{)}\backslash D_{f}\big{(}[0,T],\mathbb{S}\big{)} and $0\leq t\leq T$.

For \big{(}\underline{x}(\cdot),\alpha(\cdot)\big{)}\in D\big{(}[0,T],(\mathbb{R}^{d})^{N}\times\mathbb{S}\big{)}, we define the mapping $e_{N}$ by

[TABLE]

Denote by $\mathbb{P}_{N}$ the induced probability measure of the system \big{(}x_{1}(\cdot),x_{2}(\cdot),\ldots,x_{N}(\cdot),\alpha(\cdot)\big{)}, the solution to (2.1), on D\big{(}[0,T],(\mathbb{R}^{d})^{N}\times\mathbb{S}\big{)}. It follows that

[TABLE]

We have the following lemma.

Lemma 3.2**.**

Under the assumption of Lemma 3.1, the following statements hold:

• (i)

For f(\cdot,\cdot,i_{0})\in C^{1,2}\big{(}[0,T]\times\mathbb{R}^{d}\big{)}, $i_{0}\in\mathbb{S}$, and \big{(}\eta,\varsigma\big{)}\in D\big{(}[0,T],\mathscr{M}_{1}\times\mathbb{S}\big{)} denote

[TABLE]

*Then $M_{f}(t)$ is a continuous *$\mathscr{P}_{N}-$martingale.

• (ii)

*For f(\cdot,\cdot,i_{0}),g(\cdot,\cdot,i_{0})\in C^{1,2}\big{(}[0,T]\times\mathbb{R}^{d}\big{)}, $i_{0}\in\mathbb{S}$, the quadratic variational process of the *$\mathscr{P}_{N}-$martingales $M_{f}$ and $M_{g}$ has the form

[TABLE]

for $0\leq t\leq T$, where $\nabla$ denotes the gradient with respect to the space variables and $(\cdot,\cdot)$ is the inner product in $\mathbb{R}^{d}$.

Proof. Let $F(\cdot,\cdot,\cdot),G(\cdot,\cdot,\cdot):\mathbb{R}\times(\mathbb{R}^{d})^{N}\times\mathbb{S}\to\mathbb{R}$ satisfy

[TABLE]

for functions f(\cdot,\cdot,i_{0}),g(\cdot,\cdot,i_{0})\in C^{1,2}\big{(}[0,T]\times\mathbb{R}^{d}\big{)}, $i_{0}\in\mathbb{S}$. We put

[TABLE]

and define $M^{G}(t)$ with $F$ replaced by $G$ and $0\leq t\leq T$. In view of the Itô formula (3.8),

[TABLE]

is a continuous $\mathbb{P}_{N}$-martingle. Since the Brownian motions $w_{1}(\cdot),w_{2}(\cdot),\ldots,w_{N}(\cdot)$ are independent, from (3.16), we obtain

[TABLE]

One can easily verify the following identities

[TABLE]

Since $\mathscr{P}_{N}=\mathbb{P}_{N}\circ e_{N}^{-1}$, a combination of the above facts implies the assertions (i) and (ii).$\qquad\Box$

4 Law of Large Numbers for Mean-Field Models with Markovian Switching

In this section, we present the proof of Theorem 2.1, one of the main result of the paper, establishing the law of large numbers for the mean-field systems with Markovian switching. ​We use the martingale approach. The weak compactness of the sequence $\{(\mu_{N},\alpha_{N})\}_{N\geq 1}$ is established in Section 4.1. Its limit is characterized in Section 4.2.

4.1 Weak Compactness of \big{\{}\big{(}\mu_{N}(\cdot),\alpha(\cdot)\big{)}\big{\}}_{N\geq 1}

Lemma 4.1**.**

Under the assumptions of Theorem 2.1, for each $\delta>0$ there exists a compact set $K_{\delta}$ in \big{(}\mathscr{M}_{1},\|\cdot\|_{BL}\big{)} such that

[TABLE]

where K^{\delta}_{\delta}=\big{\{}\mu\in\mathscr{M}_{1}:\inf_{\eta\in K_{\delta}}\|\mu-\eta\|_{BL}<\delta\big{\}}.

Proof. For each $\lambda>0$, denote B_{\lambda}^{c}=\big{\{}x\in\mathbb{R}^{d}:|x|>\lambda\big{\}} and H_{\lambda}=\big{\{}\mu\in\mathscr{M}_{1}:\mu(B^{c}_{\lambda})=0\big{\}}. Because \big{(}\mathscr{M}_{1},\|\cdot\|_{BL}\big{)} is topologically equivalent to $\mathcal{P}(\mathbb{R}^{d})$ (see Theorem 12 [10]), by Prohorov theorem, $H_{\lambda}$ is relatively compact in \big{(}\mathscr{M}_{1},\|\cdot\|_{BL}\big{)}. For $0\leq t\leq T$,

[TABLE]

Let $C$ be the constant given in (3.11). It follows from (4.1) and (3.11) that

[TABLE]

For a fixed $\delta>0$, we can choose $\lambda=\lambda(\delta)$ large enough such that ${C\over\lambda^{2}\delta}\leq\delta$. Take $K_{\delta}=\bar{H}_{\lambda(\delta)}$ which is compact in \big{(}\mathscr{M}_{1},\|\cdot\|_{BL}\big{)}. For all $N\geq 1$ we have

[TABLE]

This completes the proof. $\qquad\Box$

Lemma 4.2**.**

Under the assumptions of Theorem 2.1, for each positive integer $N$ and $\delta>0$, there exists a random variable $\gamma_{N}(\delta)\geq 0$ such that

[TABLE]

for all $0\leq t\leq T-\delta$. Furthermore,

[TABLE]

Proof. By the definition of the norm $\|\cdot\|_{BL}$ and Cauchy-Schwarz inequality,

[TABLE]

for any integer $N$, and real numbers $t,\delta$ satisfying $0\leq t,t+\delta\leq T$. Therefore, by the Dynkin formula,

[TABLE]

It follows from the right-hand side and then the left-hand side of (3.11) that for $s\geq t$,

[TABLE]

Thus, (4.2) implies that

[TABLE]

As a consequence, by Cauchy-Schwarz inequality,

[TABLE]

This inequality and Lemma 3.1 conclude the proof by taking

[TABLE]

The proof is complete. $\qquad\Box$

According to Lemmas 4.1 and 4.2 we obtain the following Proposition.

Proposition 4.3**.**

The sequence \big{\{}\big{(}\mu_{N}(\cdot),\alpha(\cdot)\big{)},N\geq 1\big{\}} is weakly compact in the topology of weak convergence of probability measure on D\big{(}[0,T],\mathscr{M}_{1}\times\mathbb{S}\big{)}.

Proof. Let $K_{\delta}$ be a compact subset of \big{(}\mathscr{M}_{1},\|\cdot\|_{BL}\big{)} as in Lemma 4.1 and denote $L_{\delta}=K_{\delta}\times\mathbb{S}$. By the compactness of the space $(\mathbb{S},d_{\mathbb{S}})$, $L_{\delta}$ is also a compact set in \big{(}\mathscr{M}_{1}\times\mathbb{S},d\big{)} and $L_{\delta}^{\delta}=K_{\delta}^{\delta}\times\mathbb{S}$. In view of Lemma 4.1, we have

[TABLE]

Since $\alpha(\cdot)$ is a Markov chain, there is a constant $C$ such that \mathbb{E}\big{(}d_{\mathbb{S}}(\alpha(t+\delta),\alpha(t))\big{|}\mathcal{F}^{\alpha}_{t}\big{)}\leq C\delta for $\delta>0$ and $0\leq t\leq T$. Therefore, it follows from the definition of metric $d$ and Lemma 4.2 that for each integer $N$ and positive number $\delta$ there exists a random variable $\gamma_{N}(\delta)$ such that

[TABLE]

where $\gamma_{N}(\delta)$ defined in Lemma 4.2 satisfies

[TABLE]

Combining (4.4), (4.5), and (4.6), the Proposition follows by virtue of [11, Theorem 3.8.6]. $\qquad\Box$

4.2 Characterization of Limit

Next, we proceed to characterize \big{(}\mu_{\alpha}(\cdot),\alpha(\cdot)\big{)}, the limit of the sequence \big{\{}(\mu_{N}(\cdot),\alpha(\cdot))\big{\}}_{N\geq 1}. We have the following lemma.

Theorem 4.4**.**

Assume (A1), (A2), and that \sup_{N\geq 1}\mathbb{E}\big{\langle}\mu_{N}(0),\psi\big{\rangle}<\infty. Denote by $\mathscr{P}$ the limit of an arbitrary weakly convergent subsequence of $\mathscr{P}_{N}$. Then for $\mathscr{P}$-almost all \big{(}\eta(\cdot),\varsigma(\cdot)\big{)}\in D\big{(}[0,T],\mathscr{M}_{1}\times\mathbb{S}\big{)},

[TABLE]

holds for all test functions f(\cdot,i_{0})\in C^{2}_{c}\big{(}\mathbb{R}^{d}\big{)}, $i_{0}\in\mathbb{S}$, and $0\leq t\leq T$.

Proof. Let $\mathscr{P}_{N_{k}}$ be a subsequence of $\mathscr{P}_{N}$ that converges weakly to a probability measure $\mathscr{P}$ on D\big{(}[0,T],\mathscr{M}_{1}\times\mathbb{S}\big{)} as $k\to\infty$. It suffices to prove that (4.7) holds $\mathscr{P}-$almost surely for each test function f(\cdot,i_{0})\in C^{2}_{c}\big{(}\mathbb{R}^{d}\big{)}, $i_{0}\in\mathbb{S}$. Note from Lemma 3.2 that for each $N$,

[TABLE]

is a continuous square integrable $\mathscr{P}_{N}$-martingale with quadratic variational process

[TABLE]

By virtue of Assumption (A2) and Lemma 3.1, there exists a constant $C$ independent of $N$ such that

[TABLE]

Note that for any $N$, and $0\leq t\leq T$, $(\mu_{N}(\cdot),\alpha(\cdot))$ concentrates on the set $\mathscr{S}$ denoted by

[TABLE]

with probability $1$, i.e., \mathscr{P}_{N}\big{(}\mathscr{S}\big{)}=1 for any $N\geq 1$. For $n\geq 1$, denote

[TABLE]

It is clear that $\mathscr{S}_{n}\subset\mathscr{S}_{n+1}$ for every $n\geq 1$ and $\mathscr{S}=\cup_{n\geq 1}\mathscr{S}_{n}$. Since $\mathbb{S}$ is a discrete set and $f\in C^{2}_{c}(\mathbb{R}^{d}\times\mathbb{S})$, the set \big{\{}(\eta,\varsigma)\in\mathscr{S}_{n}:|M_{f}(t)|\leq\delta\big{\}} is closed in $D([0,T],\mathscr{M}_{1}\times\mathbb{S})$ for each $n\geq 1$. Thus, by Portmanteau Theorem, we have

[TABLE]

By Doob’s submartingale inequality and (4.9),

[TABLE]

Combining (4.10) and (4.11) yields \mathscr{P}\big{(}|M_{f}(t)|=0\big{)}=1, which implies $M_{f}(t)=0$ $\mathscr{P}$-a.s. The theorem therefore follows from (4.8).$\qquad\Box$

To proceed, we need a result from [27]. Assume the following conditions hold.

• (B1)

$\hat{b}(\cdot,\cdot):\mathbb{R}^{d}\times\mathscr{M}_{1}\to\mathbb{R}^{d}$ and $\hat{\sigma}(\cdot,\cdot):\mathbb{R}^{d}\times\mathscr{M}_{1}\to\mathbb{R}^{d\times d}$ are Lipschitz continuous in that, there is a constant $L$ such that

[TABLE]

for all $x,y\in\mathbb{R}^{d}$ and $\mu,\eta\in\mathscr{M}_{1}$.

• (B2)

The $\mathbb{R}^{d}$-valued function $\hat{b}(\cdot,\cdot)$ satisfies

[TABLE]

for some constant $C$ and the $\mathbb{R}^{d\times d}$-valued function $\hat{\sigma}(\cdot,\cdot)$ is bounded.

For $\mu\in\mathscr{M}_{1}$ and $f\in C^{2}_{c}(\mathbb{R}^{d})$ denote

[TABLE]

As a consequence of Lemma 9 and equation (8.2) in [27], we have the following theorem.

Theorem 4.5**.**

Assume (B1) and (B2). Then the equation

[TABLE]

has a unique solution $\mu(t)=\mathscr{L}(z(t))$ in D\big{(}[0,T],\mathscr{M}_{1}\big{)} that is the distribution of the unique solution of

[TABLE]

and $\tilde{w}(\cdot)$ is a standard Brownian motion.

We are now in a position to present a result on stochastic McKean-Vlasov equations with Markovian switching.

Theorem 4.6**.**

Assume (A1) and (A2). Then the system of integral equations

[TABLE]

where $0\leq t\leq T$ and $f(\cdot,i_{0})\in C^{2}_{c}(\mathbb{R}^{d})$ for each $i_{0}\in\mathbb{S}$, has a unique solution in D\big{(}[0,T],\mathscr{M}_{1}\big{)}. Moreover, this solution equals \mathscr{L}\big{(}y(t)\big{|}\mathcal{F}^{\alpha}_{t_{-}}\big{)} for all $0\leq t\leq T$, where $y(t)$ is the unique solution of

[TABLE]

where $\tilde{w}(\cdot)$ is a standard Brownian motion independent of $\alpha(\cdot)$.

Since the proof of this theorem is rather long, we give a brief explanation of the main idea. First observe that (4.12) is a special case of (4.13) for the mean-field models with the usual diffusion (i.e., without switching process). To proceed with the case involving Markovian switching, we use Theorem 4.5 to deal with equation (4.13) in the time intervals between the jumps of the Markov chain $\alpha(\cdot)$. We then consider (4.13) at any jump time point by “gluing” the solutions between jump times of the Markov chain and show that the solution obtained in this way indeed satisfies all the requirements.

Proof of Theorem 4.6. The proof is divided into several steps.

Step 1: Show that for each $0<r\leq T$ and \varsigma(\cdot)\in D_{f}\big{(}[0,r],\mathbb{S}\big{)}, there exists a unique solution \eta(\cdot)\in D\big{(}[0,r],\mathscr{M}_{1}\big{)} to the equation

[TABLE]

where $0\leq t\leq r$ and $f(\cdot,i_{0})\in C^{2}_{c}(\mathbb{R}^{d})$ for each $i_{0}\in\mathbb{S}$.

Denote $t_{0}=0$, $t_{n+1}=\inf\{t>t_{n}:\varsigma(t)\neq\varsigma(t_{-})\}$ and $\iota_{n}=\varsigma(t_{n})\in\mathbb{S}$ for $n\geq 0$. For each $i_{0}\in\mathbb{S}$, $x\in\mathbb{R}^{d}$ and $\mu\in{\mathscr{M}}_{1}$, denote $\hat{b}_{i_{0}}(x,\mu)=b(x,\mu,i_{0})$, $\hat{\sigma}_{i_{0}}(x,\mu)=\sigma(x,\mu,i_{0})$, and $\hat{a}_{i_{0}}(x,\mu)=\hat{a}(x,\mu,i_{0})$. In addition, for $\mu\in{\mathscr{M}}_{1}$, $f\in C^{2}_{c}(\mathbb{R}^{d})$ denote

[TABLE]

Then

[TABLE]

Next, we show that on each interval $[t_{k-1},t_{k}]$, $k\geq 1$, a solution to (4.14) satisfies equation (4.12) in Theorem 4.5 with the operator $\hat{\mathcal{L}}_{\iota_{k-1}}$. That is, for $t_{k-1}\leq t\leq t_{k}$,

[TABLE]

First, we consider the case $k=1$. For $t_{0}\leq t<t_{1}$ we have M^{\varsigma}_{i_{0}j_{0}}(t)=-\big{\langle}M^{\varsigma}_{\iota_{0}j_{0}}\big{\rangle}(t)=-q_{\iota_{0}j_{0}}t if $j_{0}\neq i_{0}=\iota_{0}$, and $M^{\varsigma}_{i_{0}j_{0}}(t)=0$ if $i_{0}\neq\iota_{0}$. Thus it follows from (4.14) and (4.15) that for any $f\in C^{2}_{c}(\mathbb{R}^{d}\times\mathbb{S})$ and $0<t<t_{1}$,

[TABLE]

By taking $t=t_{1}$ in (4.14) and noting that \big{\langle}M^{\varsigma}_{\iota_{0}j_{0}}\big{\rangle}(t_{1})=q_{\iota_{0}j_{0}}t_{1} for $j_{0}\neq\iota_{0}$ and \big{[}M^{\varsigma}_{\iota_{0}\iota_{1}}\big{]}(t_{1})=1, we obtain

[TABLE]

which implies that (4.17) also holds for $t=t_{1}$. In view of Theorem 4.5, for any $0\leq s\leq t_{1}$, $\eta(s)=\mathscr{L}(z_{1}(s))$, which is the unique solution to the following stochastic differential equation

[TABLE]

Likewise, we can show that $\eta(t)$ satisfies (4.16) for any $t_{k-1}\leq t\leq t_{k}$. Hence, according to Theorem 4.5 again, $\eta(s)=\mathscr{L}(z_{k}(s))$ which is the unique solution to the following equation

[TABLE]

As a consequence, for $0\leq s\leq r$, $\eta(s)=\mathscr{L}(z(s))$ is the unique solution to the equation

[TABLE]

It is clear that \eta(\cdot)\in C\big{(}[0,r],\mathscr{M}_{1}\big{)}. This completes the Step 1.

Since for each $r>0$, $\mu_{N}(r)$ depends on the history of the switching process $\alpha(t)$ for $t\in[0,r)$, it is more convenient to consider equation (4.14) for $0\leq t<r$. Similar to Step 1, that we can define a mapping \Lambda_{r}:D_{f}\big{(}[0,r),\mathbb{S}\big{)}\to D\big{(}[0,r),\mathscr{M}_{1}\big{)} that maps each \varsigma(\cdot)\in D_{f}\big{(}[0,r),\mathbb{S}\big{)} to the unique solution \eta(\cdot)\in D\big{(}[0,r),\mathscr{M}_{1}\big{)} to equation (4.14) for $0\leq t<r$. It also follows from Step 1, (4.17), and (4.18) that the unique solution \eta(\cdot)\in D\big{(}[0,r],\mathscr{M}_{1}\big{)} to equation (4.14) for $0\leq t\leq r$ satisfies

[TABLE]

for any value of $\varsigma(r)$.

For each $0<r_{1}\leq r_{2}$, denote the truncation mappings \Pi_{r_{2},r_{1}}^{\mathbb{S}}:D\big{(}[0,r_{2}),\mathbb{S}\big{)}\to D\big{(}[0,r_{1}),\mathbb{S}\big{)} and \Pi_{r_{2},r_{1}}^{\mathscr{M}_{1}}:D\big{(}[0,r_{2}),\mathscr{M}_{1}\big{)}\to D\big{(}[0,r_{1}),\mathscr{M}_{1}\big{)} by

[TABLE]

for all \varsigma\in D\big{(}[0,r_{2}),\mathbb{S}\big{)}, \eta\in D\big{(}[0,r_{2}),\mathscr{M}_{1}\big{)} and $0\leq s<r_{1}$. Then we have the following lemma that shows the continuity and consistency of the mapping $\Lambda_{r}$. To keep the continuity of the flow of presentation, we relegate the proof to the Appendix.

Lemma 4.7**.**

The following assertions hold.

• (i)

The mapping \Lambda_{r}:D_{f}\big{(}[0,r),\mathbb{S}\big{)}\to C\big{(}[0,r),\mathscr{M}_{1}\big{)} is continuous for any $0<r\leq T$.

• (ii)

For any $0<r\leq T$ and \varsigma\in D_{f}\big{(}[0,r_{2}),\mathbb{S}\big{)}, the following consistent identity holds

[TABLE]

Next, for $r>0$, we define \alpha_{r_{-}}:\Omega\to D_{f}\big{(}[0,r),\mathbb{S}\big{)} by $\alpha_{r_{-}}(s,\omega)=\alpha(s,\omega)$ for $0\leq s<r$ and $\omega\in\Omega$. Let $y_{r}(\cdot)$ be the solution to the following equation

[TABLE]

with $y_{r}(0)=y(0)$ such that $\mathscr{L}(y(0))=\mu_{0}$. It follows from Lemma 4.7 that $y_{r_{1}}(s)=y_{r_{2}}(s)$ for $0<s<r_{1}<r_{2}$. Hence, we can define $y(s)=y_{r}(s)$ for $0\leq s<r$ and obtain

[TABLE]

Step 2: Prove that \Lambda_{r}(\alpha_{r_{-}})(r_{-})=\mathscr{L}\big{(}y(r)\big{|}\mathcal{F}^{\alpha}_{r_{-}}\big{)} for each $r>0$.

First, we show \Lambda_{r}(\alpha_{r_{-}})(r_{-})=\mathscr{L}\big{(}y(r_{-})\big{|}\mathcal{F}^{\alpha}_{r_{-}}\big{)} for each $0<r\leq T$. Note that according to Step 1, on the set \alpha_{r_{-}}=\varsigma\in D_{f}\big{(}[0,r),\mathbb{S}\big{)} we have \Lambda_{r}(\alpha_{r_{-}})(s)=\mathscr{L}\big{(}y(s)\big{)} for any $0\leq s<r$. Therefore, \Lambda_{r}(\alpha_{r_{-}})(r_{-})=\mathscr{L}\big{(}y(r_{-})\big{)}. As a consequence, for any f\in C_{b}\big{(}\mathbb{R}^{d}\big{)},

[TABLE]

Next, let $\mathscr{B}^{\mathbb{S}}_{r_{-}}$ and $\mathscr{B}^{\mathscr{M}_{1}}_{r}$ be the Borel $\sigma-$fields on D\big{(}[0,r),\mathbb{S}\big{)} and D\big{(}[0,r),\mathscr{M}_{1}\big{)}, respectively. For each $0<s\leq r\leq T$ denote the mappings \pi_{r,s}^{\mathbb{S}}:D\big{(}[0,r),\mathbb{S}\big{)}\to\mathbb{S} and \pi_{r,s}^{\mathscr{M}_{1}}:D\big{(}[0,r),\mathscr{M}_{1}\big{)}\to\mathscr{M}_{1} by

[TABLE]

Then we have

[TABLE]

Hence,

[TABLE]

Since \Lambda_{r}:D_{f}\big{(}[0,r),\mathbb{S}\big{)}\to C\big{(}[0,r),\mathscr{M}_{1}\big{)} is continuous, $\Lambda_{r}(\alpha_{r_{-}})(r_{-})$ is \sigma\big{\{}\alpha_{r_{-}}\big{\}}-measurable. In addition, \big{(}\mathscr{M}_{1},\|\cdot\|_{BL}\big{)} is equivalent to the space $\mathcal{P}(\mathbb{R}^{d})$ equipped with the weak topology. Therefore, it follows from (4.22) and (4.23) that

[TABLE]

This implies \Lambda_{r}(\alpha_{r_{-}})(r_{-})=\mathscr{L}\big{(}y(r_{-})\big{|}\mathcal{F}^{\alpha}_{r_{-}}\big{)}. By part (ii) of Lemma 4.7, $\Lambda_{r}(\alpha_{r_{-}})(s)=\Lambda_{T}(\alpha_{T_{-}})(s)$ for any $0<s<r\leq T$. Thus, by Lemma 4.7(i), for each $0<s<T$, we obtain

[TABLE]

Taking $r=T$ in (4.21) we have obtain

[TABLE]

Similar to Lemma 4.7(i), we can show that solution $y(s)$ of (4.24) is continuous. This gives \mathscr{L}\big{(}y(s)\big{|}\mathcal{F}^{\alpha}_{s_{-}}\big{)}=\mathscr{L}\big{(}y(s_{-})\big{|}\mathcal{F}^{\alpha}_{s_{-}}\big{)} and $y(s)$ satisfies

[TABLE]

The assertion of Step 2 is therefore complete.

It follows from Step 1 and Step 2 (with $\varsigma$ is replaced by sample path $\alpha_{T_{-}}$) that the solution $\mu$ to (4.13) satisfies

[TABLE]

In view of (4.19) and Step 2,

[TABLE]

These imply \mu(s)=\mathscr{L}\big{(}y(s)\big{|}\mathcal{F}^{\alpha}_{s_{-}}\big{)} for all $0\leq s\leq T$.

Step 3: Prove the uniqueness of the solution of (4.25).

Suppose that $y_{1},y_{2}$ are two solutions to equation (4.25) with same initial value $y(0)$. Denote \mu_{i}(s)=\mathscr{L}\big{(}y_{i}(s)\big{|}\mathcal{F}^{\alpha}_{s_{-}}\big{)} for $0\leq s\leq T$, $i=1,2$. Then

[TABLE]

Similar to (A.14), we have

[TABLE]

Thus, by Cauchy-Schwarz and Bulkhoder-Davis-Gundy inequalities, we have

[TABLE]

In view of the Gronwall inequality, \mathbb{E}\big{|}y_{1}(t)-y_{2}(t)\big{|}^{2}=0 for $0\leq t\leq T$. This implies that $y_{1}(t)=y_{2}(t)$ a.s. $\qquad\Box$

Proof of Theorem 2.1. Since $\mathscr{P}_{N}$ is the distribution of \big{(}\mu_{N}(\cdot),\alpha(\cdot)\big{)}, \mathscr{P}\big{(}\mathcal{C}\big{)}=\mathbb{P}\big{(}\alpha_{T}\in\mathcal{C}\big{)} for any measurable set \mathcal{C}\subset D\big{(}[0,T],\mathbb{S}\big{)} where \alpha_{T}:\Omega\to D_{f}\big{(}[0,T],\mathbb{S}\big{)} is defined by $\alpha_{T}(s,\omega)=\alpha(s,\omega)$ for $0\leq s\leq T$ and $\omega\in\Omega$. The convergence \mathscr{L}\big{(}\mu_{N}(0)\big{)}\Rightarrow\delta_{\mu_{0}} in \mathcal{P}\big{(}\mathscr{M}_{1},\|\cdot\|_{BL}\big{)} implies that \mathscr{P}\big{(}\eta(0)=\mu_{0}\big{)}=1.

Let \bar{\Lambda}_{T}:D_{f}\big{(}[0,T],\mathbb{S}\big{)}\to D\big{(}[0,T],\mathscr{M}_{1}\big{)} be the mapping that maps each \varsigma(\cdot)\in D_{f}\big{(}[0,T],\mathbb{S}\big{)} to the unique solution \eta(\cdot)\in D\big{(}[0,T],\mathscr{M}_{1}\big{)} to equation (4.14) (or equivalently, (4.7)) for $0\leq t\leq T$. Similar to Lemma 4.7 we can show that \bar{\Lambda}_{T}:D_{f}\big{(}[0,T],\mathbb{S}\big{)}\to C\big{(}[0,T],\mathscr{M}_{1}\big{)} is continuous.

Denote \Gamma:D_{f}\big{(}[0,T],\mathbb{S}\big{)}\to D\big{(}[0,T],\mathscr{M}_{1}\times\mathbb{S}\big{)} by \Gamma\varsigma=\big{(}\bar{\Lambda}_{T}\varsigma,\varsigma\big{)} and let $\mathcal{S}$ be the set of all pairs (\eta,\varsigma)\in C\big{(}[0,T],\mathscr{M}_{1}\big{)}\times D_{f}\big{(}[0,T],\mathbb{S}\big{)} satisfying equation (4.7). Since C\big{(}[0,T],\mathscr{M}_{1}\big{)}\times D_{f}\big{(}[0,T],\mathbb{S}\big{)} is a closed subset of D\big{(}[0,T],\mathscr{M}_{1}\times\mathbb{S}\big{)},

[TABLE]

Thus, \mathscr{P}\big{(}C\big{(}[0,T],\mathscr{M}_{1}\big{)}\times D_{f}\big{(}[0,T],\mathbb{S}\big{)}\big{)}=1. This together with Theorem 4.4 implies that $\mathscr{P}(\mathcal{S})=1$. According to Step 1 in the proof of Theorem 4.6, \mathcal{S}=\big{\{}\Gamma\varsigma:\varsigma\in D_{f}\big{(}[0,T],\mathbb{S}\big{)}\big{\}}. Therefore, for each measurable set \mathcal{A}\subset D\big{(}[0,T],\mathscr{M}_{1}\times\mathbb{S}\big{)} we have

[TABLE]

Since \bar{\Lambda}_{T}\alpha_{T}(s)=\mathscr{L}\big{(}y(s)\big{|}\mathcal{F}^{\alpha}_{s_{-}}\big{)}, where $y(t)$ is the unique solution to (4.25), the above identities imply that $\mathscr{P}$ is the distribution of the process \big{(}\mathscr{L}\big{(}y(s)\big{|}\mathcal{F}^{\alpha}_{s_{-}}\big{)},\alpha(s)\big{)} on D\big{(}[0,T],\mathscr{M}_{1}\times\mathbb{S}\big{)}. This completes the proof. $\qquad\Box$

5 $N$-Particle Mean-Field Models with

Two-Time-Scale Markovian Switching Process

As alluded to in the introduction, this section is devoted to the case that the number of particles $N\to\infty$, meanwhile, the Markov chain displays weak and strong interactions reflected by use of a small parameter $\varepsilon\to 0$. We require that $N\wedge(1/\varepsilon)\to\infty$ as $\varepsilon\to 0$ and $N\to\infty$.

5.1 Formulation

We consider a class of mean-field processes, in which the random switching process changes much faster than the continuous state (or the switching process jump change much more frequently). The basic idea is that there are inherent two-time scales. Our interest focuses on the limit behavior of the resulting process. Suppose that $\varepsilon>0$ is a small parameter and the system of mean-field equations is given by

[TABLE]

where $w_{1}(\cdot)$, $w_{2}(\cdot)$, $\ldots$, $w_{N}(\cdot)$ are independent $d$-dimensional standard Brownian motions, and $\alpha^{\varepsilon}(t)$ is a Markov chain with state space \mathbb{S}=\big{\{}1,\ldots,m_{0}\big{\}} satisfying

[TABLE]

as $\Delta t\to 0$, where the generator

[TABLE]

satisfies $q^{\varepsilon}_{i_{0}j_{0}}\geq 0$ for $i_{0}\neq j_{0}$ and $\sum_{j_{0}\in\mathcal{M}}q^{\varepsilon}_{i_{0}j_{0}}=0$ for each $i_{0}\in\mathbb{S}$.

The model above is motivated by the work of two-time-scale Markov chains [36]. Such two-time scale Markov chains have been used widely, especially in networked systems; see for example, the manufacturing systems given in [29]. It is readily seen that the Markov chain has a fast varying part and a slowly changing part. Suppose

[TABLE]

Then the state space $\mathbb{S}$ of the underlying Markov chain is decomposable into $l$ subspaces. These subspaces are not completely separated. There are weak interactions among the subspaces due to the use of the slowly varying part of the generator $\hat{Q}$. Such a structure is often referred to as nearly decomposable Markov chain.

We relabel the states so that

[TABLE]

with \mathbb{S}_{i}=\big{\{}s_{i1},s_{i2},\ldots,s_{im_{i}}\big{\}} and $m_{0}=m_{1}+m_{2}+\ldots+m_{l}$ such that $\tilde{Q}^{i}$, the generator associated with the subspace $\mathbb{S}_{i}$ for each $i=1,\ldots,l$. Assume that each $\tilde{Q}^{i}$ is irreducible. As a consequence, the corresponding $\mathbb{S}_{i}$ for $i=1,\ldots,l$ consist of recurrent states belonging to $l$ ergodic classes. Let \nu^{i}=\big{(}\nu_{s_{i1}},\nu_{s_{i2}},\ldots,\nu_{s_{im_{i}}}\big{)} be the stationary distribution corresponding to $\tilde{Q}^{i}$, $1\leq i\leq l$, and \tilde{\nu}=\text{diag}\big{[}\nu^{1},\nu^{2},\ldots,\nu^{l}\big{]}\in\mathbb{R}^{l\times m_{0}}. Following the ideas in [36], we aim to reduce the computational complexity. The rationale is that we take advantage of the fast and slow motions and strong and weak interactions of the systems so that we can naturally divide the state space of the switching process into subsystems or groups. Within each subsystem, the states look alike in that they vary at the same speed, and among different subsystems, the variations are relatively slowly. To proceed, we lump the states of the jump component in each $\mathbb{S}_{i}$ into a single state and define

[TABLE]

Denote the state space of $\bar{\alpha}^{\varepsilon}(\cdot)$ by \bar{\mathbb{S}}=\big{\{}1,2,\ldots,l\big{\}}. It follows from [36, Theorem 5.27] that $\bar{\alpha}^{\varepsilon}(\cdot)$ converges weakly to $\bar{\alpha}(\cdot)$, a Markov chain with the state space $\bar{\mathbb{S}}$ and generator $\bar{Q}$ defined by

[TABLE]

where 1\!\!1=\text{diag}\big{[}{1\!\!1}_{m_{1}},{1\!\!1}_{m_{2}},\ldots,{1\!\!1}_{m_{l}}\big{]}\in\mathbb{R}^{m_{0}\times l} and {1\!\!1}_{k}=\big{(}1,1,\ldots,1\big{)}^{\prime}\in{\mathbb{R}}^{k}.

For $(x,\mu,i)\in\mathbb{R}^{d}\times\mathscr{M}_{1}\times\bar{\mathbb{S}}$, denote a\big{(}x,\mu,s_{ij}\big{)}=\sigma\big{(}x,\mu,s_{ij}\big{)}\sigma^{\prime}\big{(}x,\mu,s_{ij}\big{)} and

[TABLE]

For simplicity, we assume that the initial values $x^{\varepsilon}_{i}(0)=x_{0,i}$, $i=1,2,\ldots$, are independent of $\varepsilon$ and that $x_{0,i}$ are independent of $\alpha^{\varepsilon}(\cdot)$. We make the following assumption.

• (A3)

The matrix-valued function $\bar{a}(\cdot,\cdot,\cdot)$ on ${\mathbb{R}}^{d}\times\mathscr{M}_{1}\times\bar{\mathbb{S}}\to\mathbb{R}^{d\times d}$ has a representation $\bar{a}(x,\mu,i)=\bar{\sigma}(x,\mu,i)\bar{\sigma}^{\prime}(x,\mu,i)$ where $\bar{\sigma}(\cdot,\cdot,\cdot)$ is bounded and there is a constant $L$ such that

[TABLE]

for all $x,y\in\mathbb{R}^{d}$, $\mu,\eta\in\mathscr{M}_{1}$ and $i\in\bar{\mathbb{S}}$.

For $N\geq 1$, $\varepsilon>0$, $0\leq t\leq T$, and $A\in\mathcal{B}(\mathbb{R}^{d})$ denote

[TABLE]

Then $\mu^{\varepsilon}_{N}(t)$, $0\leq t\leq T$, defines a process on the space $\mathscr{M}_{1}$ of probability measures on $\mathbb{R}^{d}$. Because of the assumption on the initial values of $x^{\varepsilon}_{i}(0)$, $\mu^{\varepsilon}_{N}(0)=\mu_{N}(0)$ does not depend on $\varepsilon$. Let $d_{\bar{\mathbb{S}}}$ and $\bar{d}$, respectively, be the metrics on $\bar{\mathbb{S}}$ and $\mathscr{M}_{1}\times\bar{\mathbb{S}}$ defined by similar way to $d_{\mathbb{S}}$ and $d$ in (2.2). Denote $\bar{\mathscr{P}}_{N}^{\varepsilon}$ by the induced probability measure of \big{(}\mu_{N}^{\varepsilon}(\cdot),\bar{\alpha}^{\varepsilon}(\cdot)\big{)} on D\big{(}[0,T],\mathscr{M}_{1}\times\bar{\mathbb{S}}\big{)}. We can show that \big{(}\mu_{N}^{\varepsilon}(\cdot),\bar{\alpha}^{\varepsilon}(\cdot)\big{)}\in C([0,T],\mathscr{M}_{1})\times D_{f}([0,T],\bar{\mathbb{S}}), a closed subspace of D\big{(}[0,T],\mathscr{M}_{1}\times\bar{\mathbb{S}}\big{)}.

Theorem 5.1**.**

Assume (A1), (A2), (A3), and

[TABLE]

Then \big{(}\mu_{N}^{\varepsilon}(\cdot),\bar{\alpha}^{\varepsilon}(\cdot)\big{)} converges weakly to process \big{(}\bar{\mu}_{\bar{\alpha}}(\cdot),\bar{\alpha}(\cdot)\big{)} as $\varepsilon\to 0$ and $N\to\infty$ satisfying $(1/\varepsilon)\wedge N\to\infty$, where

[TABLE]

and $\bar{\zeta}(t)$, $0\leq t\leq T$, is the unique solution of the following stochastic differential equation

[TABLE]

where $\tilde{w}(\cdot)$ is a standard Brownian motion independent of $\bar{\alpha}(\cdot)$.

5.2 Weak Compactness and Auxiliary Estimates

For $N\geq 1$, $\varepsilon>0$ and $t>0$ denote

[TABLE]

Remark 5.2**.**

It follows from the proof of Lemma 3.1 that for each $p$, $0<p\leq 1$, there exists a constant $C$ independent of $N$ and $\varepsilon$ such that

[TABLE]

and, for $0\leq s\leq t\leq T$,

[TABLE]

Proposition 5.3**.**

Assume that all the assumptions of Theorem 5.1 hold. Then the sequence $\{\mu^{\varepsilon}_{N}(\cdot),\bar{\alpha}^{\varepsilon}(\cdot)\}$ is weakly compact in the topology of weak convergence of probability measure on $D([0,T],\mathscr{M}_{1}\times\bar{\mathbb{S}})$.

Proof. According to inequalities (5.6)-(5.7) and the arguments in Lemma 4.1 and Lemma 4.2, for any $\delta>0$ there exists a compact set $K_{\delta}$ in $(\mathscr{M}_{1},\|\cdot\|_{BL})$ such that

[TABLE]

and there exists a constant $C$ independent of $N$ and $\varepsilon$ such that

[TABLE]

Since $\bar{\alpha}^{\varepsilon}(\cdot)$ converges weakly to $\bar{\alpha}(\cdot)$ (see Theorem 7.4 [36]) we obtain the compactness of $(\mu_{N}^{\varepsilon}(\cdot),\bar{\alpha}^{\varepsilon}(\cdot))$. $\qquad\Box$

For each function $f(\cdot,\cdot)$ with $f(\cdot,s_{ij})\in C^{2}(\mathbb{R}^{d})$ for $s_{ij}\in\mathbb{S}$, denote the operator associated to (5.1) by

[TABLE]

where

[TABLE]

To approximate this operator for small values of $\varepsilon$ we define for each function $g(\cdot,\cdot)$ such that $g(\cdot,i)\in C^{2}(\mathbb{R}^{d})$ for $i\in\bar{\mathbb{S}}$,

[TABLE]

where \bar{Q}g(x)(i)=\sum_{j\in\bar{\mathbb{S}}}\bar{q}_{ij}\big{(}g(x,j)-g(x,i)\big{)}.

Note that for $f(\cdot)\in C^{2}(\mathbb{R}^{d})$, we can define \mathcal{L}^{\varepsilon}(\mu)f\big{(}x,s_{ij}\big{)} as in (5.8) by taking $f(x,s_{ij})\equiv f(x)$ for $x\in\mathbb{R}^{d}$ and $s_{ij}\in\mathbb{S}$. Similarly, for each $f(\cdot)\in C^{2}(\mathbb{R}^{d})$, we can define \bar{\mathcal{L}}(\mu)f\big{(}x,i\big{)}=\bar{\mathcal{L}}(\mu)g\big{(}x,i\big{)} where $g(x,i)\equiv f(x)$ for any $x\in\mathbb{R}^{d}$ and $i\in\bar{\mathbb{S}}$. We have the following approximation. To make the presentation more transparent, its proof is given in the Appendix.

Lemma 5.4**.**

Under Assumption A, for any $f\in C^{3}_{c}(\mathbb{R}^{d})$ there is a constant $C$ independent of $N$ and $\varepsilon$ such that

[TABLE]

5.3 Weak Convergence and Stochastic McKean-Vlasov Equation with Two-Time-Scale Markovian Switching

Similar to (3.6), we define the martingale associated with the limiting Markovian switching process $\bar{\alpha}(\cdot)$ by

[TABLE]

where \big{[}\bar{M}_{ij}\big{]}(t)=\sum_{0\leq s\leq t}{1\!\!1}\big{(}\bar{\alpha}(s_{-})=i\big{)}{1\!\!1}\big{(}\bar{\alpha}(s)=j\big{)} and \big{\langle}\bar{M}_{ij}\big{\rangle}(t)=\int_{0}^{t}\bar{q}_{i_{0}j_{0}}{1\!\!1}\big{(}\bar{\alpha}(s_{-})=i_{0}\big{)}ds. In addition, by a similar way to (3.14), we define sample path of the martingale associated with a sample path \bar{\varsigma}\in D_{f}\big{(}[0,T],\bar{\mathbb{S}}\big{)} of $\bar{\alpha}(t)$ by

[TABLE]

where \big{[}\bar{M}_{ij}^{\bar{\varsigma}}\big{]}(t)=\sum_{0\leq s\leq t}{1\!\!1}\big{(}\bar{\varsigma}(s_{-})=i\big{)}{1\!\!1}\big{(}\bar{\varsigma}(s)=j\big{)} and \big{\langle}\bar{M}_{ij}^{\bar{\varsigma}}\big{\rangle}(t)=\int_{0}^{t}\bar{q}_{ij}{1\!\!1}\big{(}\bar{\varsigma}(s_{-})=i\big{)}ds. In addition, for \bar{\varsigma}\in D\big{(}[0,T],\bar{\mathbb{S}}\big{)}\backslash D_{f}\big{(}[0,T],\bar{\mathbb{S}}\big{)} and $0\leq t\leq T$ define \bar{M}_{ij}^{\bar{\varsigma}}(t)=\big{[}\bar{M}_{ij}^{\bar{\varsigma}}\big{]}(t)=\big{\langle}\bar{M}_{ij}^{\bar{\varsigma}}\big{\rangle}(t)=0.

Proposition 5.5**.**

Assume (A1), (A2), and $\sup_{N\geq 1}\mathbb{E}\langle\mu_{N}(0),\psi\rangle<\infty$. Denote by $\bar{\mathscr{P}}$ the limit of an arbitrary weakly convergence subsequence $\bar{\mathscr{P}}_{N_{k}}^{\varepsilon_{k}}$ as $k\to\infty$ where $(\varepsilon_{k},N_{k})$ satisfies $\min\{1/\varepsilon_{k},N_{k}\}\to\infty$ as $k\to\infty$. Then for $\bar{\mathscr{P}}$-almost all (\eta,\bar{\varsigma})\in D\big{(}[0,T],\mathscr{M}_{1}\times\bar{\mathbb{S}}\big{)}, the equation

[TABLE]

holds for any test function $g(\cdot,i)\in C^{2}_{c}(\mathbb{R}^{d})$, $i\in\bar{\mathbb{S}}$.

Proof. Let $\bar{\mathscr{P}}$ be the weak limit on D\big{(}[0,T],\mathscr{M}_{1}\times\bar{\mathbb{S}}\big{)} of a subsequence $\bar{\mathscr{P}}^{\varepsilon_{k}}_{N_{k}}$ of $\bar{\mathscr{P}}^{\varepsilon}_{N}$ as $k\to\infty$ where $(\varepsilon_{k},N_{k})$ is satisfies $\min\{1/\varepsilon_{k},N_{k}\}\to\infty$ as $k\to\infty$.

First, we prove that for $\bar{\mathscr{P}}-$almost all (\eta,\bar{\varsigma})\in D\big{(}[0,T],\mathscr{M}_{1}\times\bar{\mathbb{S}}\big{)}, (5.13) holds for $g(x,i)=f(x)$ with $f\in C^{2}_{c}(\mathbb{R}^{d})$. For each $(\eta,\bar{\varsigma})$ denote

[TABLE]

which defines a function on D\big{(}[0,T],\mathscr{M}_{1}\times\bar{\mathbb{S}}\big{)}. We observe that for a fixed pair (\eta,\bar{\varsigma})\in D\big{(}[0,T],\mathscr{M}_{1}\times\bar{\mathbb{S}}\big{)}, if (5.13) holds for any test function $g(\cdot,\cdot)$ such that $g(x,i)\equiv f(x)$, $f\in C^{3}_{c}(\mathbb{R}^{d})$, it also holds for any test function $g(\cdot,\cdot)$ such that $g(x,i)\equiv f(x)$, $f\in C^{2}_{c}(\mathbb{R}^{d})$. Thus, in order to prove (5.13) holds $\bar{\mathscr{P}}-$almost surely for any $g=f\in C^{2}_{c}(\mathbb{R}^{d})$, it suffices to show that

[TABLE]

Take $\delta>0$. Since the set \big{\{}(\eta,\bar{\varsigma}):|\bar{M}_{f}(t)|\leq\delta\big{\}} is closed in D\big{(}[0,T],\mathscr{M}_{1}\times\bar{\mathbb{S}}\big{)} we have

[TABLE]

Note that since $\bar{\mathscr{P}}^{\varepsilon_{k}}_{N_{k}}$ is the distribution of \big{(}\mu^{\varepsilon_{k}}_{N_{k}}(\cdot),\bar{\alpha}^{\varepsilon_{k}}(\cdot)\big{)},

[TABLE]

By Lemma 5.4, we obtain

[TABLE]

Next, denote

[TABLE]

By the Itô formula, we observe that $M^{\varepsilon_{k}}_{N_{k},f}(t)$ is a continuous square integrable martingale with quadratic variation process

[TABLE]

Similar to (4.9), since $\sigma(\cdot,\cdot,\cdot)$ is bounded we have \sup_{0\leq t\leq T}\mathbb{E}\big{[}M^{\varepsilon_{k}}_{N_{k},f}\big{]}(t)\leq{CT\over N_{k}}. Thus, by Doob’s inequality,

[TABLE]

Combining (5.16)-(5.18) yields

[TABLE]

Since $\delta$ is taken arbitrarily, the above equation and (5.15) imply (5.14).

Next, we prove that if for some pair $(\eta,\bar{\varsigma})$, (5.13) holds for any f\in C^{2}_{c}\big{(}\mathbb{R}^{d}\big{)}, it also holds for any $g(\cdot,\cdot)$ such that g(\cdot,i)\in C^{2}_{c}\big{(}\mathbb{R}^{d}\big{)} for each $i\in\bar{\mathbb{S}}$. For f\in C^{2}_{c}\big{(}\mathbb{R}^{d}\big{)} and $i\in\bar{\mathbb{S}}$ denote

[TABLE]

Then, for any f\in C^{2}_{c}\big{(}\mathbb{R}^{d}\big{)} and g(\cdot,i)\in C^{2}_{c}\big{(}\mathbb{R}^{d}\big{)}, $i\in\bar{\mathbb{S}}$,

[TABLE]

Let $(\eta,\bar{\varsigma})$ be a pair in D\big{(}[0,T],\mathscr{M}_{1}\times\bar{\mathbb{S}}\big{)} such that (5.13) holds for any f\in C^{2}_{c}\big{(}\mathbb{R}^{d}\big{)}. By the definition of $\hat{\bar{\mathcal{L}}}_{i}$, we can rewrite (5.13) as follow

[TABLE]

for any $0\leq r\leq t\leq T$ and f\in C^{2}_{c}\big{(}\mathbb{R}^{d}\big{)}.

Denote the jump times of $\bar{\varsigma}$ by $t_{0}=0$, t_{n+1}=\inf\big{\{}t>t_{n}:\bar{\varsigma}(t)\neq\bar{\varsigma}(t_{-})\big{\}} and $\iota_{n}=\bar{\varsigma}(t_{n})$ for $n\geq 0$. For $t_{n}<t<t_{n+1}$ we have

[TABLE]

This implies

[TABLE]

In view of (5.20) with $f(x)=g(x,\iota_{n})$, (5.19) and (5.22), we have

[TABLE]

At $t=t_{n+1}$, (5.21) still holds if $j\neq\iota_{n+1}$. In addition,

[TABLE]

Thus,

[TABLE]

As a consequence, by applying (5.20) with $f(x)=g(x,\iota_{n})$ and $t=t_{n+1}$ we have

[TABLE]

This implies that (5.23) also holds for $t=t_{n+1}$ and therefore (5.23) is proved for any $g(\cdot,\cdot)$ such that g(\cdot,i)\in C^{2}_{c}\big{(}\mathbb{R}^{d}\big{)} for each $i\in\bar{\mathbb{S}}$ as desired. $\qquad\Box$

As a direct consequence of Theorem 4.6 we have the following proposition, which characterizes the limit $\bar{\mathscr{P}}$ as a solution to a stochastic McKean-Vlasov equation with Markovian switching.

Proposition 5.6**.**

Assume (A1), (A2), and (A3). Let $\mu_{0}$ be a measure in $\mathscr{M}_{1}$. Then the system of integral equations

[TABLE]

where $f(\cdot,i)\in C^{2}_{c}(\mathbb{R}^{d})$ for each $i\in\bar{\mathbb{S}}$, has a unique solution in D\big{(}[0,T],\mathscr{M}_{1}\big{)}. Moreover, this solution equals \mathscr{L}\big{(}\bar{x}(t)\big{|}\mathcal{F}^{\bar{\alpha}}_{t_{-}}\big{)} for all $0\leq t\leq T$, where $\bar{x}(t)$ is the unique solution of

[TABLE]

where $\tilde{w}(\cdot)$ is a standard Brownian motion independent of $\bar{\alpha}(\cdot)$.

Proof of Theorem 5.1. Since $\bar{\alpha}^{\varepsilon}(\cdot)$ converges weakly to $\bar{\alpha}(\cdot)$, for any \mathcal{C}\subset D\big{(}[0,T],\bar{\mathbb{S}}\big{)} we have \bar{\mathscr{P}}\big{(}\mathcal{C}\big{)}=\mathbb{P}\big{(}\bar{\alpha}_{T}\in\mathcal{C}\big{)}. By using \bar{\mathcal{L}},\big{(}\bar{\alpha}(\cdot),\bar{\mathbb{S}}\big{)}, respectively, instead of \mathcal{L},\big{(}\alpha(\cdot),{\mathbb{S}}\big{)}, Proposition 5.5 instead of Theorem 4.4, and Proposition 5.6 instead of Theorem 4.6, a similar argument to that in the proof of Theorem 2.1 yields the assertion of Theorem 5.1.$\qquad\Box$

Appendix A Appendix

A.1 Proof of Lemma 3.1

Define

[TABLE]

The sequence $\tau_{k},k=1,2,\ldots$ is monotonically increasing. Put $\tau_{\infty}=\lim_{k\to\infty}\tau_{k}$.

We are in a position to prove that $\lim_{k\to\infty}\tau_{k}=\infty$ a.s. Suppose that there exists a positive numbers $T_{0}$ and $\varepsilon$ such that P\big{(}\lim_{k\to\infty}\tau_{k}<T_{0}\big{)}>2\varepsilon. Then there is a number $k_{0}$ such that P\big{(}\tau_{k}<T_{0}\big{)}>\varepsilon for all $k\geq k_{0}$. Recall that for $x\in\mathbb{R}^{d}$, $\psi(x)=|x|^{2}$. For $\underline{x}=(x_{1},x_{2},\ldots,x_{N})\in(\mathbb{R}^{d})^{N}$, define the Lyapunov function

[TABLE]

It is easily seen that

[TABLE]

[TABLE]

where $\nabla_{x_{i}}^{2}V=[\nabla_{x_{i}}(\nabla_{x_{i}}V)]^{\prime}$ denotes the $d\times d$ Hessian matrix with respect to the variable $x_{i}$ of V. Since $w_{1}(\cdot),w_{2}(\cdot),\ldots,w_{N}(\cdot)$ are independent Brownian motions, a direct calculation yields

[TABLE]

It follows from Assumption A and equations (A.1)-(A.3) that

[TABLE]

for some constant $C$ independent of $N$. Denote \underline{x}(t)=\big{(}x_{1}(t),x_{2}(t),\ldots,x_{N}(t)\big{)}^{\prime}. Then the Dynkin formula implies

[TABLE]

By the Gronwall inequality, we obtain

[TABLE]

According to the definitions of $\tau_{k}$ and $V$,

[TABLE]

Thus, (A.5) yields

[TABLE]

As a consequence,

[TABLE]

This is a contradiction. As a result, $\lim_{k\to\infty}\tau_{k}=\infty$ a.s.

Next, by applying (A.5) for $t$ instead of $T_{0}$, with $0\leq t\leq T$, and letting $k\to\infty$, we arrive at

[TABLE]

Since V\big{(}\underline{x}(t),\alpha(t)\big{)}=\big{[}\langle\mu_{N}(t),\psi\rangle+1\big{]}^{p}, this inequality implies (3.10) as desired.

To proceed, using the Dynkin formula again, we obtain

[TABLE]

for $0\leq s\leq t$. It follows from (A.4) that

[TABLE]

Letting $k\to\infty$, by virtue of the Gronwall inequality, there exists a constant $C$ independent of $N$ such that for $0\leq s\leq t$,

[TABLE]

This gives (3.11) and completes the proof. $\qquad\Box$

A.2 Proof of Lemma 4.7

We prove the two parts of the assertions as follows.

(i) Let \varsigma_{1},\varsigma_{2}\in D_{f}\big{(}[0,r),\mathbb{S}\big{)} and denote $\eta_{1}=\Lambda_{r}(\varsigma_{1}),\eta_{2}=\Lambda_{r}(\varsigma_{2})$. It follows from Step 1 in the proof of Theorem 4.6 that for $i=1,2$, $\eta_{i}=\mathscr{L}(y_{i}(s))$ which are the unique solution to the equation

[TABLE]

where \mathscr{L}\big{(}y_{i}(0)\big{)}=\mu_{0}. Without loss of generality, we can assume that $y_{1}(0)=y_{2}(0)$.

First, we show that there is a constant $C$ independent on $\varsigma_{i}$ and $r$ such that

[TABLE]

It follows from (A.6) that

[TABLE]

Note that \langle\eta_{i}(s),\varphi\rangle^{2}=\big{(}E|y_{i}(s)|\big{)}^{2}\leq E|y_{i}(s)|^{2}. Thus, by Assumption A, there is a constant $C$ independent on $\varsigma_{i}$ and $r$ such that for each $0\leq s\leq r$, \sigma\big{(}y_{i}(s),\eta_{i}(s),\varsigma_{i}(s_{-})\big{)}\leq C and

[TABLE]

Taking these inequalities into account and using the Burkholder-Davis-Gundy inequality for the last term in the right-hand side of (A.8), we arrive at

[TABLE]

By taking expectations on both sides of the above inequality and using the Gronwall inequality we obtain (A.7).

In view of (A.7) and (A.9), by assumption (A2) we have

[TABLE]

for any $\delta>0$ such that $0\leq t<t+\delta<r$ where $C$ is independent of $\delta$. This shows that \eta_{1},\eta_{2}\in C\big{(}[0,r),\mathscr{M}_{1}\big{)}.

Next, we prove the continuity of $\Lambda_{r}$. Let $\rho$ be the metric on $D([0,r),\mathbb{S})$ defined by

[TABLE]

where the infimum is taken over the class of all strictly increasing and continuous mappings $\lambda$ of $[0,r)$ onto itself. Because of the definition of the metric $d_{\mathbb{S}}$ on the discrete set $\mathbb{S}$, if $\rho(\varsigma_{1},\varsigma_{2})<1$, $\varsigma_{1}$ and $\varsigma_{2}$ have same number of jumps on $[0,r)$. Hence, to prove the continuity of $\Lambda_{r}$, it suffices to show that there is a constant $C$ such that

[TABLE]

whenever $\delta=\rho(\varsigma_{1},\varsigma_{2})<1$ where $m$ is the number of jumps on $[0,r)$ of $\varsigma_{1}$ and $\varsigma_{2}$. According to the definition of $\|\cdot\|_{BL}$ and \big{\langle}\eta_{1}(t)-\eta_{2}(t),f\rangle=\mathbb{E}\big{(}f(y_{1}(t))-f(y_{2}(t))\big{)}, we have

[TABLE]

Since $d_{\mathbb{S}}(i_{0},j_{0})=1$ if $i_{0}\neq j_{0}$, we have

[TABLE]

where $m$ is the number of jumps of $\varsigma_{1}$ and $\varsigma_{2}$, and each $A_{k}$ is an interval with length $|A_{k}|<\delta$. Denote $I_{0}=[0,r)\backslash I_{1}$ and

[TABLE]

According to (A.6) and the Cauchy-Schwarz inequality, there is a constant $C=C(T)$ such that

[TABLE]

To proceed, we estimate $J_{0}(t)$ and $J_{1}(t)$. Since $\varsigma_{1}=\varsigma_{2}$ on $I_{0}$, assumption (A1) and (A.11) give

[TABLE]

for $s\in I_{0}$ excepts at finite points on its boundary. Therefore, by virtue of the Burkholder-Davis-Gundy inequality, (A.12) and (A.14) imply

[TABLE]

where $C=C(T)$ is a constant which only depends on $T$. Now we are in a position to estimate $J_{1}(t)$. It follows from (A.7) and (A.9) that

[TABLE]

Since $I_{1}=\cup_{k=1}^{m}A_{k}$ where $m<\infty$, by the boundedness of $\sigma(\cdot,\cdot,\cdot)$, and the Cauchy-Schwarz and the Burkholder-Davis-Gundy inequalities,

[TABLE]

Combining (A.13), (A.15), and (A.16), we obtain

[TABLE]

By the Gronwall inequality, for any $0\leq t<r$,

[TABLE]

which together with (A.11) implies (A.10).

(ii) Equation (4.20) follows from the uniqueness of the solution of (4.14) proved in Step 1 in the proof of Theorem 4.6.

A.3 Proof of Lemma 5.4

We have

[TABLE]

Thus, to prove (5.10), it suffices to show that

[TABLE]

for some constant $C$ independent of $N$, $i$, $j$, and $k$. Denote

[TABLE]

where $\lfloor\varepsilon^{-1/3}\rfloor$ is the greatest integer that is less than or equal to $\varepsilon^{-1/3}$. Then $t_{l}-t_{l-1}=O(\varepsilon^{1/3})$. To proceed, we estimate \big{|}I_{1}(t,k,i,j)\big{|}. Let $q$ be an integer $0\leq q\leq n-1$ such that $t_{q}\leq t<t_{q+1}$. Note that

[TABLE]

We have

[TABLE]

Hence, the Cauchy-Schwarz inequality implies

[TABLE]

Since $\nabla_{x}f(\cdot)$ is Lipschitz and

[TABLE]

for some constant $C$ independent of $N,i,j$, assumption A and (4.3) imply that

[TABLE]

Next, using the Cauchy-Schwarz inequality again, we obtain

[TABLE]

where the last equation is a consequence of Theorem 5.25 of [36]. Therefore,

[TABLE]

Combining (A.19), (A.20), and (A.22), we arrive at

[TABLE]

By the same argument, we can show that

[TABLE]

Combining (A.23) and (A.24) yields \sup_{0\leq t\leq T}\mathbb{E}\Big{|}I_{1}(t,k,i,j)\Big{|}\leq C\varepsilon^{1/6}. Likewise, because $f\in C^{3}_{c}(\mathbb{R}^{d})$, we obtain \mathbb{E}\Big{|}I_{2}(t,k,i,j)\Big{|}\leq C\varepsilon^{1/6}. Thus (A.18) holds. The proof of the lemma is thus complete. $\qquad\Box$