Mean-field optimal control and optimality conditions in the space of probability measures

TL;DR

This paper develops a calculus on the space of probability measures to derive optimality conditions for mean-field control problems, connecting particle systems, PDEs, and numerical methods.

Contribution

It introduces a new calculus on probability measures for optimal control, linking particle-based and mean-field formulations, and provides convergence rates for controls as particle number increases.

Findings

Derived first-order optimality system on probability measures

Established relations between particle and mean-field optimal controls

Proved convergence rate of particle controls to mean-field controls

Abstract

We derive a framework to compute optimal controls for problems with states in the space of probability measures. Since many optimal control problems constrained by a system of ordinary differential equations (ODE) modelling interacting particles converge to optimal control problems constrained by a partial differential equation (PDE) in the mean-field limit, it is interesting to have a calculus directly on the mesoscopic level of probability measures which allows us to derive the corresponding first-order optimality system. In addition to this new calculus, we provide relations for the resulting system to the first-order optimality system derived on the particle level, and the first-order optimality system based on $L^2$-calculus under additional regularity assumptions. We further justify the use of the $L^2$-adjoint in numerical simulations by establishing a link between the adjoint in the space of probability measures and the adjoint corresponding to $L^2$-calculus. Moreover, we prove a convergence rate for the convergence of the optimal controls corresponding to the particle formulation to the optimal controls of the mean-field problem as the number of particles tends to infinity.

Equations

$$(x,u)=\mathrm{argmin}{\int }_0^{T}g(x_t)\mathrm{d}t,\text{subject to}\frac{\mathrm{d}}{\mathrm{d}t}{x}_t=v(x_t,u_t),$$

$$\frac{\mathrm{d}}{\mathrm{d}t}{\xi }_t={\nabla }_{x}g(x_t)+{\nabla }_{x}v(x_t,u_t){\xi }_t,$$

$${\nabla }_{u}v(x_t,u_t)\cdot {\xi }_t=0\text{ a.e. in }(0,T).$$

$${\mathcal{L}}_{\text{micro}}(x,u,\xi )={\int }_0^{T}g(x_t)\mathrm{d}t+{\int }_0^T\left(\frac{\mathrm{d}}{\mathrm{d}t}{x}_t-v(x_t,u_t)\right)\cdot {\xi }_t\mathrm{d}t.$$

$${\partial }_t{\mu }_t+{\nabla }_x\cdot (v(x_t,u_t){\mu }_t)=0.$$

$${\partial }_t{\mu }_t+{\nabla }_x\cdot (v(x,u_t){\mu }_t)=0,{\mu }_0={\delta }_{x(0)}.$$

$${\int }_0^{T}g(x_t)\mathrm{d}t={\int }_0^T⟨g,{\mu }_t⟩\mathrm{d}t,$$

$$(\mu ,u)=\mathrm{argmin}{\int }_0^T⟨g,{\mu }_t⟩\mathrm{d}t\text{subject to eqrefeq:transporteq0}.$$

$${\mathcal{L}}_{\text{macro}}(\mu ,u,\phi )={\int }_0^T⟨g,{\mu }_t⟩\mathrm{d}t+{\int }_0^T⟨\phi ,{\partial }_t{\mu }_t+{\nabla }_x\cdot (v(x,u_t){\mu }_t)⟩\mathrm{d}t.$$

$${\partial }_t\phi +v(x,u_t)\cdot {\nabla }_x\phi =0$$

$$-⟨{\nabla }_x\phi ,{\nabla }_{u}v(x,u_t){\mu }_t⟩=0.$$

$${\nabla }_{u}v(x_t,u_t)\cdot (-{\nabla }_x\phi (x_t,t))=0,$$

$${\xi }_t=-{\nabla }_x\phi (x_t,t).$$

$${\mu }^{\prime }=-\nabla \cdot q,$$

$${\partial }_t\phi +v(\cdot ,u_t)\cdot {\nabla }_x\phi =0,{\nabla }_{u}v(\cdot ,u_t)\cdot \nabla \phi =0\text{on }{\mathbb{R}}^d\times (0,T),$$

$$(X,u)=\mathrm{argmin}{\int }_0^T{E}^x[g(X_t)]\mathrm{d}t,\text{subject to}\mathrm{d}{X}_t=v(X_t,u_t)\mathrm{d}t+\sigma (X_t,u_t)\mathrm{d}{W}_t,$$

$${\partial }_t{\mu }_t+\nabla \cdot (v(x,u_t){\mu }_t)=\frac{1}{2}\Delta ({\sigma }^2{\mu }_t),$$

$$x^i,u^{\ell }:[0,T]\to {\mathbb{R}}^d,\text{for }i=1,\dots N\text{and}\ell =1,\dots ,M.$$

$$\text{x}:=(x^i{)}_{i=1,\dots ,N},\text{u}:=(u^{\ell }{)}_{\ell =1,\dots ,M},$$

$$\frac{\mathrm{d}}{\mathrm{d}t}{\text{x}}_t=v^N({\text{x}}_t,{\text{u}}_t),{\text{x}}_0=\hat{\text{x}},$$

$$W_{2}^{2}(\mu,\nu):=\inf\limits_{\pi\in\Pi(\mu,\nu)}\biggl{\{}\int_{\mathbb{R}^{d}}|x-y|^{2}\,\mathrm{d}\pi(x,y)\biggr{\}},\qquad\mu,\nu\in\mathcal{P}_{2}(\mathbb{R}^{d}),$$

$$\pi (B\times {\mathbb{R}}^d)=\mu (B),\pi ({\mathbb{R}}^d\times B)=\nu (B)\text{for }B\in \mathcal{B}({\mathbb{R}}^d).$$

$$⟨v(\mu ,\text{u})(x)-v(\mu ,\text{u})(y),x-y⟩\le C_l|x-y{|}^2,x,y\in {\mathbb{R}}^d.$$

$$v_i^N(\text{x},\text{u}):=v({\mu }^N,\text{u})(x^i),i=1,\dots ,N,$$

$${\mu }_{\text{x}}^N(A)=\frac{1}{N}\sum _{i=1}^N{\delta }_{x^i}(A),A\in \mathcal{B}({\mathbb{R}}^d)(=\text{Borel σ-algebra}),$$

$$\|v(\mu,\textbf{u})-v(\mu^{\prime},\textbf{u}^{\prime})\|_{\sup}\leq C_{v}\Bigl{(}W_{2}(\mu,\mu^{\prime})+\|\textbf{u}-\textbf{u}^{\prime}\|_{2}\Bigr{)}.$$

$$v_i^N(\text{x},\text{u})=-\frac{1}{N}\sum _{j=1}^N{K}_1(x^i-x^j)-\sum _{\ell =1}^M{K}_2(x^i-u^{\ell }),$$

$$\partial_{t}\mu_{t}+\nabla\cdot\big{(}v(\mu_{t},\textbf{u}_{t})\mu_{t}\big{)}=0,\qquad\mu_{0}=\hat{\mu},$$

$$(t,x)\mapsto v({\mu }_t,{\text{u}}_t)(x)=-(K_1\ast {\mu }_t)(x)-\sum _{\ell =1}^M{K}_2(x-u_t^{\ell }).$$

$$\int_{0}^{T}\int_{\mathbb{R}^{d}}\big{(}\partial_{t}h_{t}+v(\mu_{t},\textbf{u}_{t})\cdot\nabla h_{t}\big{)}\,\mathrm{d}\mu_{t}\,\mathrm{d}t+\int_{\mathbb{R}^{d}}h_{0}\,\mathrm{d}\hat{\mu}=0.$$

Full text

Keywords. Optimal control with ODE/PDE constraints, interacting particle systems, mean-field limits.

AMS subject classifications. 49K15, 49K20

1. Introduction

In the past few years, the growing interest in the (optimal) control of interacting particle systems and their corresponding mean-field limits led to many contributions on their numerical behavior (see, e.g., [27, 9]) as well as their analytical properties, e.g.,[17, 6]. They can be found in various fields of applications, for example physical or biological models like crowd dynamics [9, 25, 16, 5], consensus formation [4], or even global optimization [24, 12]. Meanwhile, there are also first approaches for stochastic particle systems available [7, 23].

Since there are several points of view on this subject, the analytical techniques vary from standard ODE and PDE theory over optimal transport to measure-valued solutions. This induces also different variants for the derivation of first-order optimality conditions and/or gradient information, which clearly also has some impact on the design of appropriate numerical algorithms for the solution of the optimal control problems at hand.

Before we discuss the novelty and advantages of our approach we recall some recent contributions to the topic. In [17] the notion of mean-field optimal control problems was introduced. The authors combine well-known mean-field limit results with $\Gamma$-convergence to prove the convergence of optimal controls of the microscopic problem with $N$ interacting particles to a solution of the corresponding mean-field optimal controls. The article focuses on sparse controls and Caratheodory solutions, where the controls act linearly and additive on the dynamic of the interacting particles. In contrast to the present paper, there is no discussion of first order optimality conditions, no statement of adjoints, and no discussion of a convergence rate.

Based on these observations, the derivation of a mean-field Pontryagin maximum principle was shown in [6]. Starting from a Hamiltonian point of view, subdifferential calculus is employed to derive a gradient flow structure and a corresponding forward-backward system. Key ingredients of the proofs are semiconvexity of the functionals along geodesics and a rescaling of the adjoint variable. The article considers a dynamical system of interacting particles and additionally some policy makers. The controls enter the dynamics through the policy makers which remain finite as the number of interacting particles tends to infinity. To illustrate their methodology, explicit computations for the Cucker-Smale dynamics were presented. As in [17], discussions on the optimality conditions, adjoints or convergence rate were absent.

Another Pontryagin Maximum Principle was derived via subdifferential calculus on the space of probability measures and needle-like variations in [7] for a non-local transport equation, where the control variable enters linearly in the velocity field of the transport equation. The resulting first-order optimality system consists of a forward-backward equation similar to the one in [6], and the corresponding measure is identified by disintegration. Supplementary to the needle-like variations, we propose a different approach for the derivation of a corresponding linear system, and as a by-product, provide a direct link between the particle adjoint and its mean-field counterpart. Furthermore, the velocity fields considered in the present paper are more general. In addition, we provide a convergence rate as the number of interacting particles tends to infinity.

In contrast to these analytical results, [19] approaches the problem formally with techniques from the field of Optimization with PDE constraints. All assumptions and computations are formal and the mean-field limit is established via an BBGKY-approach. Adjoints are dervided with formal $L^{2}$-calculus and closed by moments which can be interpreted as conditional expectations. A similar formal derivation can be found in [1].

To summarize, the aim of our contribution is multi-fold:

1. (1)

We take an applied viewpoint and establish first-order optimality conditions, in the KKT-sense, on the space of probability measures via a Lagrangian approach which can be used for numerical implementations. While the derivation of the linearized system (see (24) in Lemma 3.4) bears similarities with those made in [7], we provide an alternative strategy that circumvents the explicit use of Lagrangian flows. Additionally, we provide a characterization of the corresponding adjoint system, which takes the form of a momentum equation (see Theorem 3.11). As the considerations can be lifted in a straightforward manner to second-order dynamical systems, we rigorously justify the numerical results shown in [9]. 2. (2)

We build the bridge between the Hamiltonian-based results discussed in [17, 6, 1, 7] and the ones obtained by Lagrangian approaches (see the chart in Section 4). 3. (3)

We prove the convergence, with rates, of the sequence of optimal controls, as the number of interacting particles tends to infinity (cf. Theorem 5.1). The convergence crucially relies on the optimality system obtained in (1).

The main ideas are discussed in the following model example before we present our results in full details.

1.1. An illustrative example: Controlling a single particle

Let us start with an illustrative example from classical optimal control in order to illustrate the idea without the complication of a mean-field limit. We denote the dimension of the state space by $d\geq 1$ and the time interval of interest is $[0,T]$ for some $T>0$. We assume that the control variable $u$ acts on the velocity of a single particle with trajectory $x_{t}\in\mathbb{R}^{d}$ for $t\in[0,T]$ and we want to optimize a given functional depending on the trajectory, i.e.

[TABLE]

where $g$ and $v$ are given, sufficiently regular functions.

Then, the standard Pontryagin Maximum principle yields the existence of an adjoint variable $\xi$ satisfying

[TABLE]

with terminal condition $\xi_{T}=0$.

Moreover, the control $u$ satisfies the optimality condition

[TABLE]

These conditions can be translated into the calculation of a saddle-point of the microscopic Lagrangian

[TABLE]

On the other hand, the discrete ODE can be translated into a macroscopic formulation via the method of characteristics: with initial value $\mu_{0}=\delta_{x_{0}}$ the concentrated measure $\mu_{t}=\delta_{x_{t}}$ is the unique solution of

[TABLE]

Since all measures are concentrated at $x_{t}$ we can reinterpret $u_{t}$ as the evaluation of a feedback-control $u(x,t)$ at $x=x_{t}$ and equivalently obtain

[TABLE]

Since

[TABLE]

we can formulate an optimal control problem at the macroscopic level for the measure $\mu$ and the control variable $u$, i.e.

[TABLE]

This macroscopic optimal control problem is in fact equivalent to the microscopic one for a single particle, since we can choose the state space as the Banach space of Radon measures and the control space as an appropriate space of reasonably smooth functions on $\mathbb{R}^{d}\times(0,T)$. The uniqueness of solutions to the transport equation and the special initial value will always yield a concentrated measure and the identification $u_{t}=u(x_{t},t)$ brings us back to the microscopic control.

However, with the macroscopic formulation we have another option to derive optimality conditions in these larger spaces, based on the Lagrangian

[TABLE]

Then, the macroscopic adjoint equation becomes

[TABLE]

and the optimality condition is given by

[TABLE]

Due to the equivalence of the microscopic and macroscopic optimal control problem it is natural to ask for the relation between the adjoint variables $\xi$ and $\varphi$, which is not obvious at a first glance and yet only very little discussed. For first results in this direction see [19]. Using the special structure of the solution $\mu_{t}$ and the identification with the microscopic control we can rewrite the optimality condition as

[TABLE]

which induces the identification

[TABLE]

Indeed, the method of characteristics confirms that $-\nabla_{x}\varphi(x_{t},t)$ satisfies the microscopic adjoint equation. This becomes more apparent if we consider only variations of $\mu$ that respect the nonnegativity and mass one condition of the probability measure, i.e.

[TABLE]

with a vector-valued measure $q$ being absolutely continuous with respect to $\mu$. Then, an integration by parts argument directly reveals the relation to $-\nabla\varphi$.

By using variations of this kind we reinterpret the state space as a Riemannian manifold of Borel probability measures equipped with the 2-Wasserstein distance instead of the flat Banach space of Radon measures. The analysis of particle systems and limiting nonlinear partial differential equations in the 2-Wasserstein distance has been a quite fruitful field of study in the last years following the seminal papers [22, 20]. It is hence highly overdue to study such an approach also in the optimal control setting.

We mention that the values of $\varphi$ outside the trajectory are irrelevant for the specific control problem. Solving

[TABLE]

we obtain the adjoints for all possible microscopic control problems with initial value in $\mathbb{R}^{d}$. This is just the well-known Hamilton-Jacobi-Bellmann equation, usually derived with different arguments.

Remark 1.1*.*

The above arguments can also be extended to a stochastic control system (see, e.g., [26]):

[TABLE]

with $W_{t}$ being a Wiener process and $X$ the solution to the stochastic differential equation with initial condition $X_{0}=x$. In this case the state equation for the probability density $\mu$ becomes

[TABLE]

and $\mu$ does not necessarily remain a concentrated measure in time, which corresponds to the stochasticity of the model.

1.2. Control in the Mean-field Limit

Having understood the relation between microscopic and macroscopic formulations of the optimal control problem, it seems an obvious step to consider optimal control problems for a high number of particles $N$ and their mean-field limit as $N\rightarrow\infty$, which is also the motivation for this paper. However, in the mean-field limit there is no microscopic particle system and corresponding optimal control problem, hence an additional step is needed to understand the connection in the limit. The basis for such a step is to understand the characteristic flow, which replaces the particle dynamics and naturally leads to an analysis in the Wasserstein distance. We will further investigate this mean-field setting in the remainder of the paper.

Here, we restrict our considerations to first order dynamics, but the present paper can be seen as an analytical justification of the convergence shown numerically in [9]. It is an additional contribution to the field of optimization of particle systems and their mean-field limits which is lively discussed in the recent years (e.g. [25, 4, 17, 6, 24, 12, 2, 1]). Moreover, we would like to connect the fields of optimal control and gradient flows as well as optimal transport. In particular, we show relations between the adjoints derived by $L^{2}$-calculus and adjoints derived in the space of probability measures ($W_{2}$-adjoints).

The paper is organized as follows: in Section 2 the microscopic model for $N$ particles and the corresponding mean-field equation is introduced. Further, we formulate the optimal control problems under investigation. The first main contribution of the article is the derivation of the first-order optimality conditions in the mesoscopic formulation given in Section 3. A discussion of the relation of this new calculus to the first-order optimality systems on the particle level and the first-order optimality condition based on $L^{2}$-calculus is the content of Section 4. In Section 5 we show the second main result which is the convergence rate for the optimal controls as $N\to\infty$.

2. Optimal Control Problems

First, we generalize the one-particle case to $N\in\mathbb{N}$ interacting particles, modeling, e.g., crowd dynamics [9]. Then, we derive its corresponding mean-field limit, i.e. the mesoscopic approximation. These two are the state systems for the respective optimal control problems. Further, we present the assumptions which are necessary for the well-posedness of the state systems.

2.1. The State Models

As before, $d\geq 1$ denotes the dimension of the state space and $[0,T]\subset\mathbb{R}$ with $T>0$ is the time interval of interest.

2.1.1. The particle system

The considered particle system consists of $N\in\mathbb{N}$ particles of the same type and $M\in\mathbb{N}$ controls represented by the functions

[TABLE]

The vectors

[TABLE]

denote the states of the particles and the controls, respectively.

The particle system reads explicitly

[TABLE]

with given $\hat{\textbf{x}}\in\mathbb{R}^{dN}$ defining the initial states of the particles. The operator $v^{N}$ on the right-hand side strongly depends on the type of application.

In the following, we denote by $\mathcal{P}_{2}(\mathbb{R}^{d})$ the space of Borel probability measures on $\mathbb{R}^{d}$ with finite second moment and equipped with the 2-Wasserstein distance, which makes $\mathcal{P}_{2}(\mathbb{R}^{d})$ a complete metric space, and by $\mathcal{P}_{2}^{ac}(\mathbb{R}^{d})$ the subset of $\mathcal{P}_{2}(\mathbb{R}^{d})$ containing probability measures with Lebesgue density. For the sake of completeness we recall the 2-Wasserstein distance:

[TABLE]

where $\Pi(\mu,\nu)$ denotes the set of all Borel probabililty measures on $[0,T]\times\mathbb{R}^{2d}$ that have $\mu$ and $\nu$ as first and second marginals respectively, i.e.

[TABLE]

In the rest of the article we denote by $\mathfrak{m}_{2}(\mu)$ the second moment of $\mu\in\mathcal{P}_{2}(\mathbb{R}^{d})$.

We further assume

1. (A1)

Let $v:\mathcal{P}_{2}(\mathbb{R}^{d})\times\mathbb{R}^{dM}\to\text{Lip}_{loc}(\mathbb{R}^{d})$ be given, such that for all $(\mu,\textbf{u})\in\mathcal{P}_{2}(\mathbb{R}^{d})\times\mathbb{R}^{dM}$:

[TABLE]

where the constant $C_{l}>0$ is independent of $(\mu,\textbf{u})$.

We further define $v^{N}\colon\mathbb{R}^{dN}\times\mathbb{R}^{dM}\to\mathbb{R}^{dN}$ via

[TABLE]

where

[TABLE]

is the empirical measure for the state $\textbf{x}\in\mathbb{R}^{dN}$. 2. (A2)

For any two $(\mu,\textbf{u}),(\mu^{\prime},\textbf{u}^{\prime})\in\mathcal{P}_{2}(\mathbb{R}^{d})\times\mathbb{R}^{dM}$, there exists a constant $C_{v}>0$, independent of $(\mu,\textbf{u})$ and $(\mu^{\prime},\textbf{u}^{\prime})$, such that

[TABLE]

Remark 2.1*.*

By definition, $\mu_{t}^{N}$ assigns the probability $\mu_{t}^{N}(A)$ of finding particles with states within a measurable set $A\in\mathcal{B}(\mathbb{R}^{d})$ on the state space $\mathbb{R}^{d}$ at time $t\geq 0$.

Standard results from ODE theory yield the existence and uniqueness of a global solution.

Proposition 2.2**.**

Assume (A1) and (A2). Then, for given $\textbf{u}\in\mathcal{C}([0,T],\mathbb{R}^{dM})$ and $\hat{\textbf{x}}\in\mathbb{R}^{dN}$ there exists a unique global solution $\textbf{x}\in\mathcal{C}^{1}([0,T],\mathbb{R}^{dN})$ of (12).

Remark 2.3*.*

In particular, for applications in the control of crowds we have that $v^{N}$ models interactions, i.e. particle-particle and particle-control interactions by means of forces (see [11] and the references therein). Then, $v^{N}$ is often given by

[TABLE]

for given interaction forces $K_{1}$ and $K_{2}$ modeling the interactions within the cloud of particles itself and of the particles with the controls, respectively.

2.1.2. The mean-field model

In order to define the limiting problem for an increasing number of particles $N\to\infty$ explicitly, we consider the empirical measure $\mu^{N}$.

Using the ideas from [21, 8, 14] we derive the corresponding PDE formally as

[TABLE]

which is the mean-field 1-particle distribution evolution equation, supplemented with the initial condition $\hat{\mu}\in\mathcal{P}_{2}^{ac}(\mathbb{R}^{d})$, i.e. $\hat{\mu}$ has Lebesgue density.

Remark 2.4*.*

Here $v(\mu,\textbf{u})$ denotes the mean-field representation of $v^{N}(\textbf{x},\textbf{u})$. In fact, for the structure given by (13), we obtain

[TABLE]

In the mean-field setting we consider the following notion of solution.

Definition 2.5**.**

We call $\mu\in\mathcal{C}([0,T],\mathcal{P}_{2}(\mathbb{R}^{d}))$ a weak measure solution of (14) with initial condition $\hat{\mu}\in\mathcal{P}_{2}(\mathbb{R}^{d})$ iff for any test function $h\in\mathcal{C}_{0}^{\infty}([0,T)\times\mathbb{R}^{d})$ we have

[TABLE]

An existence and uniqueness result for solutions of (14) may be found, e.g., in [8, 10, 14, 18], where the notion of solution is established in the Wasserstein space $\mathcal{P}_{2}(\mathbb{R}^{d})$:

Proposition 2.6**.**

Assume (A1) and (A2) and let $\hat{\mu}\in\mathcal{P}_{2}(\mathbb{R}^{d})$. Then, for $\textbf{u}\in\mathcal{C}([0,T],\mathbb{R}^{dM})$ there exists a unique global (weak measure) solution $\mu\in\mathcal{C}([0,T],\mathcal{P}_{2}(\mathbb{R}^{d}))$ of (14). If additionally $\hat{\mu}\in\mathcal{P}_{2}^{ac}(\mathbb{R}^{d})$, then also $\mu\in\mathcal{C}([0,T],\mathcal{P}_{2}^{ac}(\mathbb{R}^{d}))$.

Further, for $\hat{\mu}_{\hat{\textbf{x}}}=1/N\,\sum_{i=1}^{N}\delta_{\hat{x}^{i}}$ we have $\mu_{\textbf{x},t}=\mu_{\textbf{x},t}^{N}$, where $\hat{\textbf{x}}$ is the initial condition of (12).

Remark 2.7*.*

Under the assumptions (A1) and (A2) we have enough regularity to use the classical method of characteristics to deduce for any $s\in[0,T]$ the existence of an unique global flow $Q_{\cdot}(\cdot,s)\in\mathcal{C}([0,T]\times\mathbb{R}^{d};\mathbb{R}^{d})$ satisfying

[TABLE]

In particular, for $s=0$ we obtain the nonlinear flow with a random initial condition $Q_{0}(x,0)$ distributed according to $\hat{\mu}$, i.e. $\text{law}(Q_{0}(x,0))=\hat{\mu}$. The solution $\mu$ of (14) may then be explicitly expressed as $\mu_{t}=Q_{t}(\cdot,0)\#\mu_{0}$ for all $t\geq 0.$ We shall make use of this representation at several points in the remainder. For simplicity we set $Q_{t}(x):=Q_{t}(x,0).$

The following stability statement will be useful in the coming results. Its proof may be found in Appendix A.

Lemma 2.8**.**

Let the assumptions (A1) and (A2) hold, $\mu$ and $\mu^{\prime}$ be solutions to the continuity equation (20) for given controls u, $\textbf{u}^{\prime}$ and initial data $\hat{\mu}$, $\hat{\mu}^{\prime}$, respectively. Then, there exist positive constants $a$ and $b$ such that

[TABLE]

We end this section with an important observation:

Remark 2.9*.*

We emphasize that the particle problem is just a special case of the mean-field problem specified by the inital condition. Indeed, for the initial condition $\hat{\mu}=1/N\,\sum_{i=1}^{N}\delta_{\hat{x}^{i}}$ we have $\mu_{t}=\mu_{t}^{N}$, where $\hat{\textbf{x}}$ is the initial condition of (12). Strictly speaking, we have only one optimization problem to consider in the following. Whether the problem at hand is of microscopic or mesoscopic type is determined by the initial condition.

2.2. Optimal Control Problem

We define the set of admissible controls as

[TABLE]

This choice of $\mathcal{U}_{\text{ad}}$ ensures the continuity of the controls (compare also the previous existence results).

For the study of the respective optimal control problem we require:

1. (A3)

The cost functional is of separable type, i.e.

[TABLE]

where $J_{2}$ is continuously differentiable, weakly lower semicontinuous and coercive on $\mathcal{U}_{\text{ad}}$. Further, $J_{1}(\mu)$ is a cylindrical function of the form

[TABLE]

where $j\in\mathcal{C}^{1}(\mathbb{R}^{L})$ and $g_{\ell}\in\mathcal{C}^{1}(\mathbb{R}^{d}),\;\ell=1,\dots,L,$ such that $\langle g_{\ell},\mu\rangle:=\int_{\mathbb{R}^{d}}g_{\ell}\,\mathrm{d}\mu<\infty$, and

[TABLE]

for some constant $C_{g}>0$. 2. (A4)

For the microscopic case, we define $J_{1}^{N}(\textbf{x}):=J_{1}(\mu_{\textbf{x}}^{N})$ as well as

[TABLE]

and assume that $J_{1}^{N}$ is continuously differentiable.

Remark 2.10*.*

1. (1)

Note, that the differentiability properties in the previous assumptions are only necessary for the derivation of the optimality conditions in the next sections, and not for the existence of the respective optimal controls. 2. (2)

(A4) essentially restricts the type of costs that can be considered for the particle system. In particular, the microscopic cost should have a corresponding mean-field counterpart. This is indeed the case whenever $J_{1}^{N}(\textbf{x})$ may be written as function acting on its corresponding empirical measure $\mu_{\textbf{x}}^{N}$.

A direct consequence of assumption (A3) is the continuity of $J_{1}$ in the Wasserstein metric.

Lemma 2.11**.**

Assume (A3) and let $\mu,\nu\in\mathcal{C}([0,T],\mathcal{P}_{2}(\mathbb{R}^{d}))$ with

[TABLE]

Then, there exists a constant $C_{j}>0$, independent of $t\in[0,T]$ such that

[TABLE]

Proof.

Let $\mu$ and $\nu\in\mathcal{P}_{2}(\mathbb{R}^{d})$ be arbitrary. Then, for each $\ell=1,\ldots,L$, we have by (A3), the mean-value theorem and Hölder’s inequality that

[TABLE]

where $\pi$ is the optimal coupling between $\mu$ and $\nu$. In particular, the estimate above shows that the mapping $\langle g_{\ell},\cdot\rangle:\mathcal{P}_{2}(\mathbb{R}^{d})\to\mathbb{R}$ is locally Lipschitz for every $\ell=1,\ldots,L$.

Denote $p_{t}=(\langle g_{1},\mu_{t}\rangle,\ldots,\langle g_{L},\mu_{t}\rangle)$ and $q_{t}=(\langle g_{1},\nu_{t}\rangle,\ldots,\langle g_{L},\nu_{t}\rangle)$. The assumptions on $\mu$ and $\nu$, and the previous estimate yields

[TABLE]

where we used the fact that $|(1-\tau)q_{t}+\tau p_{t}|\leq LM_{1}$ for all $\tau\in[0,1]$, $t\in[0,T]$. ∎

Remark 2.12*.*

Note that cost functionals that track the center of mass and the variance of a crowd satisfy (A3) and (A4). In fact, for $\lambda_{1},\lambda_{2},\lambda_{3}>0$,

[TABLE]

fit into the setting. Therefore, the assumptions are rather general and not restrictive for applications (cf. [9]).

The well-posedness of the state problem justifies the notation $\mu(\textbf{u})$ assigning the unique solution of the state equation to the control. Then, the optimal control problem we investigate in the following is given by

Problem 1**:**

Find $\bar{\textbf{u}}\in\mathcal{U}_{\text{ad}}$ such that

[TABLE]

For later use, we note that in the particle case, i.e. for discrete initial data (cf. Remark 2.9), we can rewrite the optimization problem as follows:

For $N\in\mathbb{N}$ fixed, find $\bar{\textbf{u}}^{N}\in\mathcal{U}_{\text{ad}}$ such that

[TABLE]

Using the standard argument based on the boundedness of a minimizing sequence in $\mathcal{U}_{\text{ad}}$ and continuity properties of $J$ stated in (A3) and (A4), we obtain the following existence result:

Theorem 2.13**.**

Assume (A1)–(A4). Then, the optimal control problem (${\bf P_{\infty}}$) has a solution $(\mu(\bar{\textbf{u}}),\bar{\textbf{u}})\in\mathcal{C}([0,T],\mathcal{P}_{2}(\mathbb{R}^{d}))\times\mathcal{U}_{\text{ad}}$.

Remark 2.14*.*

The well-posedness of (${\bf P_{N}}$) follows directly from the above theorem, as the particle problem is a special case of (${\bf P_{\infty}}$), see Remark 2.9. Nevertheless, one can prove the well-posedness of (${\bf P_{N}}$) also directly using classical techniques in the optimal control of ODEs.

3. First-order optimality conditions in the Wasserstein space $\mathcal{P}_{2}(\mathbb{R}^{d})$

The main objective of this section is to derive the first-order optimality conditions (FOC) for the optimal control problem (${\bf P_{\infty}}$) in the framework of probability measures with bounded second moment equipped with the 2-Wasserstein distance. For the sake of a smooth presentation we restrict the interaction terms to the special ones defined in (13) and (15), respectively. This allows us to pose the following regularity assumption

1. (A5)

$K_{1},K_{2}\in\mathcal{C}_{b}^{2}(\mathbb{R}^{d})$.

Remark 3.1*.*

Note that assumption (A5) directly implies that $(t,x)\mapsto v(\mu_{t},\textbf{u}_{t})(x)$ defined by (15) is an element of $\mathcal{C}_{b}^{1}([0,T]\times\mathbb{R}^{d},\mathbb{R}^{d})$ for every $\mu\in\mathcal{C}([0,T],\mathcal{P}_{2}(\mathbb{R}^{d}))$ and $\textbf{u}\in\mathcal{C}([0,T],\mathbb{R}^{dM})$ with

[TABLE]

In particular, the flow $(t,x)\mapsto Q_{t}(x)$ is $\mathcal{C}_{b}^{1}([0,T]\times\mathbb{R}^{d},\mathbb{R}^{d})$ by standard arguments (cf. [15]).

For given initial condition $\hat{\mu}$ we define the state space as

[TABLE]

As the optimization in the $W_{2}$ setting is not well-known, we begin by discussing known results (see [3, Chapter 8.1]) regarding the constraint

[TABLE]

Recall Proposition 2.6 that provides for each $\textbf{u}\in\mathcal{U}_{\text{ad}}$ a unique solution $\mu\in\mathcal{C}([0,T],\mathcal{P}_{2}^{ac}(\mathbb{R}^{d}))$ of (20). In particular, $\mu$ satisfies

[TABLE]

for all $\varphi\in\mathcal{A}:=\mathcal{C}_{c}^{1}([0,T]\times\mathbb{R}^{d})$. Therefore, there is a well-defined solution operator $S\colon\textbf{u}\mapsto\mu$, which allows us to recast the constrained minimization problem as

[TABLE]

where $\hat{J}$ is the so-called reduced functional.

Definition 3.2**.**

A pair $(\mu,\textbf{u})\in\mathcal{Y}\times\mathcal{U}_{\text{ad}}$ is said to be admissible if $E(\mu,\textbf{u})[\varphi]=0$ for all $\varphi\in\mathcal{A}$.

Unfortunately, the reduced cost functional is not handy in deriving the first-order optimality conditions for (${\bf P_{\infty}}$). For this reason, we will take an extended-Lagrangian approach. We begin by observing that (${\bf P_{\infty}}$) may be recast as

[TABLE]

which may be further reformulated as

[TABLE]

Indeed, notice that $\sup_{\varphi\in\mathcal{A}}E(\mu,\textbf{u})[\varphi]\geq 0$, since $\varphi\equiv 0$ implies $E(\mu,\textbf{u})[0]=0$ for every $(\mu,\textbf{u})$. Therefore, if $E(\mu,\textbf{u})[\varphi]>0$ for some $\varphi$, the linearity in $\varphi$ of $E$ yields $E(\mu,\textbf{u})[\alpha\varphi]=\alpha E(\mu,\textbf{u})[\varphi]$ for every $\alpha>0$, which consequently shows that $\sup_{\varphi}E(\mu,\textbf{u})[\varphi]=+\infty$.

Under the separation assumption on $J$, i.e. $J(\mu,\textbf{u})=J_{1}(\mu)+J_{2}(\textbf{u})$, (22) becomes

[TABLE]

with

[TABLE]

In the following we derive a necessary condition for $(\mu,\textbf{u})$ to be a stationary point. Let $(\bar{\mu},\bar{\textbf{u}})$ be an optimal pair, $\delta\geq 0$ and $\textbf{u}^{\delta}=\bar{\textbf{u}}+\delta\textbf{h}$ be a perturbation of $\bar{\textbf{u}}$ for an arbitrary smooth map $\textbf{h}\colon(0,T)\rightarrow\mathbb{R}^{dM},$ such that $\textbf{u}^{\delta}\in\mathcal{U}_{\text{ad}}$ and there exists a unique $\mu^{\delta}\in\mathcal{C}([0,T],\mathcal{P}_{2}^{ac}(\mathbb{R}^{d}))$ satisfying $E(\mu^{\delta},\textbf{u}^{\delta})[\varphi]=0$ for all $\varphi\in\mathcal{A}.$ Then

[TABLE]

and the directional derivative of $\mathcal{G}:=J_{2}+\chi$ at $\bar{\textbf{u}}$ along h is given by

[TABLE]

which requires us to know the relationship between $\mu^{\delta}$ and $\bar{\mu}$.

Remark 3.3*.*

Note, that Lemma 2.8 above provides a stability estimate of the form

[TABLE]

for appropriate constants $a,b>0$. Hence, for each $t\in[0,T]$, the curve $[0,\infty)\ni\delta\mapsto\mu^{\delta}_{t}\in\mathcal{P}_{2}(\mathbb{R}^{d})$ starting from $\mu_{t}$ at $\delta=0$ is absolutely continuous w.r.t. the 2-Wasserstein distance. In this case, there exists a vector field $\psi_{t}\in L^{2}(\mu_{t},\mathbb{R}^{d})$ for each $t\in[0,T]$ satisfying [3, Proposition 8.4.6]

[TABLE]

Furthermore,

[TABLE]

where the explicit coupling $\pi_{t}=(id+\delta\psi_{t},id)_{\#}\mu_{t}$ was used. In particular, we have that

[TABLE]

The previous remark allows us to establish an explicit relationship between $\psi_{t}$ and h.

Lemma 3.4**.**

Let $(\mu,\textbf{u})$ be an admissible pair, $\textbf{h}\in\mathcal{C}_{c}^{\infty}((0,T),\mathbb{R}^{dM})$ and $\textbf{u}^{\delta}=\textbf{u}+\delta{\bf h}$ such that

1. (i)

$\textbf{u}^{\delta}\in\mathcal{U}_{\text{ad}}$, and 2. (ii)

there exists $\mu^{\delta}\in\mathcal{C}([0,T],\mathcal{P}_{2}^{ac}(\mathbb{R}^{d}))$ satisfying $E(\mu^{\delta},\textbf{u}^{\delta})=0$,

for $0<\delta\ll 1$ sufficiently small. If $\psi\in\mathcal{C}_{b}^{1}((0,T)\times\mathbb{R}^{d})$ with $\psi_{0}\equiv 0$ satisfies

[TABLE]

for a bounded Borel map $(t,x)\mapsto\mathcal{K}(\mu_{t},\textbf{u}_{t})[\psi_{t},\textbf{h}_{t}](x)$ satisfying

[TABLE]

then (23) holds with this $\psi$, i.e.

[TABLE]

Proof.

For each $t\in[0,T]$, we set $\nu_{t}^{\delta}:=(id+\delta\psi_{t})_{\#}\mu_{t}$. We begin by showing that the curve $t\mapsto\nu_{t}^{\delta}\in\mathcal{P}_{2}(\mathbb{R}^{d})$ is absolutely continuous. Due to the assumed regularity on $\psi$ satisfying (24), the chain-rule applies, and we obtain for any $F\in\mathcal{C}_{c}^{\infty}(\mathbb{R}^{d})$, and almost every $t\in(0,T)$,

[TABLE]

Furthermore, by the assumption on $\mathcal{K}$, we have that

[TABLE]

Along with the previous computation, we find that $t\mapsto\nu_{t}^{\delta}\in\mathcal{P}_{2}(\mathbb{R}^{d})$ is an absolutely continuous curve satisfying the continuity equation

[TABLE]

Consequently, we can consider the temporal derivative of $t\mapsto W_{2}^{2}(\mu_{t}^{\delta},\nu_{t}^{\delta})$ to obtain

[TABLE]

To estimate $\mathrm{(I)}$, we use assumptions (A1) and (A2) to obtain

[TABLE]

As for $\mathrm{(II)}$, we have

[TABLE]

which, together with the estimate for $\mathrm{(I)}$, gives

[TABLE]

for some constant $C>0$, and where

[TABLE]

Since $W_{2}(\mu_{0}^{\delta},\nu_{0}^{\delta})=0$, an application of Gronwall’s inequality yields

[TABLE]

due to the assumption on $\mathcal{K}$ in (25), thereby concluding the proof. ∎

Remark 3.5*.*

We mention that for any ${\bf h}\in\mathcal{C}_{c}^{\infty}((0,T),\mathbb{R}^{dM})$ and any sufficiently smooth mapping $F\colon\mathbb{R}^{d}\to\mathbb{R}^{d}$, we have for ${\bf x}^{\delta}={\bf x}+\delta{\bf h}$:

[TABLE]

In particular, for the velocity field $v$ given in (15) one deduces

[TABLE]

which satisfies

[TABLE]

From assumption (A5), it is not difficult to see that (25) is satisfied.

The existence of a $\psi\in\mathcal{C}_{b}^{1}((0,T)\times\mathbb{R}^{d})$ satisfying the assumptions of Lemma 3.4 is provided in the following statement.

Theorem 3.6**.**

Let the assumptions of Lemma 3.4 hold. For the velocity field $v:\mathcal{P}_{2}(\mathbb{R}^{d})\times\mathbb{R}^{dM}\to\text{Lip}_{loc}(\mathbb{R}^{d})$ given in (15) there exists $\psi\in\mathcal{C}_{b}^{1}((0,T)\times\mathbb{R}^{d})$ with $\psi_{0}=0$ satisfying

[TABLE]

where $\mathcal{K}$ is given in (26).

Proof.

We consider $\Gamma=\mathcal{C}([0,T],\mathcal{C}_{b}^{1}(\mathbb{R}^{d},\mathbb{R}^{d}))$ and the operator

[TABLE]

First, we have to show that $H(\omega)\in\Gamma.$ Due to (15), (A5) and the properties of the flow discussed in Remark 3.1, we have $DH(\omega)(t)\in\mathcal{C}_{b}(\mathbb{R}^{d})$ and continuous w.r.t. $t$. Therefore, it holds $H(\omega)\in\mathcal{C}([0,T],\mathcal{C}_{b}^{1}(\mathbb{R}^{d})).$ In particular, $H\colon\Gamma\rightarrow\Gamma$ is well-defined.

To establish the contraction property of $H,$ we equip $\mathcal{C}([0,T],\mathcal{C}_{b}^{1}(\mathbb{R}^{d},\mathbb{R}^{d}))$ with the weighted norm

[TABLE]

for some $\lambda>0$ to be specified below. Note that $(\mathcal{C}([0,T],\mathcal{C}_{b}^{1}(\mathbb{R}^{d},\mathbb{R}^{d})),\|\cdot\|_{\text{exp}})$ is complete.

Using the structure of $\mathcal{K}$ in Remark 3.5, we obtain

[TABLE]

As for the space derivative we obtain

[TABLE]

We define $C_{v}=2\|Dv\|_{\infty}+\|DK_{1}\|_{\infty}+\|D^{2}v\|_{\infty}+\|D^{2}K_{1}\|_{\infty}$ and add the two inequalities to obtain

[TABLE]

Multiplying each of the above estimates with $e^{-\lambda}$ and taking the supremum over $t$ and $x$ leads to

[TABLE]

Choosing $\lambda>C_{v}$ allows us to conclude the contraction property of $H.$ An application of the Banach fixed-point theorem yields a solution $\psi\in\mathcal{C}([0,T],\mathcal{C}_{b}^{1}(\mathbb{R}^{d},\mathbb{R}^{d}))$ given by

[TABLE]

It is straight forward to see that

[TABLE]

Finally, a direct computation shows that $\psi$ satisfies the evolution equation. ∎

Now, we are able to state the first-order necessary condition for $(\mu,\textbf{u})$ to be a stationary point.

Theorem 3.7**.**

Let $(\bar{\mu},\bar{\textbf{u}})$ be an optimal pair, $J_{2}$ be Gâteaux-differentiable, and $J_{1}$ a cylindrical function of the form given in (A3). Then, for any $\textbf{h}\in\mathcal{C}_{c}^{\infty}((0,T),\mathbb{R}^{dM})$ it holds:

[TABLE]

where

[TABLE]

and $t\mapsto\psi_{t}\in L^{2}(\mu_{t},\mathbb{R}^{d})$ satisfying (24) with initial condition $\psi_{0}=0$.

Proof.

Since $J_{1}(\mu)$ is a cylindrical function, we have that

[TABLE]

where $\psi$ satisfies (24) with $\psi_{0}=0$. Therefore, owing to the minimality of $\bar{\textbf{u}}$, we find

[TABLE]

Passing to the limit $\delta\to 0+$ yields

[TABLE]

for any ${\textbf{h}}\in\mathcal{C}_{c}^{\infty}((0,T),\mathbb{R}^{dM})$. Notice, however, that changing the sign of h leads to a change of sign of $\psi$, which then provides the equality (27). ∎

In order to provide an adjoint-based first-order optimality system, we now derive the equation for the dual variable. We consider the dual problem corresponding to (24), by testing the equation (24) with a family of vector-valued measures $(m_{t})_{t\in(0,T)}$ to obtain

[TABLE]

and set $\mathcal{K}(\mu,\textbf{u})[\psi,h]=\mathcal{K}^{1}(\mu,\textbf{u})[\psi]+\mathcal{K}^{2}(\textbf{u})[\textbf{h}]$, where

[TABLE]

Using $\psi_{0}=0$ and integrating by parts, we obtain

[TABLE]

where

[TABLE]

By choosing $\bar{m}$ to satisfy the dual problem

[TABLE]

subject to the terminal condition $\bar{m}_{T}=0,$ we find with the help of the optimality condition (27), that

[TABLE]

Remark 3.8*.*

If $|\bar{m}_{t}|\ll\bar{\mu}_{t}$ for every $t\in[0,T]$, i.e. there is a vector field $\bar{\xi}_{t}:\mathbb{R}^{d}\to\mathbb{R}^{d}$ such that $\bar{m}_{t}=\bar{\xi}_{t}\bar{\mu}_{t}$, where $\bar{\mu}$ satisfies (20), then equation (30) formally reduces to

[TABLE]

Remark 3.9*.*

If we further assume that $K_{1}$ and $K_{2}$ are gradients of potential fields, then $\nabla K_{1}$ and $\nabla K_{2}$ are symmetric and the previous equation takes the simpler form

[TABLE]

In this case, one can expect $\bar{\xi}$ to be a gradient of a potential field (compare also the results in [19]), i.e. $\bar{\xi}=\nabla\bar{\phi}$ for a function $\bar{\phi}$ satisfying the scalar equation

[TABLE]

3.1. Well-posedness of the adjoint equation

To obtain the well-posedness of the adjoint equation (30) we make use of equation (32). Indeed, due to assumptions (A3) and (A5), we can make use of the method of characteristics and Banach’s fixed-point theorem.

Theorem 3.10**.**

Let assumptions (A1)-(A5) hold and $(\mu,\textbf{u})$ be admissible with initial condition $\hat{\mu}\in\mathcal{P}(\mathbb{R}^{d})$ having compact support. Then, the equation

[TABLE]

with

[TABLE]

has a unique solution $\xi\in\mathcal{C}([0,T]\times\mathbb{R}^{d},\mathbb{R}^{d})$ with the representation

[TABLE]

where $Q$ satisfies (16). In particular, $m=\xi\mu$ yields a distributional solution of (30).

Proof.

We begin by recalling that the Lagrangian flow satisfies

[TABLE]

For any $\omega\in\Gamma:=\mathcal{C}([0,T]\times\mathbb{R}^{d},\mathbb{R}^{d})$, we define the operator

[TABLE]

Observe that $H(\omega)\in\Gamma$ due to the properties of the Lagrangian flow, and the fact that $p\in\mathcal{C}_{b}(\Omega)$ and $K_{1},K_{2}\in\mathcal{C}_{b}^{1}(\mathbb{R}^{d})$ by assumption (A5). In particular, $H\colon\Gamma\to\Gamma$ is a well-defined mapping.

To show that $H$ is a contraction on $\Gamma$, we first define a norm on $\Gamma$ given by

[TABLE]

where $c_{K}=\lVert DK_{1}\rVert_{\sup}+\lVert DK_{2}\rVert_{\sup}.$ We note that $(\Gamma,\lVert\cdot\rVert_{\exp})$ is complete and the estimate

[TABLE]

holds true for any $\omega^{1}$, $\omega^{2}\in\Gamma$. Taking the supremum over $x\in\mathbb{R}^{d}$ in the inequality above, multiplying with $e^{-4c_{K}(T-t)}$ and then taking the supremum over $t\in[0,T]$ yields

[TABLE]

Therefore, the Banach fixed-point theorem provides a unique $\xi\in\Gamma$ satisfying (35). ∎

Summarizing the above computations, we end up at the following result.

Theorem 3.11**.**

A minimizing pair $(\bar{\mu},\bar{\textbf{u}})$ of the problem (${\bf P_{\infty}}$) satisfies

[TABLE]

where the adjoint variable $\bar{m}$ satisfies

[TABLE]

subject to the conditions

[TABLE]

Note, that in the case of the cost functional given in Remark 2.12 the optimality condition turns out to be a boundary value problem in time. In fact, we obtain as explicit representation

[TABLE]

for $h=(h^{\ell})_{\ell=1,\dots,M}\in H^{1}((0,T),\mathbb{R}^{dM})$ with $h_{0}=0$. In particular, the variational lemma yields

[TABLE]

4. Relations between first-order optimality systems

In order to discuss the links of first-order optimality system in the space of probability measures derived in the previous section to the one of the ODE constrained problem and the optimization problem based on a classical $L^{2}$-approach, we shall give the respective first-order optimality systems in the interest of completeness.

4.1. First-order optimality conditions in the microscopic setting

We derive the first-order optimality conditions for the microscopic case by the classical $L^{2}$-approach. Again, the set of admissible controls $\mathcal{U}_{\text{ad}}$ is defined as above. The state space $Y$ is the Hilbert space

[TABLE]

Further, we define

[TABLE]

the space of Lagrange multipliers with the dual $Z^{*}=Z$. This allows us to define the state operator $e_{N}\colon Y\times\mathcal{U}_{\text{ad}}\rightarrow Z$ for the microscopic system as

[TABLE]

and the weak form

[TABLE]

We note that due to $Y\hookrightarrow\mathcal{C}([0,T],\mathbb{R}^{dN})$ the evaluation of $\textbf{x}_{0}$ is justified. Let $(\boldsymbol{\xi},\boldsymbol{\eta})\in Z$ denote the Lagrange multipliers. Then, the Lagrangian corresponding to (${\bf P_{N}}$) with $N\in\mathbb{N}$ fixed reads

[TABLE]

Remark 4.1*.*

Note that the $J_{N}$ is multiplied with $N$ to obtain the appropriate balance between the two terms in the Lagrangian as $N\to\infty$.

As usual, the first-order necessary optimality condition is derived by solving

[TABLE]

Exploiting (A3)–(A5) we can calculate for any $h=(h_{\textbf{x}},h_{\textbf{u}})\in Y\times\mathcal{U}_{\text{ad}}$ the following Gâteaux derivatives of the cost functional

[TABLE]

and for the second part of the Lagrangian

[TABLE]

Assuming further, that $\boldsymbol{\xi}\in Y$, one may formally derive the strong formulation of the adjoint system. Indeed, using integration by parts we arrive at the following result.

Theorem 4.2**.**

Let $(\bar{x}^{N},\bar{u}^{N})$ be an optimal pair. The optimality condition corresponding to (${\bf P_{N}}$), with $N\in\mathbb{N}$ fixed, reads

[TABLE]

where $\bar{\boldsymbol{\xi}}^{N}\in Y$ satisfies the adjoint system given by

[TABLE]

supplemented with the terminal condition $\bar{\xi}^{N}_{T}=0.$

Similar to the previous case, we obtain for the cost functional given in Remark 2.12 the boundary value problem

[TABLE]

Further, for the special structure of the interaction forces defined in (13) and $J$ given by (A3) we obtain for the adjoint equation

[TABLE]

with terminal condition $\xi^{i}_{T}=0$.

Remark 4.3*.*

Using a similar idea as in the proof in the Appendix (Gronwall inequality), it is not difficult to see that under assumption (A5), $\boldsymbol{\xi}^{N}$ satisfying (39) enjoys the uniform bound

[TABLE]

where $C_{\xi}>0$ is independent of $N\in\mathbb{N}$, and depends only on $DK_{i}$, $Dj$ and $Dg$.

Remark 4.4*.*

Defining the vector-valued measure $m_{t}^{N}:=(1/N)\sum_{i=1}^{N}\xi_{t}^{i}\delta_{x_{t}^{i}}$, we have by construction that $m_{t}$ satisfies

[TABLE]

for all $\varphi\in\mathcal{C}_{c}^{\infty}(\mathbb{R}^{d})$. In other words, $m_{t}^{N}$ is a distributional solution of the equation

[TABLE]

We emphasize that (40) coincides with the adjoint equation in the mean-field setting (30).

4.2. First-order optimality conditions in the mean-field setting: $L^{2}$-approach

To be able to work in the classical $L^{2}$-setting, we will need additional assumptions to obtain Lebesgue integrable solutions:

1. (A6)

There exists a compact $\Omega_{0}\subset\mathbb{R}^{d}$ such that the initial condition satisfies $\text{supp}(\hat{\mu})\in\Omega_{0}.$ 2. (A7)

The initial measure $\hat{\mu}$ has a Lebesgue density $\hat{f}\in L^{2}(\Omega_{0})$.

In particular, (A5)-(A7) ensure the boundedness of the support of $\mu_{t}$ for all times $t\in[0,T]$. Hence, we can fix a bounded domain $\Omega\subset\mathbb{R}^{d}$ with smooth boundary containing the support of $\mu_{t}$ for all times $t\in[0,T].$ In this section we strongly use that $\mu$ is absolutely continuous w.r.t. the Lebesgue measure and denote its density by $f_{t}=d\mu_{t}/dx$ with initial condition $f_{0}=\hat{f}$. Then, we define the state space of the PDE optimization problem as

[TABLE]

Let $\mathcal{X}=L^{2}((0,T),H^{1}(\Omega))$ and $\mathcal{Z}=\mathcal{X}\times L^{2}(\Omega)$ be the space of adjoint states with dual $\mathcal{Z}^{*}.$ The control space $\mathcal{U}_{\text{ad}}$ was already defined in (17). For the derivation of the adjoints we consider here only the special case given by (13) and (A4). We define the mapping $e_{\infty}\colon\mathcal{Y}\times\mathcal{U}_{\text{ad}}\rightarrow\mathcal{Z}^{*}$ by

[TABLE]

with adjont state $(q,\eta)\in\mathcal{Z}$. The Lagrangian corresponding to (${\bf P_{\infty}}$) reads

[TABLE]

Analogously to the microscopic case, we derive the adjoint system and the optimality condition by calculating the derivatives of $\mathcal{L}_{\text{meso}}$ w.r.t. the state variable and the control. The standard $L^{2}$-calculus yields

[TABLE]

for the cost functional and

[TABLE]

for the state operator. Assuming additionally $q\in\mathcal{Y}$, we may integrate by parts to obtain a strong formulation of the adjoint system. This yields the following optimality system.

Theorem 4.5**.**

Let $(f,u)$ be an optimal pair. The optimality condition corresponding to (${\bf P_{\infty}}$) reads

[TABLE]

where $q\in\mathcal{Y}$ satisfies the adjoint PDE given by

[TABLE]

supplemented with the terminal condition $g_{T}=0$.

The adjoint equation (41) derived via the $L^{2}$-approach clearly resembles (33).

Remark 4.6*.*

As before, in the case (13) the optimality conditions can be given explicity as

[TABLE]

A comparison with the optimality condition on the micro indicates a relation between $\nabla q$ and $\xi$ which will be further discussed in the following.

4.3. Relations between the approaches

In this section we discuss the relation between the adjoint derived w.r.t. the 2-Wasserstein distance and the gradient flow equation corresponding to the Hamiltonian approach (cf. [17]). In order to define the probability measure containing forward and backward information we first recall the flow formulation of the state system

[TABLE]

Further, we introduce the adjoint flow $A_{t}$ corresponding to $\xi_{t}$, defined by $A_{t}=\xi_{t}\circ Q_{t}$. Its evolution equation is given by

[TABLE]

with terminal condition $A_{T}(x)=0$.

Remark 4.7*.*

We would like to point out that (43) can also be derived directly from the state flow with the help of a Lagrangian-approach w.r.t. the $L^{2}$-scalar product. A change of coordinates from the Lagrangian to the Eulerian perspective leads to (32).

Due to the strong dependence of the adjoint flow on the forward flow, one may understand (42) and (43) as a coupled system of equations. Let us consider the measure $\nu\in\mathcal{C}([0,T],\mathcal{P}_{1}(\mathbb{R}^{d}\times\mathbb{R}^{d}))$ defined by the push-forward of $\mu_{0}$ along the map $S_{t}(x)=(Q_{t}(x),A_{t}(x))$ for all $x\in\mathbb{R}^{d}$ and $t\in[0,T]$:

[TABLE]

Notice that since

[TABLE]

the first marginal of $\nu_{t}$ corresponds to $\mu_{t}$. In particular,

[TABLE]

Furthermore,

[TABLE]

i.e. $\nu_{T}(\mathbb{R}^{d}\times B)=\delta_{0}(B)$ for all $B\in\mathcal{B}(\mathbb{R}^{d})$.

From the definition of $\nu$, it is not difficult to see that $\nu$ satisfies

[TABLE]

with mixed initial and terminal data given by

[TABLE]

where the Hamiltonian (cf. [6]) corresponding to (${\bf P_{\infty}}$) is given by

[TABLE]

On the other hand, (44) can also be derived from a mean-field Ansatz [17]. Indeed, starting from the system of forward and adjoint ODEs, leads to the empirical measure $\nu^{N}$ defined as

[TABLE]

which satisfies (44). More details can be found in, e.g., [27, 9].

We conclude this section with a discussion of the relation of $\nu$ and the vector-valued adjoint variable $m$ defined by (30). More precisely, we show that $m$ satisfying (30) can be characterized as first moment of $\nu$ with respect to $\xi$. We use the notation $\omega_{t}(\,\mathrm{d}x):=\int_{\mathbb{R}^{d}}\xi\nu_{t}(\,\mathrm{d}x,\,\mathrm{d}\xi)$. Since by construction,

[TABLE]

the measure $\omega_{t}$ is well-defined, and satisfies (30) with the terminal condition

The above discussion yields the following result:

Proposition 4.8**.**

The adjoint corresponding to (${\bf P_{\infty}}$) derived in the Wasserstein space $\mathcal{P}_{2}(\mathbb{R}^{d})$ solves (32) and can be characterized as the first moment w.r.t. $\xi$ of the probability measure $\nu$ corresponding to the Hamiltonian flow (44) of (${\bf P_{\infty}}$) with Hamiltonian given by (45).

The findings of this section are summarized in Figure 1. On the ODE level the adjoints can be computed using the $L^{2}$-approach. Passing to the mean-field limit with the empirical measure (46) yields an evolution equation for a probability measure on the state and adjoint space (44). The first $\xi$-moment of $\nu$ satisfies the same equation as the adjoint equation derived in the space of probability measures equipped with the 2-Wasserstein distance, i.e. (30). The evolution of point masses following the characteristics of the mean-field adjoint equation equals the solution of the adjoint ODE with states initialized at the corresponding points. Moreover, we formally obtain a relation of the $L^{2}$-adjoint (41) and (32), whenever $K_{1}$ and $K_{2}$ are gradients of potential fields. Indeed, taking the gradient of the evolution equation of $g$ yields (32) for $\nabla g=\xi$ (see Remarks 3.9 and 4.6).

Remark 4.9*.*

As the adjoint equation obtained using the calculus in the space of probability measures is vector-valued, it may be infeasible for numerical simulations for higher space dimensions. The link between the vector-valued adjoint and the $L^{2}$-adjoint discussed in this section can be seen as justification to use the $L^{2}$-adjoint for numerics. Indeed, in [9] this procedure leads to very convincing results.

5. Convergence Rate

In this section we investigate the convergence of the microscopic optimal controls to the optimal control of the mean-field problem as $N\rightarrow\infty$. Our strategy for the proof is to use flows to pull the information back to the initial data. For simplicity we assume that $J_{2}(u)$ and $v$ have the structures given in Remark 2.12 and (15), respectively, in more detail:

[TABLE]

For the initial data we assume convergence as $N\to\infty$. This can be realized by drawing samples from the initial measure $\hat{\mu}$ for the particles (see Remark 5.3).

To summarize, the goal of this section is to prove

Theorem 5.1**.**

Let the assumptions (A1)-(A6) hold and $J_{2}(\textbf{u})$ as above. Further, let $(\bar{\textbf{x}}^{N},\bar{\textbf{u}}^{N})$ and $(\bar{\mu},\bar{\textbf{u}})$ be optimal pairs for (${\bf P_{N}}$) and (${\bf P_{\infty}}$) with initial data $\hat{\textbf{x}}^{N}$, $\hat{\mu}$, respectively. Moreover, let the adjoint velocity for the pair $(\bar{\mu},\bar{\textbf{u}})$ satisfy $\bar{\xi}\in\mathcal{C}([0,T],\text{Lip}_{b}(\mathbb{R}^{d}))$. Then there exists a constant $\gamma>0$ depending only on $T$, $K_{1}$, $K_{2}$, $J_{1}$ and Lipschitz bounds on $\bar{\xi}$, such that for $\lambda>\gamma$ it holds

[TABLE]

where $\mu_{\hat{x}^{N}}^{N}$ denotes the empirical measures corresponding to the initial configurations $\hat{\textbf{x}}^{N}$.

Remark 5.2*.*

Note, that we cannot expect that the solutions of the respective optimal control problems are unique. Hence, we need to ensure that our problem is convex enough, i.e. $\lambda$ is large enough. Essentially, we require here some kind of second-order sufficient condition or, equivalently, a quadratic growth condition near to the optimal state (see also [13]).

Remark 5.3*.*

Theorem 5.1 show that the convergence rate strongly depends on the convergence of the initial measures $W_{2}(\mu_{\hat{x}^{N}}^{N},\hat{\mu})\to 0$. Since $\mu$ is assumed to have compact support, we obtain a convergence of order $\sqrt{N}$ (cf. [28]), if one chooses $\hat{\textbf{x}}^{N}$ as random variables with distribution $\hat{\mu}$.

Remark 5.4*.*

The proof of the convergence rate can be obtained as well in a slightly different setting, i.e. without fixing the initial positions of the controls. Indeed, for

[TABLE]

one obtains a similar proof without using a Poincaré inequality.

We begin with a simple result (without proof) on $v=v(\mu,\textbf{u})$ and $J_{1}$.

Lemma 5.5**.**

1. (i)

Under assumption (A5), the mapping $v:\mathcal{P}_{2}(\mathbb{R}^{d})\times\mathbb{R}^{dM}\to\mathcal{C}_{b}^{2}(\mathbb{R}^{d})$ defined by (15) satisfies for any $\mu,\mu^{\prime}\in\mathcal{P}_{2}(\mathbb{R}^{d})$ and $\textbf{u},\textbf{u}^{\prime}\in\mathbb{R}^{dM}$

[TABLE]

for some constant $C_{v}$, independent of $(\mu,\textbf{u})$ and $(\mu^{\prime},\textbf{u}^{\prime})$. 2. (ii)

If in addition to (A3), $j\in\mathcal{C}^{2}(\mathbb{R}^{K})$ and $g_{l}\in\mathcal{C}^{2}(\mathbb{R}^{d})$, $l=1,\ldots,K$ then $\delta_{\mu}J_{1}$ defined in (28) satisfies

[TABLE]

for some constant $C_{J_{1}}$, depending only on $J_{1}$, $\mathfrak{m}_{2}(\mu)$ and $\mathfrak{m}(\mu^{\prime})$.

Remark 5.6*.*

Note that if $\sup_{t\in[0,T]}\{\mathfrak{m}_{2}(\mu_{t})+\mathfrak{m}_{2}(\mu_{t}^{\prime})\}<\infty$, then the time dependent constants $C_{J_{1}}(t)$ in Lemma 5.5 are uniformly bounded in $t$, i.e. $\sup_{t\in[0,T]}C_{J_{1}}(t)<\infty$.

We now proceed with a stability estimate for the adjoint velocities ${\boldsymbol{\xi}}^{N}$ and $\xi$ corresponding to the equations (39) and (32), respectively.

Remark 5.7*.*

Equation (39) can be written in the concise integral form

[TABLE]

where $\Psi^{N}_{i}$ is given by

[TABLE]

in connection to the operator $\Psi$ defined in (34).

Lemma 5.8**.**

Let the assumptions (A1)-(A6) hold. Further, let $\textbf{x}^{N}$ and $\mu$ be solutions to (12) and (20) for given controls $\textbf{u}^{N}$, u and initial data $\hat{\textbf{x}}^{N}$, $\hat{\mu}$, respectively. If $\boldsymbol{\xi}^{N}$ satisfies (39) for the pair $(\textbf{x}^{N},\textbf{u}^{N})$ and $\xi\in\mathcal{C}([0,T],\text{Lip}_{b}(\mathbb{R}^{d}))$ satisfies (32) for the pair $(\mu,\textbf{u})$, then there exist positive constants $a$ and $b$, independent of $N\in\mathbb{N}$ such that

[TABLE]

Proof.

Denote by $\mu^{N}$ the empirical measure corresponding to the particles $\textbf{x}^{N}$. We further denote $C_{v,J_{1}}:=C_{v}+\sup\nolimits_{t\in[0,T]}C_{J_{1}}(t)$ with $C_{v}$ and $C_{J_{1}}(t)$ given in Lemma 5.5 for each $t\in[0,T]$. Due to Remark 5.6, $C_{v,J_{1}}<\infty$. From Remark 5.7, we see that $\boldsymbol{\xi}^{N}$ satisfies (47), and therefore,

[TABLE]

where

[TABLE]

From Lemma 5.5, we easily deduce that

[TABLE]

As for (II), we have

[TABLE]

where $\pi_{s}$ is an optimal coupling between $\mu_{s}^{N}$ and $\mu_{s}$.

Defining

[TABLE]

we find positive constants $a,b>0$, independent of $N$ (cf. Remark 4.3) such that

[TABLE]

An application of Gronwall’s inequality gives

[TABLE]

Taking the supremum over $t\in[0,T]$ yields the required estimate. ∎

Remark 5.9*.*

Putting Lemma 5.8 and Lemma 2.8 together, we obtain the estimate

[TABLE]

for some positive constant $C_{T}$, independent of $N\in\mathbb{N}$.

Proof of Theorem 5.1.

In the following, let $(\bar{\textbf{x}}^{N},\bar{\textbf{u}}^{N})$ and $(\bar{\mu},\bar{\textbf{u}})$ be optimal pairs for (${\bf P_{N}}$) and (${\bf P_{\infty}}$) respectively. Further, let $\bar{\boldsymbol{\xi}}^{N}$ and $\bar{\xi}$ be adjoint velocities of the $N$-particle trajectories and mean-field limit corresponding to (39) and (32) respectively. We also denote by $\bar{\mu}^{N}$ the empirical measure corresponding to the particles $\bar{\textbf{x}}^{N}$.

Recall the optimality conditions for $\bar{\textbf{u}}^{N}$ and $\bar{\textbf{u}}$, given by (37) and (31), respectively. Taking their differences and using $\textbf{h}^{N}=\bar{\textbf{u}}^{N}-\bar{\textbf{u}}$ as a test function, we arrive at

[TABLE]

with

[TABLE]

where

[TABLE]

For (I), we obtain from Remark 5.9

[TABLE]

As for (II), we obtain in a similar manner as in the proof of Lemma 5.8

[TABLE]

Altogether, we obtain a positive constant $c_{0}$, depending only on $T$, $K_{1}$, $K_{2}$, $j$, $g_{l}$, $l=1,\ldots,L$ and Lipschitz bound on $\bar{\xi}$ such that

[TABLE]

On the other hand, from the Poincaré inequality, we have a constant $c_{P}>0$ such that

[TABLE]

and consequently gives

[TABLE]

For $\lambda>\gamma:=2c_{P}c_{0}$, we may simply reformulate the inequality above and conclude the proof. ∎

Remark 5.10*.*

Note, that the same estimates in the proof of Theorem 5.1 may be used to provide uniqueness of minimizers to (${\bf P_{N}}$) and (${\bf P_{\infty}}$). See also Remark 5.2.

Appendix A

Proof of Lemma 2.8.

Under the given assumptions, the solutions $\mu$ and $\mu^{\prime}$ satisfy the continuity equations

[TABLE]

with locally Lipschitz vector fields $v(\mu_{t},\textbf{u}_{t})$ and $v(\mu_{t}^{\prime},\textbf{u}_{t}^{\prime})$ for every $t\in[0,T]$ satisfying

[TABLE]

In this case, we can take the temporal derivative of $W_{2}^{2}(\mu_{t}^{N},\mu_{t})$ to obtain

[TABLE]

where $\pi_{t}$ is the optimal transference plan of $\mu_{t}$ and $\mu_{t}^{\prime}$ for each $t\in[0,T]$.

For the first term, we easily deduce from (A1) the following estimate

[TABLE]

As for the other term, we have, due to (A2),

[TABLE]

where the Young inequality was used in the last inequality. Altogether, we obtain

[TABLE]

with time-independent constants $a,b>0$. Applying the Gronwall inequality on the quantity $e^{-at}W_{2}^{2}(\mu_{t},\mu_{t}^{\prime})$, we finally obtain the required estimate. ∎

Acknowledgements

CT acknowleges the discussions with J.A. Carrillo during her stay at Imperial College London, partially supported by the German Academic Exchange Service, where she studied the Wasserstein distance and related topics that influenced this manuscript. OT acknowledges support from NWO Vidi grant 016.Vidi.189.102, “Dynamical-Variational Transport Costs and Application to Variational Evolutions”.