A substitute for the classical Neumann--Morgenstern characteristic function in cooperative differential games

TL;DR

This paper introduces a new, computationally efficient characteristic function for cooperative differential games, offering an alternative to the classical Neumann-Morgenstern function with practical and interpretative advantages.

Contribution

It defines and analyzes an $ ext{eta}$-characteristic function that simplifies computation and retains economic interpretability, expanding the toolkit for cooperative differential game analysis.

Findings

The $ ext{eta}$-characteristic function is easier to compute than classical functions.

The new function maintains symmetry properties in the pollution control game.

All characteristic functions yield solutions with symmetry relations.

Abstract

In this paper, we present a systematic overview of different endogenous optimization-based characteristic functions and discuss their properties. Furthermore, we define and analyze in detail a new, $\eta$-characteristic function. This characteristic function has a substantial advantage over other characteristic functions in that it can be obtained with a minimal computational effort and has a reasonable economic interpretation. In particular, the new characteristic function can be seen as a reduced version of the classical Neumann-Morgenstern characteristic function, where the players both from the coalition and from the complementary coalition use their previously computed strategies instead of solving respective optimization problems. Our finding are illustrated by a pollution control game with $n$ non-identical players. For the considered game, we compute all characteristic functions and compare their properties. Quite surprisingly, it turns out that both the characteristic functions and the resulting cooperative solutions satisfy some symmetry relations.

Tables

Table 1. Table 1: Computational effort required for computing different characteristic functions.

C.F.	# of optimization problems (# of variables)	# of Nash equilibrium problems
$\frac{1}{1}$ $\alpha$ ($\beta$)	$2^n-1\left(n\right)$	0
$\frac{1}{1}$$\delta$	$2^n-n-1\left(2÷n\right)$	1
$\frac{1}{1}$$\zeta$	$2^n-1\left(1÷n\right)$	0
$\frac{1}{1}$$\eta$	1 (n)	1

Table 2. Table 2: Possible strategic interactions between the coalition S 𝑆 S and its complement N ∖ S 𝑁 𝑆 N\setminus S and the respective characteristic functions.

		$S$
		$\underset{\frac{u_i\in {𝒰}_i}{i\in S}}{\max }\sum _{i\in S}{J}_i$	$u_i=u_i^{NE}$, $i\in S$	$u_i=u_i^{\ast }$, $i\in S$
	$\underset{\frac{u_j\in {𝒰}_j}{j\in N\setminus S}}{\min }\sum _{i\in S}{J}_i$	$\alpha /\beta$	$F_1$	$\zeta$
$N\setminus S$	$u_j=u_j^{NE}$, $j\in N\setminus S$	$\delta$	Nash equilibrium	$\eta$
	$u_j=u_j^{\ast }$, $j\in N\setminus S$	$F_2$	$F_3$	Cooperative agreement

Equations

$$\dot{x}=f(x,u_1,\dots ,u_n),x(t_0)=x_0,$$

$$J_i(t_0,x_0,u)=\underset{t_0}{\overset{T}{\int }}{h}_i(\tau ,x(\tau ),u(\tau ))d\tau ,$$

$$J_i(t_0,x_0,u^{NE})\ge J_i(t_0,x_0,{\bar{u}}_i,u_{-i}^{NE})\forall u_i\in {\mathcal{U}}_i$$

$$\sum _{i=1}^n{J}_i(x_0,u^{\ast })\ge \sum _{i=1}^n{J}_i(x_0,u)\forall u\in \mathcal{U}.$$

$$V(S)+V(Q)\le V(S\cup Q).$$

$${\Xi }_k=\left(\begin{array}{ccccc}V\left(\{1\}\right),& \dots & V(N)-\left[\sum _{\genfrac{}{}{0ex}{}{i=1}{i\ne k}}^{n}V\left(\{i\}\right)\right],& \dots & V\left(\{n\}\right)\end{array}\right),k=1,\dots ,n.$$

$$Sh_{i}=\sum\limits_{S\subset N,\atop{i\in S}}\frac{(n-s)!(s-1)!}{n!}\Big{[}V(S)-V(S\setminus\{i\})\Big{]}.$$

$$V^{\alpha }(S)=\{\begin{array}{ll}0,& S=\varnothing ,\\ {\mathrm{val}}^{-}{\Gamma }_{S,N\setminus S},& S\subseteq N,\end{array}$$

$${\mathrm{val}}^{-}{\Gamma }_{S,N\setminus S}=\underset{\genfrac{}{}{0ex}{}{u_i\in {\mathcal{U}}_i,}{i\in S}}{\sup }\underset{\genfrac{}{}{0ex}{}{u_j\in {\mathcal{U}}_j,}{j\in N\setminus S}}{\inf }\sum _{k\in S}{J}_k(t_0,x_0,u_i,u_j(u_i)).$$

$${\mathrm{val}}^{-}{\Gamma }_{S,N\setminus S}=\underset{\genfrac{}{}{0ex}{}{u_i\in {\mathcal{U}}_i,}{i\in S}}{\max }\underset{\genfrac{}{}{0ex}{}{u_j\in {\mathcal{U}}_j,}{j\in N\setminus S}}{\min }\sum _{k\in S}{J}_k(t_0,x_0,u_i,u_j(u_i)).$$

$$V^{\alpha }(S\cup Q)\ge V^{\alpha }(S)+V^{\alpha }(Q).$$

$$V^{\beta }(S)=\{\begin{array}{ll}0,& S=\varnothing ,\\ \underset{\genfrac{}{}{0ex}{}{u_j\in {\mathcal{U}}_j,}{j\in N\setminus S}}{\min }\underset{\genfrac{}{}{0ex}{}{u_i\in {\mathcal{U}}_i,}{i\in S}}{\max }\sum _{k\in S}{J}_k(t_0,x_0,u_i(u_j),u_j),& S\subseteq N.\end{array}$$

$$V^{\delta }(S)=\{\begin{array}{ll}0,& S=\varnothing ,\\ \underset{\genfrac{}{}{0ex}{}{u_i\in {\mathcal{U}}_i,}{i\in S}}{\max }\sum _{k\in S}{J}_k(t_0,x_0,u_S,u_{N\setminus S}^{NE}),& S\subseteq N,\end{array}$$

$$V^{\zeta }(S)=\{\begin{array}{ll}0,& S=\varnothing ,\\ \underset{\genfrac{}{}{0ex}{}{u_j\in {\mathcal{U}}_j,}{j\in N\setminus S}}{\min }\sum _{i\in S}{J}_i(t_0,x_0,u_S^{\ast },u_{N\setminus S}),& S\subseteq N.\end{array}$$

$$V^{\zeta }(S\cup Q)\ge V^{\zeta }(S)+V^{\zeta }(Q).$$

$$V^{\eta }(S)=\{\begin{array}{ll}0,& S=\varnothing ,\\ \sum _{i\in S}{J}_i(t_0,x_0,u_S^{\ast },u_{N\setminus S}^{NE}),& S\subseteq N.\end{array}$$

$$\bar{V}^{\eta}(S)=\max\left\{\sum_{k}V^{\eta}(Q_{k})\big{|}\{Q_{k}\}_{k=1,\dots,l}\mbox{ s.t. }\bigcup_{k}Q_{k}=S,Q_{i}\bigcap Q_{j}=\emptyset,1\leq i\neq j\leq l\right\},$$

$$\begin{array}{r}{\overline{V}}^x(S)=V^{\delta }(S)-V^x(S),\\ {\underline{V}}^x(S)=V^x(S)-V^{\zeta }(S).\end{array}$$

$$V^x(S)=k_x(S)V^{\delta }(S)+(1-k_x(S))V^{\zeta }(S)\forall S\subseteq N.$$

$$\dot{x}(t)=\sum _{i=1}^n{u}_i(t),x(t_0)=x_0\ge 0,$$

$$J_i(x_0,u)={\int }_{t_0}^T\left(b_i{u}_i-\frac{1}{2}{u}_i^2-d_{i}x\right)dt.$$

$$b_i\ge D_N(T-t_0),$$

$$u^{NE}(t)=\left[\begin{array}{cccc}{b}_1-d_1(T-t),& b_2-d_2(T-t)& ...& b_n-d_n(T-t)\end{array}\right],$$

$$u^{\ast }(t)=\left[\begin{array}{cccc}{b}_1-D_N(T-t),& b_2-D_N(T-t)& ...& b_n-D_N(T-t)\end{array}\right].$$

$$x^{NE}(t)=x_0+(B_N-D_{N}T)(t-t_0)+\frac{D_N}{2}(t^2-t_0^2)$$

$$x^{\ast }(t)=x_0+(B_N-n{D}_{N}T)(t-t_0)+\frac{n{D}_N}{2}(t^2-t_0^2),$$

$$x^{NE}(t)-x^{\ast }(t)=\frac{1}{2}(n-1)D_N(t-t_0)(2T-t-t_0))\ge 0,t\in [t_0,T].$$

$$V^{\alpha }(S,x(t),t)=-D_S(T-t)x(t)+\frac{1}{2}{\tilde{B}}_S(T-t)-\frac{1}{2}{B}_N{D}_S(T-t{)}^2+\frac{1}{6}s{D}_S^2(T-t{)}^3,$$

$$V^{\delta }(S,x(t),t)=-D_S(T-t)x(t)+\frac{1}{2}{\tilde{B}}_S(T-t)-\frac{1}{2}{B}_N{D}_S(T-t{)}^2+\frac{1}{6}(2D_{N\setminus S}{D}_S+s{D}_S^2)(T-t{)}^3,$$

$$V^{\zeta }(S,x(t),t)=-D_S(T-t)x(t)+\frac{1}{2}{\tilde{B}}_S(T-t)-\frac{1}{2}{B}_N{D}_S(T-t{)}^2-\frac{1}{6}s{D}_N(D_N-2D_S)(T-t{)}^3,$$

Full text

1 Introduction

It would not be an exaggeration to say that the notion of a characteristic function plays a central role in cooperative game theory. The characteristic function describes the worth of a coalition and hence is used in coalition formation, [9, 15]: two groups of players (i.e., smaller coalitions) are likely to join to form a larger coalition if the characteristic function of the larger coalition is greater than the sum of the characteristic functions of the original coalitions. On the other hand, an importance of an individual player within a coalition can be measured by its marginal contribution to the characteristic function. This is the idea upon which the computation of the Shapley value [29, 31], the Banzhaf power index and other extensions of the Shapley value are based, [27].

Since the seminal book by von Neumann and Morgenstern, [30], there has been active and continuing research on the role and different forms of characteristic functions. In general, there are two main approaches to defining a characteristic function: an endogenous and an exogenous one. The former approach derives the value of the characteristic function from the properties and the structure of the game alone, while the latter makes use of some external information (see, e.g., [6], where the external data of trade flows was used). However, it is in general rather difficult to formalize exogenous effects. Therefore we stay with the endogenous formulation, which in turn can be subdivided into two classes: equilibrium- and optimization-based characteristic functions. While the former ones are obtained as equilibrium solutions in a non-cooperative game played between the coalition and the rest of the players (which can form a coalition as well or can play individually), the latter ones are obtained under the assumption that both the coalition and the remaining players a priori make certain decisions about their strategies and follow them. As an example of an equilibrium-based characteristic function we mention the $\gamma$-characteristic function introduced in [5].

Admittedly, equilibrium-based characteristic functions are more realistic from economic viewpoint and can be justified from the viewpoint of players’ rational behavior. However, they have a serious drawback that solving a Nash equilibrium problem is in general much more complicated compared to solving an optimization problem. Another problem is that even for relatively innocent problem formulation there is a chance that the Nash equilibrium will be non-unique, which presents an additional difficulty – especially in the context of equilibrium-based characteristic functions – as there can appear many incomparable Nash equilibria (see a discussion in [23]). We thus confine ourselves to studying only optimization-based characteristic functions.

The contribution of this paper is two-fold: first, we wish to present a systematic overview of different endogenous optimization-based characteristic functions (i.e., $\alpha$, $\beta$, $\delta$, and $\zeta$ - c.f.) and discuss their properties. Second, we present and analyze in detail a new characteristic function, termed the $\eta$-characteristic function. This characteristic function has a substantial advantage over other characteristic functions in that it can be obtained with a minimal computational effort and has a reasonable economic interpretation. In particular, we show that the new characteristic function can be seen as a reduced version of the classical Neumann-Morgenstern characteristic function, where the players both from the coalition and from the complementary coalition use their previously computed strategies instead of solving respective optimization problems.

As an illustration we consider a game of pollution control with $n$ non-identical players. For the considered game, we compute all characteristic functions and compare their properties. Quite surprisingly, it turns out that both the characteristic functions and the resulting cooperative solutions satisfy some symmetry relations. While this result does not need to hold in general, it shows that there is a certain interconnection between different types of charateristic functions that is yet to be studied.

This paper extends and generalizes a number of results scattered through the series of works by the first author, [14, 12, 13]. See also [25, 28] for related results.

The paper is organized as follows. In Sec. 2, we introduce a number of basic facts and definitions from game theory. Section 3 contains an extensive analysis and comparison of a number of characteristic functions, while in Section 4, the described results are illustrated with a differential game of $n$ players. In particular, it is shown that for the considered example the $\zeta$ characteristic function possesses a number of additional properties of particular interest. Finally, a number of conclusions and directions for the future work are presented in the Conclusion section. All technical results as well as the proofs of the theorems are collected in two Appendices.

2 Basic definitions

2.1 Differential games in normal form

We consider an $n$-person differential game $\Gamma(x_{0},t_{0},T)$ that starts from the initial state $x_{0}$ at time $t_{0}$ and evolves over the interval $[t_{0},T]$. The dynamics of the game are described by the system

[TABLE]

where $x(t)\in X$, $u_{i}(t)\in U_{i}$, $\forall t\in[t_{0},T]$; $X$ and $U_{i}$ are compact subsets of $\mathbb{R}^{n}$ and $\mathbb{R}^{n_{i}}$, respectively. We denote by $\mathcal{U}_{i}$ the set of admissible controls (actions) of the $i$th player, it can be the set of all measurable or piecewise continuous functions on the interval $[t_{0},T]$ with values in $U_{i}$. In the following, we will employ the short notations $u(t)$ and $\mathcal{U}$ for $(u_{1}(\cdot),\ldots,u_{n}(\cdot))$ and $(\mathcal{U}_{1},\ldots,\mathcal{U}_{n})$, respectively. We assume that for any $n$-tuple of admissible controls $u(t)\in\mathcal{U}$, the solution to (1) exists, unique and is defined over the whole interval $[t_{0},T]$.

Let the set of the players be denoted by $N=\{1,\ldots,n\}$. We define the payoff of the $i$th player as

[TABLE]

where $h_{i}(\tau,x,u)$ is a smooth function and $x(t)$ is the solution of (1) for controls $u(t)$. In the rest of the paper, we will write $J_{i}(x_{0},u)$ keeping in mind that the game is played on the interval $[t_{0},T]$.

2.2 Solution concepts

We will consider two most typical definitions of the solution of a differential game: a Nash equilibrium and a cooperative agreement.

Definition 2.1**.**

The $n$-tuple $u^{NE}=\{u_{1}^{NE},\dots,u^{NE}_{n}\}$ is a Nash equilibrium if for every $i\in N$

[TABLE]

where $u_{-i}^{NE}=\{u_{1}^{NE},\dots,u^{NE}_{i-1},u^{NE}_{i+1},\dots,u^{NE}_{n}\}$.

Definition 2.2**.**

A cooperative agreement is a tuple $u^{*}(t)=(u^{*}_{1}(t),\ldots,u^{*}_{n}(t))$ s.t.

[TABLE]

We assume that both the Nash equilibrium and the cooperative agreement exist and unique. While this may not be true in general, this holds for a sufficiently large class of practically relevant cases, see, e.g., [1, 7]. It should be noted that computing a Nash equilibrium is a substantially more complicated task compared to that of computing the cooperative agreement: while the latter boils down to solving a single optimization problem, the former requires solving $n$ coupled optimization problems.

The Nash equilibrium and the cooperative agreement represent two extreme cases in the range of all possible strategic interactions between the players: the former correspond to the case when all players act individually while the latter implies that all the players have agreed to cooperate in order to maximize their joint payoff, i.e., they form a grand coalition. In many applications, there is a need to compare different strategic decisions. This issue is addressed using the notion of the characteristic function as described below.

3 Classes of characteristic functions in differential games

3.1 Definition of a characteristic function

We define the coalition $S$ to be a subset of the set of all players $N$, i.e., $S\subseteq N$ or, using set-theoretic notation, $S\in 2^{N}$. Furthermore, we will call $N\setminus S$ the coalition complementary to $S$.

Definition 3.1**.**

Given a differential game $\Gamma(x_{0},t_{0})$, a characteristic function is defined as the map $V:\mathbb{R}\times X\times 2^{N}\rightarrow\mathbb{R}_{\geq 0}$ that satisfies two conditions:

$V(x_{0},t_{0},\emptyset)=0$; 2. 2.

$V(x_{0},t_{0},N)=\sum_{i=1}^{n}J_{i}(t_{0},x_{0},u^{*})$.

We stress the fact that a characteristic function depends on the initial conditions and hence can be extended to subgames. However, for the sake of brevity we will drop the first two arguments and write a characteristic function as $V(S)$. Later, when considering a real example in Sec. 4 we will recover the original notation.

There is a third condition that until recently was considered as an essential component of the definition of the characteristic function. Nowadays it is considered merely as a property of a characteristic function.

Definition 3.2**.**

A characteristic function $V$ is said to be super-additive if for any two disjoint coalitions $S,Q\subseteq N$, $S\cap Q=\emptyset$, it holds that

[TABLE]

The value $V(S)$ is typically interpreted as the worth or the power of the coalition $S$. Thus super-additivity implies that the power of the union of any two disjoint coalitions is not less than the sum of the powers of individual coalitions. Suppose that two coalitions merge if the characteristic function of the joint coalition exceeds the sum of the characteristic functions of the individual coalitions. In this sense, the super-additivity property of a characteristic function formalizes an incentive for the players to form ever bigger coalitions and serves as a foundation for the study of coalition formation, see [9, 15].

Another major field of application of characteristic function consists in determining a rule for sharing the total payoff obtained under the cooperative agreement222Obviously, the main prerequisite for using imputations is the condition that the players agree to redistribute their payoffs. The games for which this condition is fulfilled are called the games with transferable utility (TU games).. This rule is called an imputation.

Definition 3.3**.**

A vector $\xi=(\xi_{1},\ldots,\xi_{n})$ is an imputation if it satisfies the following conditions:

Individual rationality condition: $\xi_{i}\geq V(\{i\})$ for all $i=1,\ldots,n$; 2. 2.

Group rationality condition: $\sum_{i=1}^{n}\xi_{i}=V(N)$.

Obviously, the set of imputations is never empty as $V(N)\geq\sum_{i=1}^{n}V(\{i\})$. In general, it can be defined as an $(n+1)$ simplex formed by $n$ vectors

[TABLE]

However, if the Nash equilibrium turns out to be Pareto optimal, as in the case of Strong Nash Equilibrium, the set of imputations reduces to a trivial solution $\Xi^{o}=\begin{pmatrix}V(\{1\})&\ldots&V(\{n\})\end{pmatrix}$. If, furthermore, the characteristic function is merely additive, but not superadditive, we have an inessential game, i.e., the game in which neither player can profit from forming a coalition.

The main subject of cooperative game theory consists in determining a subset of imputations satisfying certain properties and devising a rule to choose between them. These subsets are called the cooperative solutions (see, e.g., [22, Ch. 13-14] for a description of different concepts of cooperative solution). Remarkably, most cooperative solutions except for the proportional one [21] are based upon the use of a characteristic function.

One particularly important type of cooperative solution is the classical Shapley value, [16]. This is a single valued cooperative solution that is defined as

[TABLE]

In the following, we will define characteristic functions as an outcome of the strategic interaction between players. Namely, the players for the coalition $S$ and its complementary coalition $N\setminus S$ choose to use certain strategies. The characteristic function is thus defined as the value of the total payoff of the players from $S$. Loosely speaking, a characteristic function can be seen as a score table in a tournament in which each coalition plays again its complement following a predetermined set of rules. Obviously, these competitions never occur in real life and do not aim at reflecting any realistic scenario of the interactions between the players. Therefore, the exact choice of the rules adopted in a particular tournament is in principle not substantial. However, it turns out that for certain sets of rules the characteristic function is easier to compute or possesses some advantageous properties. Below, we consider different approaches to compute a characteristic function and make a comparison.

3.2 Types of characteristic functions

3.2.1 $\alpha$- and $\beta$-characteristic functions (Neumann-Morgenstern)

The first and by far the most popular method for characteristic function construction was introduced in the seminal work [30], and now is referred to as the $\alpha$-characteristic function. The characteristic function is defined as the lower value of the zero-sum game $\Gamma_{S,N\setminus S}$ between the coalition $S$ acting as the first, maximizing player and the complementary coalition $N\setminus S$ acting as the second, minimizing player:

[TABLE]

where the lower value of the game is defined by

[TABLE]

Here we assume that the players from the complementary coalition know the strategies adopted by the players from $S$. Put differently, the players from the complementary coalition use their best response strategies that are defined on the union $\bigcup_{i\in S}\mathcal{U}_{i}$. In the rest of the paper we will assume that the lower and upper bounds in the respective optimization problems belong to the admissible control sets and hence we can write ${\mathop{\rm val}}^{-}\Gamma_{S,N\setminus S}$ in terms of minima and maxima:

[TABLE]

The characteristic function $V^{\alpha}(S)$ is interpreted as the maximum guaranteed payoff that the coalition $S$ can secure in the worst case. Under sufficiently mild conditions, the lower value of the game is uniquely defined as the viscosity solution of the associated Hamilton-Jacobi-Bellaman-Isaacs equation (see [8] for details; an extensive and systematic treatment of the problem is given in [3]). The described max-min optimization problems can be solved in the class of open-loop controls as well [2, Sec. 6.5]. Note that computation of the $\alpha$-characteristic function requires solving $2^{n}-1$ optimization problems (5), which is a rather demanding task. An example of constructing the $\alpha$-characteristic function can be found in [20].

On the other hand, the $\alpha$-characteristic function has certain important advantages. In particular, we have the following result.

Theorem 3.1** ([24]).**

The $\alpha$-characteristic function for the game $\Gamma(x_{0},t_{0},T)$ is superadditive, i.e., for any $S,Q\in 2^{N}$, $S\cap Q=\emptyset$ we have

[TABLE]

Since the proof of this result for differential games is not readily available, we present it in Appendix 1.

An alternative option is the $\beta$-characteristic function which is defined as the upper value of the game:

[TABLE]

This characteristic function can be seen as the smallest payoff that a complementary coalition can securely force the coalition $S$ to receive. Obviously, $V^{\beta}(S)\geq V^{\alpha}(S)$ for all $S\subseteq N$.

It has been argued, e.g., in [5, 23], that the use of the $\alpha$- (and also $\beta$-) characteristic function is not justified from the economic point of view. Namely, it is quite unlikely that the members of the complementary coalition would act with the sole objective of minimizing the payoff of the coalition $S$. Despite the fact that – as was discussed at the end of Sec. 3.1 – a characteristic function does not reflect any real strategic interaction between players, this argument has a point. When using a characteristic function as a measure of the coalition’s power it is desirable to measure that power according to some realistic scenario.

3.2.2 $\delta$-characteristic function

An alternative approach to the construction of a characteristic function, termed the $\delta$-characteristic function was proposed in [23]. The calculation of a $\delta$-characteristic function consists of two steps. First, the Nash equilibrium strategies $u^{NE}$ are computed. Next, the characteristic function is computed by letting the players from $S$ maximize their total payoff $\sum_{i\in S}J_{i}(t_{0},x_{0},u)$ while the players from $N\setminus S$ use the previously computed Nash equilibrium strategies:

[TABLE]

where $u_{S}=\{u_{i}\}_{i\in S}$ and $u_{N\setminus S}^{NE}=\{u^{NE}_{j}\}_{j\in N\setminus S}$.

Computation of the $\delta$-characteristic function requires computing the Nash equilibrium for the game $\Gamma(x_{0},t_{0},T)$ and solving $2^{n}-n-1$ optimization problems. Note that by definition $V^{\delta}(\{i\})=u_{i}^{NE}$ for any $i\in N$. Determining the value of the $\delta$-characteristic function for any other coalition consists in solving an optimization problem with the controls of the players from the complementary coalition being fixed to their Nash equilibrium values. Such optimization problems may or may not present a challenge depending on whether the Nash equilibrium solution can be found analytically or not.

In contrast to the $\alpha$-characteristic function, the $\delta$-characteristic function has a neat economical interpretation. When confronted with an emerging coalition $S$, the players from the complement $N\setminus S$, i.e., those not involved in forming the coalition remain neutral and stay committed to their Nash equilibrium strategies. Perhaps the most realistic scenario would be to consider the situation when the value of the characteristic function was computed as the Nash equilibrium value in a game played between the coalition acting as a single player and the remaining players acting independently (cf. the $\gamma$-characteristic function described in [5], see also [18] for a similar approach applied to partition games). However, this would require solving $2^{n}-n$ eqiulibrium problems which is a substantially more difficult task compared to those described above.

In [26] it was proved that the $\delta$-characterisctic function is superadditive for a certain class of static games if some conditions on the payoff function are fulfilled. However, in a general case the $\delta$-characteristic function can turn out to be not superadditive as the recent result shows [12].

3.2.3 $\zeta$-characteristic function

The $\zeta$-characteristic function was initially proposed in [25] and later developed in [14]. Its construction is carried out in two steps. First, the cooperative agreement $u^{*}$ in the game $\Gamma(x_{0},t_{0},T)$ is computed. Next, when computing $V^{\zeta}(S)$, the players from the coalition $S$ use their optimal cooperative strategies $u^{*}_{S}$ while the players from $N\setminus S$ use the strategies minimizing the total payoff of the players from $S$:

[TABLE]

This characteristic function can be seen as a reduced version of the $\alpha$-characteristic function in which the players from the coalition $S$ do not attempt to maximize their joint payoff, but rather stick to their previously computed cooperative solution while the left-out players play against them. In some sense, this situation can be interpreted as a breakup of the grand coalition in which the split-off players play against the rest.

Similarly to the $\alpha$-characteristic function, computation of the $\zeta$-characteristic function requires solving $2^{n}-1$ optimization problems, albeit with less decision variables. However, whether this results in a simplification of the problem depends on whether the sooperative agreement can be obtained analytically or not. Similarly to the $\alpha$-characteristic function, $V^{\zeta}(S)$ is superadditve.

Theorem 3.2** ([14]).**

The $\zeta$-characteristic function for the game $\Gamma(x_{0},t_{0},T)$ is superadditive, i.e., for any $S,Q\in 2^{N}$, $S\cap Q=\emptyset$ we have

[TABLE]

See Appendix 1 for the proof.

Lastly, $\zeta$-characteristic function is applicable for games with fixed coalition structures [25].

3.2.4 $\eta$-characteristic function

Most recently, a new characteristic function was introduced in [13]. Following the established tradition, we term it the $\eta$-characteristic function. It is defined by

[TABLE]

This characteristic function has a number of advantages. First, it is computed using only the cooperative agreement and the Nash eqilibria. This substantially simplifies the problem, especially if the number of players is sufficiently large.

Furthermore, the new characteristic function has a pretty intuitive interpretation. In contrast to the $\delta$-characteristic functions (and similar to the $\zeta$-characteristic function) it describes the process of the grand coalition’s breakup. However, as the players split off the coalition, their complement do not act agains them, but rather choose to use Nash equilibrium strategies, which seems to be more realistic.

The $\eta$-characteristic function is not superadditive in general. However, we can construct a superadditive extension of the $V^{\eta}$-characteristic function

[TABLE]

where the maximum is taken over all partitions of the set $S$. Note that for a superadditive function we have $\bar{V}(S)=V(S)$ for all $S\subseteq N$.

3.3 Partial order relations on the set of characteristic functions

For the introduced characteristic functions, a partial order relation can be defined. Before presenting the result we note that by definition, all characteristic functions have equal values for the empty coalition and for the grand coalition. We have the following result.

Theorem 3.3**.**

For the differential game $\Gamma(x_{0},t_{0},T)$ and for any $S\in 2^{N}$ the following inequalities hold:

$V^{\alpha}(S)\geq V^{\zeta}(S)$. 2. 2.

$V^{\delta}(S)\geq V^{\alpha}(S)$. 3. 3.

$V^{\delta}(S)\geq V^{\beta}(S)$** 4. 4.

$V^{\delta}(S)\geq V^{\eta}(S)$. 5. 5.

$V^{\eta}(S)\geq V^{\zeta}(S)$**

The proof is given in Appendix 1. The results of Theorem 3.3 can be represented by a diagram as shown in Fig. 1. An arrow corresponds to the “greater or equal” relation. Note that we added an arrow corresponding to the $V^{\beta}(S)\geq V^{\zeta}(S)$ inequality that immediately follows from the transitivity of the $\geq$ relation.

3.4 Analysis and discussion of characteristic functions

Before we proceed to a further analysis of characteristic functions it is worth reiterating on the computational advantages provided by the $\eta$-characteristic function. In Table 1, we present a summary of the computational effort required for constructing different characteristic functions. We show the number of optimization problems to be solved and the number of variables involved. It can be seen that the $\eta$-characteristic function has a tremendous advantage in terms of computation power required. This is particularly important when considering realistic games in which the number of players is expressed by several-digit numbers.

The described characteristic functions do not exhaust all possible variants of the strategic interaction between players. In Table 2 we describe all meaningful characteristic functions that involve at most one computation of the equilibrium. That is to say, we exclude the cases where the characteristic functions are obtained as the Nash equilibria in which either the coalition or the complementary coalition play as a single player (for instance, the $\gamma$-characteristic function [5]). Furthermore, by meaningful we mean that, e.g., the members of the coalition $S$ do not attempt to minimize their total payoff and so on.

There are three more characteristic functions that can be split in two groups (with a single overlap): when the members of $S$ use their Nash equilibrium strategies ($F_{1}$ and $F_{3}$) and when the members of the complementary coalition use their cooperative strategies ($F_{2}$ and $F_{3}$). Both cases are pretty unrealistic because they correspond to the scenarios where the players make an inappropriate use of their strategies. That is to say, a coalition $S$ is supposed to either maximize its total payoff or stick to previously computed cooperative agreement, while the complementary coalition either plays against $S$ by minimizing its payoff or remains neutral by sticking to the previously computed Nash equilibrium solution. In contrast to that, the characteristic functions $F_{1}$, $F_{2}$, $F_{3}$ correspond to the cases where at least one coalition takes an irrational decision. Thus we dismiss the respective cases as non-relevant.

We observe that the $\delta$ and the $\zeta$ functions form the lower and the upper bound for the considered characteristic functions. When considering any other characteristic function one might ask how much does it deviates for the respective bounds. Thus for each characteristic function $V^{x}$, $x\in\{\alpha,\beta,\delta,\zeta,\eta\}$ we introduce the functions $\overline{V}^{x}(S)$ and $\underline{V}^{x}$ that are defined as follows:

[TABLE]

Functions $\overline{V}^{x}(S)$ and $\underline{V}^{x}$ describe the distance of the characteristic function to the upper, respectively lower bound. Both functions are non-negative and it holds that $\overline{V}^{\delta}(S)=\underline{V}^{\zeta}=0$.

Furthermore, we say that a characteristic function $V^{x}(S)$ is aligned with the bounds if there exists a function $k(S):S\mapsto[0,1]$, called the alignement coefficient, such that

[TABLE]

In the following paragraph compute the whole range of characteristic functions for a particular $n$-players differential game and analyze their properties.

4 Illustration. Differential game of pollution control for $n$ players

4.1 The model of the game

We consider a differential game of pollution control with prescribed duration based upon the model described in [17, 4], see also [11]. In contrast to the mentioned results, we consider a general setup involving $n$ non-identical players (countries, companies).

Let $N$, $|N|=n$, denote the set of all players and $x(t)$ be the total pollution at time $t$. Its dynamics are governed by the differential equation

[TABLE]

where $u_{i}(t)$ is the pollution flow emitted by the $i$th player. We assume that each player sets an upper bound on the pollution flow, i.e., $u_{i}(t)\in[0,b_{i}]$. The $i$th player’s revenue is approximated by a strictly concave function of the pollution flow: $R_{i}(u_{i}(t))=b_{i}u_{i}-\frac{1}{2}u^{2}_{i}$. Note that $R_{i}(u_{i})$ attains its maximum at the upper bound $u_{i}=b_{i}$. Furthermore, there is a cost associated with pollution that the player has to pay (taxes, environmental costs etc). This cost is assumed to be a linear function of the pollution stock, albeit with the coefficient that is specific for each player: $Q_{i}(x)=-d_{i}x$, $d_{i}>0$.

Thus, the $i$th player’s payoff is given by

[TABLE]

Assume that for all $i=1,\ldots,n$, the following regularity constraints are satisfied:

[TABLE]

where $D_{N}=\sum\limits_{i=1}^{n}d_{i}$.

4.2 Solutions of the game

We start by computing the Nash equilibrium solution and the cooperative agreement. We will stick to the open-loop solutions and will thus employ the Pontryagin maximum principle [19]. However, for the problem under study the solution can be obtained within the class of feedback strategies, see, e.g., [10, Sec. 6]. The Nash equilibrium is found to be

[TABLE]

while the cooperative agreement is given by

[TABLE]

The corresponding trajectories are

[TABLE]

and

[TABLE]

where $B_{N}=\sum_{i=1}^{n}b_{i}$. The derivation of (16)-(19) is quite straightforward and therefore omitted.

We note that

[TABLE]

This has the straightforward interpretation that the level of pollution is smaller when the players agree to cooperate.

4.3 Analytic expressions for the characteristic functions

In this subsection we present analytic expressions for the previously considered characteristic functions in the pollution control game. Note that we write the characteristic functions in a slightly more general form, as functions of the coalition $S$, the initial state $x(t)$ and the initial time $t\leq T$. This form can be useful when we consider the characteristic functions of the subgames. At the same time, we can easily recover the total characteristic function of the game by setting $t$ and $x(t)$ to $t_{0}$ and $x(t_{0})$, respectively.

Thus, for a coalition $S\in 2^{N}$ we have the following expressions for the characteristic functions (see Appendix 2 for the details on the derivation of the characteristic functions):

[TABLE]

[TABLE]

[TABLE]

and, finally,

[TABLE]

Here we used the following notation: $B_{Q}=\sum_{i\in Q}b_{i}$, $\tilde{B}_{Q}=\sum_{i\in Q}b_{i}^{2}$, and $D_{Q}=\sum_{i\in Q}d_{i}$, where $Q\subseteq N$. It turns out that in this particular case, the computed characteristic function is superadditive.

Proposition 4.1**.**

The characteristic function (23) is superadditive, i.e.,

[TABLE]

Proof 1**.**

Let $S_{1}$ and $S_{2}$ be two disjoint subsets of $N$. We have

[TABLE]

whence the result follows.∎

4.4 Analysis and comparison of the obtained characteristic functions

Using the results presented in the previous subsection we can compute further characteristics of the $\alpha$ and $\eta$ characteristic functions (12), computed for the game (13), (14).

Proposition 4.2**.**

For the game (13), (14), the distance of the characteristic functions $V^{\alpha}$ and $V^{\eta}$ to the respective upper and lower bounds are given by

[TABLE]

Proof 2**.**

The result is obtained by straightforward computation using (12) and the expressions for characteristic functions (20)-(23).∎

We can observe that the considered characteristic functions are located reflection symmetric with respect to the bounds:

[TABLE]

Moreover, it turns out that both $V^{\alpha}$ and $V^{\eta}$ are aligned with respect to the bounds. It suffices to show that only for $V^{\eta}$, the result for $V^{\alpha}$ follows from the reflection symmetry of the characteristic functions. The following lemma formally states this result.

Proposition 4.3**.**

For the characteristic function $V^{\eta}$ there exists a function $k_{\eta}(S)$ such that

[TABLE]

and this alignment coefficient is given by $k_{\eta}=\frac{2D_{S}}{2D_{S}+sD_{N\setminus S}}$.

Proof 3**.**

The result is obtained by straightforward computation using (21), (22), and (23).∎

Note that due to the reflective symmetry of $V^{\alpha}$ and $V^{\eta}$, the respective coefficient $k_{\alpha}$ is obtained as

[TABLE]

A remarkable property of the computed alignment coefficient $k_{\eta}$ is that it does not depend on the initial time and the initial condition, but only on the parameters of the model.

Finally, we present an important characterization of cooperative solution (Shapley value) for the considered characteristic functions.

Proposition 4.4**.**

For the components of the Shapley values $Sh^{x}$ computed on the basis of the respective characteristic functions $V^{x}$, $x\in\{\alpha,\beta,\zeta,\eta\}$ it holds that

[TABLE]

[TABLE]

For the proof of this result see Appendix 2.

Conclusion

In the paper four endogenous optimization-based characteristic functions were described and analyzed in detail. In particular, it was shown that a new, $\eta$-characteristic function appears to be a promising candidate for the use in various game-theoretic applications. All results were illustrated with an $n$ player differential game of pollution control. It was shown that for this specific game the $\eta$-characteristic function has a number of additional important properties. The future work will be concentrated on investigating further properties of the introduced characteristic function and its application to a wide class of dynamic games.

Appendix 1. Proofs of the theorems

Proof 4** (Theorem 3.1).**

We prove the result by performing a series of transformations as shown below. At each step, the right-hand side does not increase.

[TABLE]

Here, if $u_{i}$, $u^{\prime}_{i}$, and $u^{\prime\prime}_{i}$ are not maximized upon, they are assumed to take any value from the respective set $\mathcal{U}_{i}$. We observe that the minimum of a sum of two functions is not less than the sum of minima of respective functions, whence inequality (24c) follows. Note that in the latter case, the minimization is performed separately. The transition from (24d) to (24e) is justified by the fact that (24d) is valid for all controls $u_{i}\in\mathcal{U}_{i}$, $i\in S$ in the first summand, and for all $u_{i}\in\mathcal{U}_{i}$, $i\in Q$ in the second summand.∎

Proof 5** (Theorem 3.2).**

In proving (10) we follow the same lines as in the proof of Thm. 3.1. We have

[TABLE]

where the inequality (25c) relies on the fact that minimization over a larger set, say, $\bigcup_{i\in N\setminus S}\mathcal{U}_{i}$ yields a result that does not exceed that one obtained when minimizing over $\bigcup_{i\in N\setminus(S\cup Q)}\mathcal{U}_{i}$.∎

Proof 6** (Theorem 3.3).**

Obviously, the maximum of a function over a set is not less than the value obtained by picking up any element from the set and substituting it into that function. Thus, it follows that for any $S\subseteq N$

[TABLE] 2. 2.

If – in the definition of the $\alpha$-characteristic function – instead of using their best response (minimizing) strategies, the players from the complementary coalition stick to some arbitrary strategies, the outcome will not diminish:

[TABLE] 3. 3.

Taken that the players from the coalition $S$ use their best response strategies, the minimum over a set cannot exceed the value of the payoff computed for a fixed set of strategies $u^{NE}_{N\setminus S}$:

[TABLE] 4. 4.

This inequality is similar to the one in item 1.

[TABLE] 5. 5.

The minimum of a function over a set is not larger than the value obtained by picking up any element from the set and substituting it into that function. Thus, we have

[TABLE]

This concludes the proof.∎

Appendix 2. Computation of the characteristic functions and related characteristics

To compute the $\alpha$-characteristic function we employ a two-step procedure. First, we solve the minimization problem

[TABLE]

with respect to the players from the complementary coalition $N\setminus S$. The respective optimal controls are found to be $\bar{u}_{j}=b_{j}$, $j\in N\setminus S$. This has a straightforward interpretation. First observe that the players are coupled only through the state variable $x$. As the state growth, the costs incurred grow as well. Thus the best strategy of the complementary coalition, acting as an adversary w.r.t. the coalition $S$, is to emit as much pollution as possible.

Next we consider the maximization problem

[TABLE]

where the controls $u_{j}$, $j\in N\setminus S$ are set to their optimal values $\bar{u}_{j}=b_{j}$. The respective optimal controls are computed to be $\bar{u}_{i}(t)=b_{i}-D_{S}(T-t)$, $i\in S$, and the state trajectory is

[TABLE]

where $D_{S}=\sum_{i\in S}d_{i}$, $D_{N\setminus S}=\sum_{i\in{N\setminus S}}d_{i}$, and $s=|S|$.

Substituting the obtained expression for the state (28) and the optimal controls $\bar{u}(t)$ into (7) we obtain the expression for the $\alpha$-characteristic function (20).

Note that since the controls of the players from the coalition and its complement are decoupled, the $\alpha$-characteristic function coincides with the $\beta$-characteristic function for the studied problem.

The $\delta$- (resp., $\zeta$-) characteristic function is obtained by solving the maximization problem (27) (resp., minimization problem (26)), while the remaining players use precomputed strategies: the Nash equilibrium solution in the first case and the cooperative agreement in the second case. We omit the details and only show the expressions for the state trajectories:

[TABLE]

and

[TABLE]

Substituting the expressions for optimal controls and the state variable into (8) and (9) we recover the expressions (21) and (22).

Finally, the expression for the $\eta$-characteristic function is obtained by a mere substitution of the precomputed strategies in the formula (11). This yields (23).

It is worth noting that the regularity assumption (15) guarantees that the optimal controls in all the considered cases take on the values from the admissible control sets $U_{i}=[0,b_{i}]$.

Proof 7** (Proposition 4.4).**

We will prove the first equality. The second one can be shown along the same lines. For the sake of brevity, we will denote $V^{x}(S,x(t),T-t)$ by $V^{x}(S)$ for $x\in\{\alpha,\delta\}$.

For any coalition $S\subset N$ we have

[TABLE]

and

[TABLE]

Thus it holds that

[TABLE]

At the next step we wish to show that for any $i\in N$ and for all $S\subseteq N$ such that $i\in S$ the respective component of the difference between two Shapley values is equal to [math]:

[TABLE]

We note that for any $i\in N$, the set of all subsets of $N$ containing $i$ can be represented as a number of pairs: $S$ and $(N\setminus S)\cup\{i\}$, where $S$ is an arbitrary subset of $N$ containing $i$. For any $S\subset N$ such that $i\in S$ we have

[TABLE]

This implies that the respective summands in (29) cancel thus (29) turns to zero. The second equality in Prop. 4.4 is shown in the same way.∎