Game-theoretic Analysis to Content-adaptive Reversible Watermarking

TL;DR

This paper models the interaction between two encoders in reversible watermarking as a game, analyzing equilibrium strategies for optimal payload embedding under distortion constraints.

Contribution

It introduces a novel game-theoretic framework for analyzing content-adaptive reversible watermarking involving two encoders with cooperative and non-cooperative strategies.

Findings

Equilibrium strategies are derived for both cooperative and non-cooperative scenarios.

The impact of one encoder's strategy on the other's embedding capacity is quantified.

The model provides insights into optimal embedding strategies under distortion limits.

Abstract

While many games were designed for steganography and robust watermarking, few focused on reversible watermarking. We present a two-encoder game related to the rate-distortion optimization of content-adaptive reversible watermarking. In the game, Alice first hides a payload into a cover. Then, Bob hides another payload into the modified cover. The embedding strategy of Alice affects the embedding capacity of Bob. The embedding strategy of Bob may produce data-extraction errors to Alice. Both want to embed as many pure secret bits as possible, subjected to an upper-bounded distortion. We investigate non-cooperative game and cooperative game between Alice and Bob. When they cooperate with each other, one may consider them as a whole, i.e., an encoder uses a cover for data embedding with two times. When they do not cooperate with each other, the game corresponds to a separable system, i.e., both want to independently hide a payload within the cover, but recovering the cover may need cooperation. We find equilibrium strategies for both players under constraints.

Equations

$$H(p_i)=-p_i\cdot {\text{log}}_2(p_i)-(1-p_i)\cdot {\text{log}}_2(1-p_i)$$

$$H(p_1)\le H(p_2)\le ...\le H(p_l),$$

$$A_i(s_i,t_i)=n{s}_i\left(1-H(\frac{t_i}{2})-H(p_i)\right).$$

$$B_i(s_i,t_i)=n{t}_i\left(1-H(p_i+\frac{s_i}{2}-p_i{s}_i)\right).$$

$$P_{\text{A}}(\text{s},\text{t})=\sum _{i=1}^l{A}_i(s_i,t_i),$$

$$P_{\text{B}}(\text{s},\text{t})=\sum _{i=1}^l{B}_i(s_i,t_i).$$

$$\begin{array}{rl}{P}_{\text{A}}(\text{s},\text{t})& \le \sum _{i=1}^{l}n{s}_i\left(1-H(p_i)\right)\\ & \text{and}{P}_{\text{B}}(\text{s},\text{t})\le \sum _{i=1}^{l}n{t}_i\left(1-H(p_i)\right).\end{array}$$

$$\rho ({\text{x}}_i)\ge \rho ({\text{x}}_{i+1}),\forall i\in [1,l-1],$$

$$\rho ({\text{x}}_i)\propto \frac{1}{H(p_i)},\forall i\in [1,l].$$

$$D_{\text{A}}(\text{s})=\sum _{i=1}^l\frac{n{s}_i}{2}\rho ({\text{x}}_i)\le d,$$

$$D_{\text{B}}(\text{t})=\sum _{i=1}^l\frac{n{t}_i}{2}\rho ({\text{x}}_i)\le d.$$

$$P_{\text{A}}(\text{s},{\text{t}}^{\ast })\le P_{\text{A}}({\text{s}}^{\ast },{\text{t}}^{\ast }),$$

$$P_{\text{B}}({\text{s}}^{\ast },\text{t})\le P_{\text{B}}({\text{s}}^{\ast },{\text{t}}^{\ast }).$$

$$s_1\le p_{\text{max}}=\text{min}\{1,\frac{2d}{n\rho ({\text{x}}_1)}\}\text{and}{t}_1\le p_{\text{max}}.$$

$$P_{\text{A}}(\text{s},\text{t})=\sum _{i=1}^{l}n{s}_i{T}_i\text{and}{P}_{\text{B}}(\text{s},\text{t})=\sum _{i=1}^{l}n{t}_i{S}_i.$$

$$\frac{\partial P_{\text{B}}(\text{s},\text{t})}{\partial t_i}(t_i^{\ast })=0\text{and}\frac{\partial P_{\text{A}}(\text{s},\text{t})}{\partial s_i}(s_i^{\ast })=0,\forall i\in \{1,2\},$$

$$\frac{S_1}{S_2}=\frac{T_1}{T_2}=\frac{\rho ({\text{x}}_1)}{\rho ({\text{x}}_2)}.$$

$$\frac{n{s}_i}{2}\rho ({\text{x}}_i)\le d_i^{\text{A}}\text{and}\frac{n{t}_i}{2}\rho ({\text{x}}_i)\le d_i^{\text{B}}.$$

$$1-H(t_i^{\ast }/2)\ge H(p_i),\forall i\in [1,l].$$

$$d_{\text{B}}^{j,k}=d-\sum _{i\ne j,k}\frac{n{t}_i}{2}\rho ({\text{x}}_i)$$

$$B_{j,k}(t_j)=n{t}_j{S}_j+n{t}_k{S}_k=n{t}_j{S}_j+\frac{2d_{\text{B}}^{j,k}-n{t}_j\rho ({\text{x}}_j)}{\rho ({\text{x}}_k)}{S}_k.$$

$$\frac{\partial B_{j,k}}{\partial t_j}(t_j^{\ast })=0$$

$$d_{\text{A}}^{j,k}=d-\sum _{i\ne j,k}\frac{n{s}_i}{2}\rho ({\text{x}}_i)$$

$$A_{j,k}(s_j)=n{s}_j{T}_j+n{s}_k{T}_k=n{s}_j{T}_j+\frac{2d_{\text{A}}^{j,k}-n{s}_j\rho ({\text{x}}_j)}{\rho ({\text{x}}_k)}{T}_k.$$

$$\frac{\partial A_{j,k}}{\partial s_j}(s_j^{\ast })=0$$

$$\frac{S_j}{S_k}=\frac{T_j}{T_k}=\frac{\rho ({\text{x}}_j)}{\rho ({\text{x}}_k)}\ge 1,\forall j<k.$$

$$S_i=\frac{\rho ({\text{x}}_i)}{\rho ({\text{x}}_1)}\alpha \text{and}{T}_i=\frac{\rho ({\text{x}}_i)}{\rho ({\text{x}}_1)}\beta .$$

$$s_i^{\ast }=\frac{H^{-1}\left(1-{\alpha }^{\ast }\rho ({\text{x}}_i)/\rho ({\text{x}}_1)\right)-p_i}{1/2-p_i},$$

$$t_i^{\ast }=2H^{-1}\left(1-H(p_i)-{\beta }^{\ast }\rho ({\text{x}}_i)/\rho ({\text{x}}_1)\right),$$

$$\sum _{i=1}^l\frac{n{s}_i^{\ast }}{2}\rho ({\text{x}}_i)=\sum _{i=1}^l\frac{n{t}_i^{\ast }}{2}\rho ({\text{x}}_i)=d.$$

Full text

1 Introduction

Reversible watermarking (RW), also called reversible data hiding (RDH) [1], [2], enables us to embed a secret payload in a cover. Both the embedded payload and the original cover content can be perfectly retrieved from the marked content. RW is helpful in sensitive scenarios requiring no permanent distortion of the cover such as military communication and multimedia archive management. RW is fragile, i.e., one will find it is not authentic if the marked data was tampered. RW can be evaluated by the rate-distortion performance. That is, we always expect to embed as many message bits as possible for a fixed distortion level. In other words, we hope to reduce the distortion as much as possible for a payload.

A straightforward idea to achieve reversibility is applying lossless compression [3], i.e., an encoder selects the noise-like component of the cover such as the least significant bits (LSBs) of a grayscale image, and then losslessly compress them to reserve additional space. The secret message will be embedded into the reserved space. Though altering the noise-like component does not introduce noticeable visual artifacts, the pure embedding capacity is limited due to the low compression rate. To this end, more efficient algorithms such as difference expansion [1], histogram shifting [2] and their variants [4], [5], [6], are proposed to enlarge capacity or reduce distortion. We will not review these works since it is not the main interest of this paper.

Though current advanced RW algorithms differ from lossless compression intuitively, they are essentially similar to each other. They all aim to find the compressible component of the cover to carry the secret message. The difference is, lossless compression considers cover elements as compressible variables, while the others try to fully exploit the correlations between elements, which can be regarded as a kind of “semantic lossless compression” [7].

From the theoretical perspective, studying lossless compression based RW is still desirable, though people have developed a number of new practical algorithms. The theoretical results is helpful to guide or inspire us to design new RW systems, which has motivated the authors to revisit lossless compression based RW in this paper. However, different from traditional works, we use game theory to analyze RW. Unlike previous games designed to robust or saying irreversible watermarking games [8], [9], [10], in the proposed game, there are two players named Alice and Bob, both of whom want to embed a different payload into a specific cover object. It is different from previous practical RW systems, where only one encoder is considered. We point that, our work actually can generalize them. The reason is, Alice and Bob could cooperate with each other. In this way, one may consider Alice and Bob as a whole, indicating that, an encoder will use a cover for data embedding with two times, which corresponds to multi-layer embedding. When they do not cooperate with each other, the game between Alice and Bob actually corresponds to a separable RW system, i.e., both want to independently hide a payload within the cover, but recovering the cover may need cooperation.

The rest are organized as follows. In Section 2, we review prior arts. In Section 3, we introduce the basic setup of our game model. In Section 4, we analyze the non-cooperative game between Alice and Bob, followed by the cooperative game in Section 5. We conclude this work in Section 6.

2 Prior Arts

Though steganography should be distinguished from watermarking, they are correlated to each other. The first work combining game theory and steganography was probably proposed by Ettinger [8]. In the work, a zero-sum game between the data hider and the attacker was presented, for which the equilibrium is such a strategy profile that none of the two players wants to profit from adjusting two different partial strategies, for a fixed strategy of the opponent. Moulin and O’Sullivan [9] characterized the watermarking codes as a capacity game between the data hider and the attacker, in which two models, i.e., private game and public game, of watermarking systems were presented. Baruch and Merhav [10] investigated the capacity and error exponents games of private watermarking games, where the attack channel is completely general and unknown to the hider and receiver.

Cohen and Lapidoth [11] derived a coding capacity formula of the watermarking game for a Gaussian covertext and squared-error distortions. They showed that the capacities of the public and private watermarking are the same, which is in some analogy to Costa’s result [12] on channel coding with side information under the Gaussian quadratic regime. Baruch and Merhav [13] extended the public game [9] by dropping the assumptions that the receiver knows the attack channel and that this channel is memoryless or blockwise memoryless.

Ker [14] introduced a threshold game to batch steganography [15], in which a steganographer should decide how to distribute a payload into multiple pieces each embedded into a selective cover. The result indicated that, the optimal strategy of the steganographer is likely to be extreme concentration of the payload into as few as covers as possible, or the payload is spread as thinly as possible.

Sch$\ddot{\text{o}}$ttle and B$\ddot{\text{o}}$hme [16] defined the heterogeneity as a necessary condition for adaptive steganography, and presented a game-theoretic model for the whole steganographic process including cover generation, adaptive embedding, and a detector which anticipates the adaptivity. Though the model exhibited a unique equilibrium in mixed strategies, it investigated a cover model with only two locations. To this end, Sch$\ddot{\text{o}}$ttle and B$\ddot{\text{o}}$hme [17] further extended the model from the very artificial case to covers sized $n$. Their results are constructive in sense that an equilibrium can be efficiently found for any vector of predictability. More games designed for adaptive steganography can be found in [18], [19], [20].

3 Game Setup

3.1 Description of the Cover

The cover to be embedded is written as X = $(\textbf{x}_{1},\textbf{x}_{2},...,\textbf{x}_{l})$, where $\textbf{x}_{i}=\{x_{i,j}\}_{j=1}^{n}$ represents a binary sequence with the distribution $p_{i,1}=\sum_{j=1}^{n}x_{i,j}/n$ and $p_{i,0}=1-p_{i,1}$. It is assumed that, for any $i\neq j$, there has no intersection between $\textbf{x}_{i}$ and $\textbf{x}_{j}$. We point that, the term ‘cover’ here actually means the data-embedding channel. For example, if we use the LSBs of an image for data embedding, we can separate the LSB plane from the image, and divide it into disjoint bit-vectors to constitute X. On the one hand, such definition has good generalization for lossless compression based reversible watermarking. On the other hand, it allows us to well model the rate-distortion game between two encoders latter.

3.2 Rate-distortion Game between Two Players

In RW, to achieve superior rate-distortion performance, it is generally required that, the secret bits should be preferentially embedded in the smooth area of the cover, rather than the complex one. Since we use binary sequences here, the smoothness of a binary sequence is defined as the information entropy. The smaller the entropy, the more the embeddable bits, implying better rate-distortion performance. We define the smoothness of $\textbf{x}_{i}$ as $H(p_{i})$, where $p_{i}=\text{min}\{p_{i,0},p_{i,1}\}\leq\frac{1}{2}$, and

[TABLE]

means the binary entropy. Without the loss of generality, we assume that

[TABLE]

meaning that, $\textbf{x}_{i}$ is more suitable for data embedding than $\textbf{x}_{j}$, for any $i<j$.

Suppose that, there are two players Alice and Bob, both expecting to hide a payload within X. Alice first hides a payload by modifying the bits throughout X under the constraint that the total amount of distortion is no more than $d$. A strategy for Alice is an $l$-tuple of probabilities $(s_{1},s_{2},...,s_{l})$, where $s_{i}\in[0,1],\forall i\in[1,l]$. It indicates that, for each $i\in[1,l]$, Alice chooses $s_{i}\times 100\%$ pixels from $\textbf{x}_{i}$ with a secret key and losslessly compress the bits. These bit-positions will carry the compressed code and the secret payload.

The resulting data Y will be sent to Bob, who embeds another payload into Y by the same means subjected to $d$ as well, resulting in another marked data Z. A strategy for Bob is therefore another $l$-tuple of probabilities $(t_{1},t_{2},...,t_{l})$. Though both strategies are real vectors, the number of bit-positions is an integer. We use real numbers for better analysis.

With Z, Bob can directly retrieve the hidden data and his compressed code. However, Alice should correct the errors produced by Bob after data extraction. For Alice, the compressed code for $\textbf{x}_{i}$ requires $ns_{i}H(p_{i})$ bits. Since the channel capacity for a binary symmetric channel with bit error probability $p$ is $1-H(p)$, the size of embeddable payload of $\textbf{x}_{i}$ in bits for Alice is therefore

[TABLE]

The size of embeddable payload of $\textbf{x}_{i}$ in bits for Bob is

[TABLE]

For maximizing the embedding capacity, a strategy profile $(\textbf{s},\textbf{t})$ = $(s_{1},s_{2},...,s_{l};$ $t_{1},t_{2},...,t_{l})$ for Alice gives a payoff

[TABLE]

and the payoff for Bob is

[TABLE]

It can be easily proved that

[TABLE]

Both want to maximize their own payoff, while subjected to a bounded distortion. Since data embedding in smooth area gives a larger capacity, intuitively, both will absolutely embed their own payload into $\textbf{x}_{i}$ with a small index (if there has no opponent). Thus, when introducing competition, as limited and precious resource, the cost of data embedding in $\textbf{x}_{i}$ will be surely higher than that of $\textbf{x}_{j}$, for any $i<j$ for both players. Let $\rho(\textbf{x}_{i})$ denote the cost (distortion) of flipping any individual bit in $\textbf{x}_{i}$. It can be therefore assumed that,

[TABLE]

i.e.,

[TABLE]

We limit us to additive distortion. Namely, we have

[TABLE]

and

[TABLE]

By using different $\rho$, sophisticated models of the effects of modifying X may be accommodated in this game. It is pointed that, though there has an embedding order for two players, they both know complete information about the game except the data-embedding key held by the opponent.

4 Non-cooperative Game

When Alice and Bob do not cooperate with each other, an equilibrium is a profile $(\textbf{s}^{*},\textbf{t}^{*})$ = $(s_{1}^{*},s_{2}^{*},...,s_{l}^{*};$ $t_{1}^{*},t_{2}^{*},...,t_{l}^{*})$ that

[TABLE]

and

[TABLE]

4.1 Case $l=1$

We first analyze the case $l=1$. It can be easily obtained from Eq. (8, 9) that

[TABLE]

It is observed from Eq. (2) that, no matter what strategy Alice takes, Bob will always choose $t_{1}=p_{\text{max}}$ since it achieves the maximum payoff. For a fixed $t_{1}$, the optimal response for Alice will be $s_{1}=p_{\text{max}}$. Therefore, the equilibrium is uniquely $(\textbf{s}^{*},\textbf{t}^{*})=(p_{\text{max}},p_{\text{max}})$. Fig. 1 shows the equilibrium payoffs due to different $p_{\text{max}}$. It is observed that, the payoff of Bob is never less than Alice. Since Alice needs correct errors produced by Bob, when the distortion threshold is larger than a threshold (which allows Bob to produce more errors), Alice may not embed any extra bits (i.e., the payoff is negative), e.g., in Fig. 1, when $p_{\text{max}}>0.75$ for the left case, Alice cannot embed a pure payload.

4.2 Case $l=2$

Let $T_{i}=1-H(\frac{t_{i}}{2})-H(p_{i})$ and $S_{i}=1-H(p_{i}+\frac{s_{i}}{2}-p_{i}s_{i})$, we have

[TABLE]

Neither Alice or Bob wants to deviate from an equilibrium. In case $l=2$, to find an equilibrium, we must insure that Bob does not profit from readjusting $t_{1}$ and $t_{2}$ for a fixed s. Similarly, Alice does not profit from readjusting $s_{1}$ and $s_{2}$ for a fixed t. Therefore we have

[TABLE]

from which we can obtain

[TABLE]

When $\frac{\rho(\textbf{x}_{1})}{\rho(\textbf{x}_{2})}$ is specified at the very beginning, it is straightforward to draw out all strategy profiles meeting Eq. (15). For the ‘sub-cover’ $\textbf{x}_{i}$, let $d_{i}^{\text{A}}$ and $d_{i}^{\text{B}}$ be the bounded distortion for both players, i.e.,

[TABLE]

The optimal ‘sub-strategy’ profile is therefore determined as $(\text{min}\{1,\frac{2d_{i}^{\text{A}}}{n\rho(\textbf{x}_{i})}\}$, $\text{min}\{1,\frac{2d_{i}^{\text{B}}}{n\rho(\textbf{x}_{i})}\})$. It implies that, the optimal response for an individual player is only dependent of his or her own distortion distribution. And, the distortion upper-bound itself for a sub-cover gives the maximum ‘sub-payoff’. It can be derived that, $(\textbf{s}^{*},\textbf{t}^{*})$ can be determined by moving a specified initial strategy profile towards the corresponding direction until the distortion bound is reached.

We take Fig. 2 for example, where $p_{1}=0.005$, $p_{2}=0.05$, $\frac{\rho(\textbf{x}_{1})}{\rho(\textbf{x}_{2})}=2$. The left draws out all strategies that meet Eq. (15). It reduces the raw strategy space for determining the final equilibrium. By sampling a set of points with a small step (in our simulation, we set its value as $0.1\times 10^{3}$) on both curves, we can produce two strategy sequences, each element in which can be orderly indexed by a value. Thus, we can draw out all strategy profiles as the right figure shown in Fig. 2. It is observed that, fixing the strategy of either player, the payoff curve of the other player is a strictly monotone increasing function. It means that, the equilibrium will be such a strategy profile that maximizes the distortion, which can be determined during the process of moving from a start point to the end point, as shown in the left. Notice that, we always force Bob to allow Alice to embed a non-negative pure payload in arbitrary sub-cover; otherwise, the game is unfair to Alice. Namely, it is always required that

[TABLE]

4.3 Case $l>2$

Similarly, in case $l>2$, neither Alice or Bob deviates from an equilibrium. We fix all components of t except for $t_{j}$ and $t_{k}~{}(j\neq k)$, and distribute the distortion

[TABLE]

between these two components. Let

[TABLE]

To find an equilibrium, we must insure that Bob does not profit from readjusting $t_{j}$ and $t_{k}$ for a fixed s. Therefore we have

[TABLE]

for an equilibrium $t_{j}^{*}$. We continue to fix all the components of s except for $s_{j}$ and $s_{k}$ where $1\leq j<k\leq l$, and distribute the remaining distortion

[TABLE]

between these two components. Let

[TABLE]

Since Alice does not profit from readjusting $s_{j}$ and $s_{k}$, we have

[TABLE]

for an equilibrium $s_{j}^{*}$. Thus, according to Eq. (20, 23), we have

[TABLE]

Suppose that, $S_{1}=\alpha\in[0,1]$ and $T_{1}=\beta\in[0,1]$, we have

[TABLE]

It can be inferred that, the equilibrium $(\textbf{s}^{*},\textbf{t}^{*})$ should correspond to such a pair $(\alpha^{*},\beta^{*})\in[0,1]^{2}$ that,

[TABLE]

[TABLE]

and

[TABLE]

which can be effectively solved by binary search since the data-embedding distortion functions for both players are strictly monotone w.r.t. $(\alpha,\beta)$. Notice that, the binary search operation is used twice for determining $\textbf{s}^{*}$ and $\textbf{t}^{*}$ respectively.

5 Cooperative Game

When Alice and Bob cooperate with each other, the payoff can be rewritten as

[TABLE]

for which an equilibrium is such a pair $(\textbf{s}^{*},\textbf{t}^{*})$ that preferentially meets:

[TABLE]

and then meets:

[TABLE]

Unlike other cooperative games, to find the equilibrium, we have to consider the side information shared between Alice and Bob. That is, we have to analyze two situations below:

• •

Case 1: They share the key of choosing the bit-positions to be embedded.

• •

Case 2: They do not share the key mentioned above.

Case 1 corresponds to the double embedding strategy commonly used in RW. In this case, we can consider Alice and Bob as a whole. When Alice chooses $ns_{i}$ positions in $\textbf{x}_{i}$, Bob can use the rest positions. We thus have $s_{i}+t_{i}\leq 1,\forall i\in[1,l]$. It indicates that, Bob will not produce extraction errors. Therefore,

[TABLE]

which allows

[TABLE]

It can be then inferred that, $\textbf{s}^{*}$ = $\textbf{t}^{*}$ = ($\frac{w_{1}^{*}}{2}$, $\frac{w_{2}^{*}}{2}$, …, $\frac{w_{l}^{*}}{2}$) will be an equilibrium. To this end, we have to solve the following task:

[TABLE]

subjected to $\textbf{w}\in[0,1]^{l}$ and

[TABLE]

This is a classical linear programming problem, which can be solved by simplex algorithm. Accordingly, an equilibrium $\textbf{w}^{*}$ can be determined as a vertex of the corresponding high-dimensional polytope. Actually, when Alice and Bob share the secret key of choosing the bit-positions to be embedded, they can be considered as a whole. That means, for a strategy profile $(\textbf{s},\textbf{t})$, essentially, it requires us to randomly choose $(s_{i}+t_{i})\times 100\%$ of the bit-positions of $\textbf{x}_{i}$ for data embedding for all $1\leq i\leq l$. Thus, we only need consider one encoder. In this way, we can rewrite the payoff as:

[TABLE]

where $\textbf{w}\in[0,1]^{l}$ indicates the strategy. It is required that:

[TABLE]

Obviously, maximizing $P(\textbf{w})$ subjected to Eq. (37) is equivalent to solving Eq. (34). It is true for Case 2 that,

[TABLE]

which allows that

[TABLE]

Neither Alice or Bob deviates from an equilibrium. It is inferred that, the equilibrium $(\textbf{s}^{*},\textbf{t}^{*})$ should be corresponding to such a pair $(\alpha^{*},\beta^{*})\in[0,1]^{2}$ that, $s_{i}^{*}=\frac{H^{-1}\left(1-\alpha^{*}\rho(\textbf{x}_{i})/\rho(\textbf{x}_{1})\right)-p_{i}}{1/2-p_{i}}$ and $t_{i}^{*}=2H^{-1}\left(1-H(p_{i})-\beta^{*}\rho(\textbf{x}_{i})/\rho(\textbf{x}_{1})\right)$, where

[TABLE]

This can be effectively solved by triple search111Triple search allows us to find the minimum or maximum point of a parabola within a time complexity of $O(\text{log}_{3}N)$. or gradient descent since Eq. (29, 30, 31) show a concave payoff function. Notice that, here, we do not require that both Alice and Bob introduce the maximum distortion $d$.

6 Conclusion

We present game-theoretic analysis to content-adaptive reversible watermarking. Different from games designed for steganography and robust watermarking, which aim to find the equilibrium between the encoder and the attacker, we focus on two encoders, who, however, do not intentionally attack each other, but rather compete with each other for maximizing their own embedding capacity. We show that, when both players do not cooperate with each other, the optimal response for each player, surprisingly, depends on his or her own distortion constraint. When they cooperate with each other, the equilibrium depends on the side information shared between them. We have analyzed two different cases for the cooperative game and provided the effective ways to find the equilibriums.