Regulating Clients' Noise Adding in Federated Learning without Verification

TL;DR

This paper introduces a privacy-preserving pricing mechanism in federated learning that regulates clients' artificial noise addition without requiring verification, balancing training accuracy and privacy, and improving convergence.

Contribution

It proposes a novel, verification-free pricing scheme for privacy regulation in federated learning, optimizing social welfare and ensuring truthful client reporting.

Findings

The mechanism achieves social optimum balancing privacy and accuracy.

It improves convergence bounds by accounting for noise variances.

Simulations demonstrate enhanced system performance with diverse privacy sensitivities.

Abstract

In federated learning (FL), clients cooperatively train a global model without revealing their raw data but gradients or parameters, while the local information can still be disclosed from local outputs transmitted to the parameter server. With such privacy concerns, a client may overly add artificial noise to his local updates to compromise the global model training, and we prove the selfish noise adding leads to an infinite price of anarchy (PoA). This paper proposes a novel pricing mechanism to regulate privacy-sensitive clients without verifying their parameter updates, unlike existing privacy mechanisms that assume the server's full knowledge of added noise. Without knowing the ground truth, our mechanism reaches the social optimum to best balance the global training error and privacy loss, according to the difference between a client's updated parameter and all clients' average parameter. We also improve the FL convergence bound by refining the aggregation rule at the server to account for different clients' noise variances. Moreover, we extend our pricing scheme to fit incomplete information of clients' privacy sensitivities, ensuring their truthful type reporting and the system's ex-ante budget balance. Simulations show that our pricing scheme greatly improves the system performance especially when clients have diverse privacy sensitivities.

Equations

$${\mathbf{w}}_i=\arg \underset{\mathbf{w}}{\min }{F}_i(\mathbf{w},{\mathcal{D}}_i).$$

$${\mathbf{w}}^{\ast }=\arg \underset{\mathbf{w}}{\min }F(\mathbf{w},\mathcal{D})=\arg \underset{\mathbf{w}}{\min }\sum _{i\in \mathcal{N}}{\rho }_i{F}_i(\mathbf{w},{\mathcal{D}}_i),$$

$$\bar{\mathbf{w}}(\mathbf{\rho },\mathbf{w})=\sum _{i\in \mathcal{N}}{\rho }_i{\mathbf{w}}_i,$$

$${\mathcal{L}}^u=\mathbb{E}\{F({\mathbf{w}}^{(T)})-F({\mathbf{w}}^{\ast })\},$$

$$\mathrm{Pr}[\mathcal{P}(x)\in \mathcal{S}]\le e^{\epsilon }\mathrm{Pr}[\mathcal{P}(y)\in \mathcal{S}]+\delta ,$$

$${\tilde{\mathbf{w}}}_i^{(t)}={\mathbf{w}}_i^{(t)}+{\mathbf{n}}_i^{(t)}.$$

$${\mathcal{L}}_i^p=\ln \left(\frac{\mathrm{Pr}[\mathcal{P}({\mathbf{w}}_i)=o]}{\mathrm{Pr}[\mathcal{P}({\mathbf{w}}_i^{\prime })=o]}\right),$$

$$\mathrm{Pr}({\mathcal{L}}_i^p\le {\epsilon }_i)>1-\delta .$$

$${\bar{\mathcal{L}}}_i^p({\sigma }_i)=\frac{cS}{{\sigma }_i}$$

$$J_i({\alpha }_i,{\sigma }_i,{\mathbf{\sigma }}_{-i},\bar{\mathbf{w}})$$

$$\begin{gathered} {\bar{J}}_i(f(\hat{\mathbf{\alpha }},\tilde{\mathbf{w}})|\mathbf{\sigma },\mathbf{\alpha })=J_i({\alpha }_i,{\sigma }_i,{\mathbf{\sigma }}_{-i},\bar{\mathbf{w}})+P_i(\hat{\mathbf{\alpha }},\tilde{\mathbf{w}}) \\ =(1-{\alpha }_i){\bar{\mathcal{L}}}^u(\mathbf{\sigma },\bar{\mathbf{w}})+{\alpha }_i{\bar{\mathcal{L}}}_i^p({\sigma }_i)+P_i(\hat{\mathbf{\alpha }},\tilde{\mathbf{w}}), \end{gathered}$$

$${\bar{J}}_i(f({\alpha }_i,{\mathbf{\alpha }}_{-i},\tilde{\mathbf{w}})|\mathbf{\sigma },\mathbf{\alpha })\le {\bar{J}}_i(f({\hat{\alpha }}_i,{\mathbf{\alpha }}_{-i},\tilde{\mathbf{w}})|\mathbf{\sigma },\mathbf{\alpha }).$$

$$SC(f(\hat{\mathbf{\alpha }},\tilde{\mathbf{w}})|\mathbf{\sigma },\mathbf{\alpha })$$

$$OPT(\mathbf{\alpha })=\underset{\mathbf{\sigma }}{\min }SC(f(\hat{\mathbf{\alpha }},\tilde{\mathbf{w}})|\mathbf{\sigma },\mathbf{\alpha }).$$

$$\gamma =\frac{SC(f(\hat{\mathbf{\alpha }},\tilde{\mathbf{w}})|\mathbf{\sigma },\mathbf{\alpha })}{OPT(\mathbf{\alpha })}\ge 1.$$

$$PoA=\underset{\mathbf{\alpha },N,c,S,T,{\lambda }_F,L_F}{\max }\gamma .$$

$${\bar{\mathbf{w}}}^{mean}(\mathbf{\alpha },\tilde{\mathbf{w}})=\frac{{\sum }_{i\in \mathcal{N}}{\tilde{\mathbf{w}}}_i}{N},$$

$${\bar{\mathbf{w}}}^{MLE}(\mathbf{\alpha },\tilde{\mathbf{w}})=\frac{{\sum }_{i\in \mathcal{N}}\frac{{\tilde{\mathbf{w}}}_i}{{\sigma }_i^2({\alpha }_i)}}{{\sum }_{i\in \mathcal{N}}\frac{1}{{\sigma }_i^2({\alpha }_i)}}.$$

$${\bar{\mathcal{L}}}^u(\mathbf{\sigma },\bar{\mathbf{w}})=\kappa {\Delta }^{MLE}(\mathbf{\sigma })\left(1+\frac{1}{2L_F}{\Delta }^{MLE}(\mathbf{\sigma })\right),$$

$$J_i({\sigma }_i,{\mathbf{\sigma }}_{-i})$$

$$\kappa (1-{\alpha }_i)(\sum _{j\in \mathcal{N}}{\sigma }_j^{-2}{)}^{-\frac{3}{2}}\frac{{\left({\sum }_{j\in \mathcal{N}}{\sigma }_j^{-2}\right)}^{-\frac{1}{2}}+L_F}{L_F}={\alpha }_{i}cS{\sigma }_i,$$

$$\begin{gathered} \kappa \left(\sum _{i\in \mathcal{N}}(1-{\alpha }_i)\right)(\sum _{j\in \mathcal{N}}{\sigma }_j^{-2}{)}^{-\frac{3}{2}}\frac{{\left({\sum }_{j\in \mathcal{N}}{\sigma }_j^{-2}\right)}^{-\frac{1}{2}}+L_F}{L_F} \\ ={\alpha }_{i}cS{\sigma }_i. \end{gathered}$$

$$P_i(\mathbf{\alpha },\tilde{\mathbf{w}})$$

$${\beta }_i^{\ast }(\mathbf{\alpha })=\frac{N^2{\sum }_{j\ne i,j\in \mathcal{N}}(1-{\alpha }_j){\alpha }_{j}cS}{2(N-1{)}^2({\sum }_{i\in \mathcal{N}}(1-{\alpha }_i)({\sigma }_i^{\ast \ast }{)}^3)},$$

$$q=\frac{2(N-1{)}^2}{N}\sum _{i\in \mathcal{N}}{\beta }_i^{\ast }(\mathbf{\alpha }){\sigma }_i^{\ast \ast },$$

$$P_i(\hat{\mathbf{\alpha }},\tilde{\mathbf{w}})$$

$${\Delta }^{inc}({\mathbf{\sigma }}^{\ast },\hat{\mathbf{\alpha }})=\frac{{\left({\sum }_{i=1}^N\frac{({\sigma }_i^{\ast }({\alpha }_i){)}^2}{({\sigma }_i^{\ast }({\alpha }_i={\hat{\alpha }}_i){)}^4}\right)}^{\frac{1}{2}}}{{\sum }_{i=1}^N\frac{1}{({\sigma }_i^{\ast }({\alpha }_i={\hat{\alpha }}_i){)}^2}},$$

$${\mathbb{E}}_{{\mathbf{\alpha }}_{-i}}{J}_i({\alpha }_i,{\hat{\alpha }}_i,{\sigma }_i|r_i(\hat{\mathbf{\alpha }}),p_i(\hat{\mathbf{\alpha }},\tilde{\mathbf{w}}),q)$$

$$=$$

$$-$$

$$+$$

$${\alpha }_i=\{\begin{array}{ll}{\alpha }_L,& \text{ with probability }\eta ,\\ {\alpha }_H,& \text{ with probability }1-\eta .\end{array}$$

Full text

I Introduction

The proliferation of Internet of Things (IoT) technologies has led to the creation of enormous amounts of user data. In an effort to safeguard data privacy, decentralized machine learning approaches such as Federated Learning (FL) [1] have been proposed without sharing individuals’ raw data. During the training phase, each client periodically downloads the global model from a parameter server and updates their local model with their own data. By only transferring gradient parameters instead of raw data, data privacy can be preserved.

Sharing model updates or gradients, however, can also compromise clients’ private information, as analyzing the differences in training parameters uploaded by clients can reveal it. For example, model inverse attack [2, 3] allows training data reconstruction by matching model gradients and optimizing randomly initialized inputs. These attacks for data reconstruction or membership inference [4] are significant threats to FL security and privacy. To prevent privacy leakage, each client can add artificial noise [5, 6, 7] to the transmitted parameters. However, this approach can lead to significant training accuracy loss while making it harder for attackers to inverse the original local data.

Despite of the privacy concerns in FL, clients might also take advantage of the returned global model for benefits. For example, in the “cross-silo” scenario of FL [1] (e.g., in medical and financial institutions), an AI model is trained on clients with large amounts of data and clients also care about the convergence performance. Given both privacy concerns and model training benefits, Noble et al. [8] and Sun et al. [9] studied the tradeoff between utility and privacy for FL. It is also critical to study the clients’ selfish noise-adding strategy based on each individual’s preference of learning performance and privacy requirement.

Our analysis of the worst-case ratio of social costs under clients’ selfish behavior and the social optimum prompts us to study incentive mechanisms to regulate clients’ noise adding and approach the social optimum. Existing works on noise-adding privacy protection mechanisms [5, 6, 7]) assume that the server can verify and know all clients’ contributions, which is challenging to achieve in practice without access to clients’ local data. In contrast, our paper incentivizes honest contributions from clients with performance-based pricing schemes that penalize individuals based on the difference between their local parameters and the average parameter benchmark of all clients. Although the average benchmark may not be the ground truth, we prove its effectiveness in preventing clients from adding excessive local noise.

Specifically, considering clients’ varying privacy sensitivities, we analyse their equilibrium strategies under a properly improved weighted aggregation rule, and optimally design the price charged to each client for achieving the social optimum. Additionally, we extend our pricing scheme to account for incomplete information about clients’ privacy sensitivity, where some clients may not truthfully report their types. Our refined pricing mechanism includes a reward function, a penalty function, and a compensation payment to ensure truthful reporting, social efficiency, and FL system’s ex-ante budget balance. Simulation results demonstrate significant improvement in equilibrium performance, particularly when clients have diverse privacy sensitivities.

The rest of this paper is structured as follows. In Section II we introduce the system model and problem formulation. In Sections III and IV we present the equilibrium analysis and pricing mechanism design under complete and incomplete information of clients’ privacy sensitivity types, respectively. We conclude the paper in Section V. Due to page limit, we put the detailed proofs in the on-line technical report[10].

II System Model and Problem Formulation

In this section, we introduce the system model and formulate the privacy-preserving problem with noise adding.

II-A Preliminaries

First, we present preliminaries about backgrounds on federated learning (FL) and differential privacy (DP).

Federated Learning. Consider an FL system consisting of one parameter server (PS) and $N$ clients in set $\mathcal{N}=\{1,2,\cdots,N\}$. Each client $i\in\mathcal{N}$ holds the local database $\mathcal{D}_{i}$ and aims to iteratively minimize a local loss function $F_{i}$ by training the local parameter vector $\mathbf{w}_{i}$, i.e.,

[TABLE]

The goal of the server is to learn a global model over data at all $N$ distributed clients. The problem to minimize the global loss function $F$ is:

[TABLE]

where $\mathcal{D}=\cup_{i\in\mathcal{N}}\mathcal{D}_{i}$ includes the datasets from all $N$ clients. Formally, in each iteration the server aggregates the updated parameters received from all clients as

[TABLE]

where $\bar{\mathbf{w}}$ is the parameter vector aggregated at the server and $\rho_{i}\geq 0$ is the weightage parameter for aggregation.

Each training iteration of such an FL system usually contains the following three steps: clients’ local training and updating, global aggregation at the server, and parameter broadcast to clients. After a sufficient number of training iterations (say, $T$ rounds) and update exchanges between the server and the distributed clients, the solution to the optimization problem (1) converges to that of the global model. Thus, the training accuracy loss can be defined as the expected global error, i.e.,

[TABLE]

which depends on the aggregation rule in (2). To quantify the convergence bound for the learning, we here make some typical assumptions about the loss functions.

Assumption 1**.**

The global loss function $F(\cdot)$ is convex and $L_{F}$-smooth, i.e., $\left\|\nabla F(\mathbf{x})-\nabla F\left(\mathbf{x}^{\prime}\right)\right\|_{2}\leq L_{F}\left\|\mathbf{x}-\mathbf{x}^{\prime}\right\|_{2},\forall\mathbf{x},\mathbf{x}^{\prime}$.

As in most of the literature (e.g., [6, 17]), the convergence or accuracy error of the FL process is difficult to quantify and is analysed using the convergence bound. Let $\bar{\mathcal{L}}^{u}(\bm{\sigma},{\bar{\mathbf{w}}})$ denote the upper bound of the expected global error ${\mathcal{L}}^{u}$ in (3), then clients will take it into account when making noise-adding decisions in each iteration. Note that the bound of the training accuracy loss function is for all FL clients, and depends on all clients’ noise-adding strategies as well as the server’s aggregation rule.

Differential Privacy. Despite that all clients locally compute training gradient parameter $\mathbf{w}_{i}$ and send it to the server, attacks such as gradient inverse attack[2, 3] might disclose the original training data without accessing the datasets. As proposed in [13], $(\varepsilon,\delta)$-Differential Privacy (DP) provides a criterion for privacy preservation of distributed data processing systems, and the definition is formally given as follows.

Definition 1** (Differential Privacy).**

A randomized algorithm $\mathcal{P}:\mathcal{X}\rightarrow\mathcal{R}$ with domain $\mathcal{X}$ and range $\mathcal{R}$ is $(\varepsilon,\delta)$-differentially-private if for all $\mathcal{S}\subseteq\mathcal{R}$ and for all $x,y\in\mathcal{X}$ such that $||x-y||_{1}\leq 1$:

[TABLE]

where $\varepsilon$ is the distinguishable bound of all outputs on adjacent datasets $x$ and $y$ in a database, and $\delta$ represents the event that the ratio of the probabilities for two adjacent datasets $x$ and $y$ cannot be bounded by $e^{\varepsilon}$ after the algorithm $\mathcal{P}$.

Given this privacy definition, we aim to provide useful privacy protection to the transmitted parameters $\mathbf{w}_{i}$ and thus the raw dataset $\mathcal{D}_{i}$, as in [6]. Specifically, in the $t$-th round, suppose client $i$ adds Gaussian noise $\mathbf{n}_{i}\sim\mathcal{N}(0,\sigma_{i}^{2})$ locally to the transmitted parameter $\mathbf{w}_{i}$, as

[TABLE]

According to the Gaussian mechanism[13], the noise111For the sake of simplicity, we neglect the superscript $(t)$ for each round. ${\mathbf{n}}_{i}$ with standard deviation $\sigma_{i}=\frac{cS}{\varepsilon_{i}}$ ensures $(\varepsilon_{i},\delta)$-DP for any $\varepsilon_{i}\in(0,1)$, where $c\geqslant\sqrt{2\ln(1.25/\delta)}$ is a constant, $S=\max_{\mathcal{D}_{i},\mathcal{D}_{i}^{\prime}:d(\mathcal{D}_{i},\mathcal{D}_{i}^{\prime})=1}||w_{i}(\mathcal{D}_{i})-w_{i}(\mathcal{D}_{i}^{\prime})||_{2}$ is the sensitivity of the local training process.

Under the DP definition and the Gaussian mechanism, the privacy algorithm $\mathcal{P}$ outputs the noise-adding parameter $\tilde{{\mathbf{w}}}_{i}$ and provides a privacy guarantee to clients’ transmitted parameters $\mathbf{w}_{i}$. Following Definition 1, [13] also defines a quantity to measure the privacy loss of the privacy algorithm after observing the output $\tilde{{\mathbf{w}}}_{i}$ as $o$, i.e.,

[TABLE]

where ${{\mathbf{w}}}_{i}$ and ${{\mathbf{w}}}^{\prime}_{i}$ are two adjacent local parameters (without noise) and $\mathcal{L}_{i}^{p}$ can be positive or negative. Then we have

[TABLE]

That is, the privacy loss $\mathcal{L}_{i}^{p}$ is upper-bounded by $\varepsilon_{i}$ with a large probability (given $\delta$ is small). Thus, we use $\bar{\mathcal{L}}_{i}^{p}=\varepsilon_{i}$ as the approximate bound of privacy loss for client $i$, to be a privacy factor affecting the client’s noise-adding decision. Equivalently, we have the bound of the privacy loss

[TABLE]

from the Gaussian mechanism for client $i$.

II-B Our Mechanism Design at the Server

Given the bounds for the global training error $\bar{\mathcal{L}}^{u}$ and privacy loss $\bar{\mathcal{L}}_{i}^{p}$ in (4), we denote $\bm{\alpha}=\{\alpha_{i}\in\mathcal{G}=[0,1]|i\in\mathcal{N}\}$ as the $N$ clients’ privacy sensitivity types, which are private information and need to be reported by themselves to the server. Each client $i$ minimizes a total cost function $J_{i}$:

[TABLE]

by designing the noise scale $\sigma_{i}$ based on their own privacy sensitivity types $\alpha_{i}$. Note that such a type $\alpha_{i}$ is a constant parameter, to measure a client’s individual preference on the utility-privacy tradeoff[18].

After receiving $\tilde{\mathbf{w}}_{i}$ with the artificial noise, the server aggregates all the local parameters as a weighted sum in (2). With the aim to utilize pricing schemes for achieving better social performance at the server, a deterministic mechanism $f$ outputs a pricing scheme $\mathbf{P}$ according to clients’ reporting $\bm{\hat{\alpha}}$ of their privacy sensitivities and local training outcomes $\tilde{\mathbf{w}}$, i.e., $f(\bm{\hat{\alpha}},\tilde{\mathbf{w}})=\{\mathbf{P}(\bm{\hat{\alpha}},\tilde{\mathbf{w}})\}$.

Considering the server’s pricing scheme, each client $i$’s cost function $J_{i}(\alpha_{i},\sigma_{i},\bm{\sigma}_{-i},{\bar{\mathbf{w}}})$ in (5) can be modified as

[TABLE]

where $P_{i}(\bm{\hat{\alpha}},\tilde{\mathbf{w}})$ is the ex-post price charged to client $i$ after his reporting and computation.

FL clients are self-interested and cost-driven, such that they will respond to the pricing with the best noise-adding strategies to minimize their costs. Next, we explicitly model the interaction between FL clients and the server in a two-stage dynamic Bayesian game as follows.

• •

In Stage I, the server announces the number of whole training rounds or end-time $T$, the mechanism including the global aggregation rule $\bar{\textbf{w}}(\tilde{\mathbf{w}},\bm{\hat{\alpha}})$ and the (ex-post) pricing scheme $\bm{P}(\bm{\hat{\alpha}},\tilde{\mathbf{w}})$ to clients. The pricing schemes aim to incentivize truthful reporting of clients’ types and socially optimal strategies. The clients’ cost functions $J_{i}$’s in (5) are common knowledge.

• •

In Stage II, clients report their types ${\hat{\alpha}_{i}}$’s to the server and upload parameters $\tilde{\mathbf{w}}_{i}$ with noise level $\sigma_{i}$ in distributed computation. Correspondingly, the server returns the global parameter ${\bar{\mathbf{w}}}$ and charges a price $P_{i}(\bm{\hat{\alpha}},\tilde{\mathbf{w}})$ to each client $i$ at the end of each iteration.

Given that clients’ privacy sensitivity $\alpha_{i}$’s are private information, for achieving better social performance, the mechanism design at the server needs to be truthful and efficient. Next we formally define the truthfulness.

Definition 2** (Truthfulness).**

A mechanism is truthful if no agent can benefit from misreporting his type $i$. Formally, given client $i$, type profile $\bm{\alpha}=\{\alpha_{i},\bm{\alpha}_{-i}\}\in\mathcal{G}^{N}$, and any misreported type ${\hat{\alpha}_{i}}\in\mathcal{G}$, it holds that

[TABLE]

We are also interested in designing truthful mechanisms that perform well with respect to minimizing the social cost. Define the social cost of a mechanism $f(\bm{\hat{\alpha}},\tilde{\mathbf{w}})$ as the sum of costs of $N$ agents, i.e.,

[TABLE]

Let $OPT(\bm{\alpha})$ denote the optimal minimum social cost, i.e.,

[TABLE]

Then we use the concept of price of anarchy (PoA) to tell the maximum efficiency loss due to clients’ selfish behavior by adding too much noise or misreporting types.

Definition 3** (Price of Anarchy).**

The ratio between the social cost and the socially optimal cost for mechanism $f(\bm{\hat{\alpha}},\tilde{\mathbf{w}})$ is

[TABLE]

The Price of Anarchy (PoA) is defined as the maximum of the ratio by scanning through all possible parameters:

[TABLE]

The PoA definition describes an efficiency gap between clients’ equilibrium strategy and the social optimum. If it is non-small, it motivates us to design a mechanism which approaches or even achieves social optimum with $\gamma=1$. Next, we will analyse clients’ equilibrium strategy and prepare the server’s mechanism design.

III Game Analysis and Mechanism Design under Complete information

In this section, we assume that all clients’ types are known to the parameter server (or equivalently, ${\hat{\alpha}_{i}}=\alpha_{i}$), which may be done by checking their historical activities on similar FL tasks. In this scenario, the server’s mechanism $f(\bm{\hat{\alpha}},\tilde{\mathbf{w}})$ reduces to a function of parameters $\tilde{\mathbf{w}}$. We derive clients’ equilibrium noise-adding strategy after introducing a properly improved aggregation rule at the server, and then we design a mechanism to achieve the minimum social cost.

III-A Improved Aggregation Rule at the Server

Given the clients’ training accuracy loss $\bar{\mathcal{L}}^{u}(\bm{\sigma},{\bar{\mathbf{w}}})$ in (5) depends on the server’s aggregation rule, we first consider an aggregation rule ${\bar{\mathbf{w}}}(\bm{\hat{\alpha}},\tilde{\mathbf{w}})={\bar{\mathbf{w}}}(\bm{\alpha},\tilde{\mathbf{w}})$ from (2) to ensure the training convergence performance.

The standard aggregation rule in the FedAvg algorithm [17] uses a simple mean aggregation as

[TABLE]

which does not take clients’ strategic behaviours or added noise variances into consideration. Alternatively, predicting any client $i$’ strategy with distinct standard deviation $\sigma_{i}(\alpha_{i})$ from cost function (5), we properly use the maximum likelihood estimator (MLE) for better aggregation with $\bm{\hat{\alpha}}=\bm{\alpha}$:

[TABLE]

Proposition 1**.**

With the MLE aggregation rule in (8), we give the upper bound of the training accuracy loss function ${\mathcal{L}}^{u}$ in (3) as

[TABLE]

where $\kappa=16||\mathbf{w}^{0}-\mathbf{w}^{*}||_{2}$ with $T=\frac{L_{F}}{\Delta}||\mathbf{w}^{0}-\mathbf{w}^{*}||_{2}$, step size $\eta=1/L_{F}$, $\Delta^{MLE}=(\sum_{i=1}^{N}\sigma_{i}^{-2})^{-\frac{1}{2}}$ measures the bound of the expected error $\mathbb{E}|\hat{\mu}-\mu|$ achieved by MLE. Our improved aggregation rule in (8) leads to a smaller convergence bound than the mean aggregation in (7).

Proposition 1 guides us to choose the MLE aggregation rule for better convergence performance at the server.

III-B Clients’ Strategy under Equilibrium and Social Optimum

By substituting (9) into the cost function (without pricing) for client $i$ in (5), the cost for client $i$ becomes

[TABLE]

By minimizing the individual’s and social costs, we give the equilibrium and social optimal noise-adding strategies for client $i$.

Proposition 2**.**

Given that the server chooses the MLE aggregation rule in (8), the equilibrium noise level $\sigma_{i}^{*}$ for client $i$ with privacy sensitivity type $\alpha_{i}$ is the unique solution to

[TABLE]

while under social optimum, the noise level $\sigma_{i}^{**}$ for client $i$ is the unique solution to

[TABLE]

To show the maximum gap of social costs defined in (6) under clients’ equilibrium in (10) and the social optimum in (11), we use the PoA definition in Definition 3.

Proposition 3**.**

Price of anarchy under complete information goes to infinity under the worst case.

III-C Pricing Mechanism for Achieving Social Optimum

To incentivize client $i$ to choose noise level $\sigma_{i}^{**}$ rather than $\sigma_{i}^{*}$, we design the ex-post pricing scheme $\mathbf{P}(\bm{\hat{\alpha}},\tilde{\mathbf{w}})=\mathbf{P}(\bm{\alpha},\tilde{\mathbf{w}})$ to impose penalty to clients who add too much noise to their local parameters $\mathbf{w}_{i}$’s.

Definition 4** (Pricing Mechanism under Complete Information).**

Under complete information of clients’ types $\bm{\alpha}$, the mechanism $f({\bm{\alpha}},\tilde{\mathbf{w}})=\{\mathbf{P}(\bm{\alpha},\tilde{\mathbf{w}})\}$ decides pricing function

[TABLE]

charged to client $i$, where the penalty coefficient is

[TABLE]

and the compensation payment is

[TABLE]

where $\sigma_{i}^{**}$ is given in (11).

Proposition 4**.**

Under complete information, our pricing mechanism in Definition 4 simultaneously reaches the social optimum with $\gamma=1$ and ex-ante budget balance.

Though our pricing scheme with (12) does not verify each client’s noise/contribution and simply compares each client’s performance to the noisy average benchmark, it already achieves the social optimum. Even when all clients have the same sensitivity types, they worry their randomly computed parameters to be an outlier and do not dare to add much noise. Ex-ante budget balance for each client (achieved by compensation $q$) is an important property in economics[19] and helps ensure long-term client participation.

Without a pricing scheme, Fig. 1 shows a substantial increase in social cost as the variance of $\alpha_{i}$ rises. This trend can be attributed to the increasingly diverse objectives of varying clients, leading to differences in their training goals and privacy concerns. Self-interested client behavior to protect privacy leads to more noise and higher training accuracy loss. However, our pricing scheme maintains lower social costs due to clients’ sensitivity to penalties.

IV Game Analysis and Mechanism Design under Incomplete information

Under incomplete information, all clients’ real types $\bm{\alpha}$ are private information while the only common knowledge is the random type distribution $\alpha_{i}\sim\mathcal{G}(\alpha)$ in $[0,1]$. We propose that the server asks each client $i$ to report his type ${\hat{\alpha}_{i}}$, which might not be truthful due to clients’ selfish behaviours. The Bayesian game for server-client interaction is formulated as in Section II-B. We refine our pricing scheme in Definition 4 to fit.

Definition 5** (Pricing Mechanism under Incomplete Information).**

Under incomplete information of clients’ types $\bm{\alpha}$, our mechanism $f(\bm{\hat{\alpha}},\tilde{\mathbf{w}})=\{\mathbf{P}(\bm{\hat{\alpha}},\tilde{\mathbf{w}})\}$ decides:

[TABLE]

• •

The penalty $p_{i}(\bm{\hat{\alpha}},\tilde{\mathbf{w}})$ aims to incentivize client $i$ to update local parameters with only socially optimal noise variance ${\sigma}_{i}^{**}$. It takes a similar form from Definition 4 as $p_{i}(\bm{\hat{\alpha}},\tilde{\mathbf{w}})=\beta_{i}(\bm{\hat{\alpha}})\left(\tilde{\mathbf{w}}_{i}-\frac{1}{N}\sum_{i=1}^{N}\tilde{\mathbf{w}}_{i}\right)^{2}$, yet here we need to design new parameters $\bm{\beta}(\bm{\hat{\alpha}})$ under incomplete information.

• •

The reward function $r_{i}(\bm{\hat{\alpha}})$ is to incentivize client $i$’s truthful reporting of $\alpha_{i}$ under incomplete information.

• •

The constant compensation payment $q$ is refunded to each client to make the FL system ex-ante budget balanced.

Next, we analyse the Bayesian game between clients and the server under incomplete information by backward induction.

IV-A Each Client’s Decision of ${\hat{\alpha}_{i}}$ and $\sigma_{i}$ in Stage II

Given the pricing mechanism $\mathbf{P}=(\bm{r}(\bm{\hat{\alpha}}),\bm{p}(\bm{\hat{\alpha}},\tilde{\mathbf{w}}),q)$ announced in Stage I from Definition 5, client $i$’s strategy based on his own private type $\alpha_{i}$ includes two parts: the type reporting strategy ${\hat{\alpha}_{i}}(\alpha_{i})$ and the noise variance $\sigma_{i}(\alpha_{i})$.

Aggregation Rule under Incomplete Information. Similar to the aggregation rule under complete information in Section III-A, we still use the MLE aggregation rule in (8), yet we have to replace real $\sigma_{i}$ with $\sigma_{i}^{*}(\alpha_{i}={\hat{\alpha}_{i}})$ according to clients’ reported type ${\hat{\alpha}_{i}}$. In this case, we modify the training accuracy loss function $\bar{\mathcal{L}}^{u}(\bm{\sigma},{\bar{\mathbf{w}}})$ in (9) with

[TABLE]

which depends on clients’ reported types $\bm{\hat{\alpha}}$ and actual noise strategies $\bm{\sigma}^{*}(\bm{\alpha})$ at the Bayesian Nash equilibrium (BNE).

Client $i$’s Expected Cost. Given all clients’ types follow an i.i.d. distribution $\mathcal{G}$, client $i$ needs to estimate all other $N-1$ clients’ strategies $\bm{\hat{\alpha}}_{-i}(\bm{\alpha}_{-i})$ and $\bm{\sigma}_{-i}(\bm{\alpha}_{-i})$ with $\alpha_{j}\sim\mathcal{G}(\alpha)$ $(j\in\mathcal{N}/i)$ when making his own strategies. With the pricing scheme $P_{i}(\bm{\hat{\alpha}},\tilde{\mathbf{w}})$ in Definition 5, we take expectation over $\bm{\alpha}_{-i}$ to obtain the expected cost function for client $i$:

[TABLE]

Given the pricing mechanism $\mathbf{P}=(\bm{r}(\bm{\hat{\alpha}}),\bm{p}(\bm{\hat{\alpha}},\tilde{\mathbf{w}}),q)$ announced in Stage I, client $i$ will optimize noise level $\sigma_{i}^{*}$ as the solution to $\frac{\partial\mathbb{E}_{\bm{\alpha}_{-i}}J_{i}(\alpha_{i},{\hat{\alpha}_{i}},\sigma_{i})}{\partial\sigma_{i}}=0$ and $\hat{\alpha}_{i}^{*}$ as the solution to $\frac{\partial\mathbb{E}_{\bm{\alpha}_{-i}}J_{i}(\alpha_{i},{\hat{\alpha}_{i}},\sigma_{i})}{\partial{\hat{\alpha}_{i}}}=0$ at the BNE to minimize $\mathbb{E}_{\bm{\alpha}_{-i}}J_{i}$ in (LABEL:Equ:_expected_cost_under_incom-_infor-). Next we only focus on the symmetric BNE strategy for each client due to the symmetry of (LABEL:Equ:_expected_cost_under_incom-_infor-), i.e., the noise-adding strategy $\sigma(\alpha_{i})$ and the type reporting strategy $\hat{\alpha}(\alpha_{i})$ without subscripts in these two functions.

IV-B Clients’ Equilibrium Strategies with Binary Type Values

To simplify the analysis, we limit clients’ i.i.d. private type distribution of $\alpha_{i}$ to be two discrete values in $\{\alpha_{L},\alpha_{H}\}$ with $0\leq\alpha_{L}<\alpha_{H}\leq 1$, though similar analysis methods holds for the continuous distribution. The common knowledge is the proportion $\eta$ of all clients’ types:

[TABLE]

Considering the deterministic reporting strategy ${\hat{\alpha}_{i}}\in\{\alpha_{L},\alpha_{H}\}$, clients may truthfully report or misreport. Due to the symmetry of both types of clients, there are three cases for all clients’ type reporting strategies $\bm{\hat{\alpha}}$ in general.

IV-B1 Case 1

all clients reports the same type $\alpha_{L}$ or $\alpha_{H}$, i.e., $\hat{\alpha}(\alpha_{L})=\hat{\alpha}(\alpha_{H})\in\{\alpha_{L},\alpha_{H}\}$ for any $i\in\mathcal{N}$.

In this case, receiving the same type reporting from all clients, the server has to assign the same weight with $\rho_{i}=1$ in the aggregation rule in (2). Also, the reward function and penalty coefficient are indistinctive among all agents.

Lemma 1**.**

In Case 1 where all clients report the same type $\hat{\alpha}(\alpha_{L})=\hat{\alpha}(\alpha_{H})\in\{\alpha_{L},\alpha_{H}\}$ to the server, all clients’ noise adding strategies $\sigma^{case1}(\alpha_{L})$ and $\sigma^{case1}(\alpha_{H})$ at the BNE are solutions to the equation set:

[TABLE]

where $\beta=\beta^{L}$ if $\hat{\alpha}(\alpha_{L})=\hat{\alpha}(\alpha_{L})=\alpha_{L}$, and $\beta=\beta^{H}$ if $\hat{\alpha}(\alpha_{L})=\hat{\alpha}(\alpha_{L})=\alpha_{H}$, $p(n)=\operatorname{Pr}(N_{L}=n)=C_{N-1}^{n}\eta^{n}(1-\eta)^{N-1-n}$.

IV-B2 Case 2

All clients misreport their private types to the server, i.e., ${\hat{\alpha}}(\alpha_{L})=\alpha_{H}$ and ${\hat{\alpha}}(\alpha_{H})=\alpha_{L}$. In this case, the server will use the aggregation rule (8) with $\sigma_{i}^{*}(\alpha_{i}={\hat{\alpha}_{i}})$ according to clients’ misreported type ${\hat{\alpha}_{i}}$.

Lemma 2**.**

In Case 2 where all clients misreport their types to the server, all clients’ noise adding strategies $\sigma^{case2}(\alpha_{L})$ and $\sigma^{case2}(\alpha_{H})$ at the BNE are solutions to the equation set:

[TABLE]

IV-B3 Case 3

All clients report their types truthfully, i.e., ${\hat{\alpha}}(\alpha_{L})=\alpha_{L}$, ${\hat{\alpha}}(\alpha_{H})=\alpha_{H}$. In this case, the server will use the aggregation rule (8) with $\sigma_{i}^{*}(\alpha_{i}={\hat{\alpha}_{i}})$ according to clients’ reported type ${\hat{\alpha}_{i}}$. Similarly, the reward function and penalty coefficient $\bm{\beta}(\bm{\hat{\alpha}})$ include two parts, depending on the outcome type ${\hat{\alpha}_{i}}$.

Lemma 3**.**

In Case 3 where all clients reports their types ${\hat{\alpha}_{L}}(\alpha_{L})=\alpha_{L}$ and ${\hat{\alpha}_{H}}(\alpha_{H})=\alpha_{H}$ truthfully to the server, all clients’ noise adding strategies $\sigma^{case3}(\alpha_{L})$ and $\sigma^{case3}(\alpha_{H})$ at the BNE are solutions to the equation set:

[TABLE]

Notice that in all cases, the server imposes penalties and rewards predicated on all clients’ reported types $\bm{\hat{\alpha}}$. For example, we give the penalty coefficient $\beta_{i}(\hat{\alpha}_{i})$ as

[TABLE]

Both noise-adding strategies $\sigma(\alpha_{L})$ and $\sigma(\alpha_{H})$ are functions of clients’ actual type $\alpha_{i}\in\{\alpha_{L},\alpha_{H}\}$, and depends on clients’ type distribution parameter $\eta$, total number of clients $N$, and the penalty coefficients $\bm{\beta}$. By comparing the expected cost for the minimum, each client $i$ will choose the type reporting $\hat{\alpha}_{i}$ at the BNE from the three cases above.

IV-C Pricing Mechanism Design in Stage I

After analysing all clients’ type reporting and noise adding strategies in Stage II, in this subsection we are ready to proactively design the pricing mechanism $\mathbf{P}=(\bm{r}(\bm{\hat{\alpha}}),\bm{p}(\bm{\hat{\alpha}},\tilde{\mathbf{w}}),q)$ from Definition 5 in Stage I, with the goal of truthful reporting, social efficiency and ex-ante budget balance.

First we define some notations to simplify the BNE results: let $Q_{1}=\sum_{n=0}^{N-1}C_{N-1}^{n}\eta^{n}(1-\eta)^{N-1-n}\cdot p(N-1-n)$, $Q_{2}=\sum_{n=0}^{N-1}C_{N-1}^{n}\eta^{n}(1-\eta)^{N-1-n}\cdot p(n+1)$,

[TABLE]

[TABLE]

where $(\sigma^{**}(\alpha_{L}),\sigma^{**}(\alpha_{H}))$ is the social optimal noise level, $p(n)=C_{N}^{n}\eta^{n}(1-\eta)^{N-n}$ denote the probability that the true number of clients with type $\alpha_{L}$ is exactly $n$.

Proposition 5**.**

In the pricing mechanism $\mathbf{P}=(\bm{r}(\bm{\hat{\alpha}}),\bm{p}(\bm{\hat{\alpha}},\tilde{\mathbf{w}}),q)$ in Definition 5, by setting the penalty coefficient $\bm{\beta}(\bm{\hat{\alpha}})$ with

[TABLE]

[TABLE]

and the reward function

[TABLE]

with $r^{L}=\max\left(\frac{Q_{2}(A_{3}-A_{1})-Q_{1}(A_{2}-A_{4})}{Q_{2}^{2}-Q_{1}^{2}},0\right)$, $r^{H}=\max\left(\frac{Q_{1}(A_{3}-A_{1})-Q_{2}(A_{2}-A_{4})}{Q_{2}^{2}-Q_{1}^{2}},0\right)$, and ${\hat{N}_{L}}(\hat{\bm{\alpha}})=\sum_{i\in\mathcal{N}}1_{\hat{\alpha}_{i}=\alpha_{L}}$ denoting the number of received reporting types of $\alpha_{L}$, our pricing mechanism ensures all clients’ truthful type reporting and achieves the social optimum of noise-adding. The detailed forms of $A_{i}$ (i=1,2,3,4) are given in the on-line technical report [10].

Notice that $p({\hat{N}_{L}}(\hat{\bm{\alpha}}))$ denote the probability that the real number of clients with type $\alpha_{L}$ is exactly the number ${\hat{N}_{L}}(\hat{\bm{\alpha}})$ of received number of reporting with type $\alpha_{L}$. This probability guarantees the incorporation of all reporting decisions into the reward function, thereby preventing clients from deviating. For instance, one cannot misreport for a higher reward, as such symmetric misreporting by other clients of the same type would result in a reduced reward for all. The penalty coefficients are designed through a comparison of clients’ equilibrium and socially optimal strategies, as seen in complete information scenarios. We also ensure the ex-ante budget balance by numerically designing the compensation payment in Definition 5 as:

[TABLE]

When we fix $\alpha_{L}$ and $\alpha_{H}$, the parameter $\eta$ can be used to measure the divergence of clients’ types $\alpha_{i}$’s. When $\eta=0.5$, the diversity of clients’ type distribution is at its maximum. In our experiments on FL systems, we use a squared-SVM training model based on the original MNIST dataset with a loss function $F_{i}\left(\mathbf{w},\mathbf{x}_{j},y_{j}\right)=\frac{\lambda}{2}\|\mathbf{w}\|^{2}+\frac{1}{2}\max\left\{0;1-y_{j}\mathbf{w}^{\mathrm{T}}\mathbf{x}_{j}\right\}^{2}$ and $\lambda$ is a constant. The training output is a binary model, and each client has 1000 training and 1000 testing data samples in each simulation round. Fig. 2 shows the social costs under our proposed pricing scheme in Definition 5 and the no-pricing benchmark. Both social costs increase as the variance of $\alpha_{i}$’s increases (i.e., $\eta\rightarrow 0.5$), and our pricing scheme more significantly reduces the social cost compared to the benchmark with the maximum , which are consistent with our findings under complete information.

V Conclusion

In this paper, we adopt a game-theoretic perspective to examine clients’ privacy-preserving strategies in federated learning systems. We propose a two-stage game model to capture the interactions between clients and the server, and introduce pricing mechanisms to regulate clients’ selfish behaviors. Our pricing scheme is designed to reach the social optimum without verifying the actual noise-adding level. Even under incomplete information of clients’ private types, our mechanism incentivizes truthful reporting. Our analysis of the ratio of social costs shows that our approach outperforms the unguided one, with simulation results revealing greater improvement as the diversity of clients’ types increases.