Generalization Error Bounds for Noisy, Iterative Algorithms via Maximal Leakage

TL;DR

This paper introduces an information-theoretic approach using maximal leakage to analyze the generalization error of iterative, noisy algorithms like SGLD, providing explicit bounds and insights on noise and update functions.

Contribution

It develops a semi-closed form bound on maximal leakage for noisy iterative algorithms, linking update function properties and noise to generalization performance.

Findings

Bound on maximal leakage with Gaussian noise and bounded update functions

Explicit tight bounds for various scenarios

Insights on optimal noise choice for minimizing leakage

Abstract

We adopt an information-theoretic framework to analyze the generalization behavior of the class of iterative, noisy learning algorithms. This class is particularly suitable for study under information-theoretic metrics as the algorithms are inherently randomized, and it includes commonly used algorithms such as Stochastic Gradient Langevin Dynamics (SGLD). Herein, we use the maximal leakage (equivalently, the Sibson mutual information of order infinity) metric, as it is simple to analyze, and it implies both bounds on the probability of having a large generalization error and on its expected value. We show that, if the update function (e.g., gradient) is bounded in $L_2$-norm and the additive noise is isotropic Gaussian noise, then one can obtain an upper-bound on maximal leakage in semi-closed form. Furthermore, we demonstrate how the assumptions on the update function affect the optimal (in the sense of minimizing the induced maximal leakage) choice of the noise. Finally, we compute explicit tight upper bounds on the induced maximal leakage for other scenarios of interest.

Equations

$$\mathbf{Pr}(|\text{gen-err}(\mathcal{A},S)|\ge \eta )\le 2\exp (-2n{\eta }^2+\mathcal{L}\left(S\to \mathcal{A}(S)\right)),$$

$$I_{\infty }(X,Y)=\log {\mathbb{E}}_{P_Y}\left[\underset{P_X}{ess-sup}\frac{d{P}_{XY}}{d{P}_X{P}_Y}\right]=\mathcal{L}\left(X\to Y\right),$$

$$\mathcal{L}\left(X\to Y\right)=\log \int \underset{P_X}{ess-sup}{f}_{Y|X}(y|x)dy,$$

$$\mathcal{L}\left(X\to Y_1,Y_2\right)\le \mathcal{L}\left(X\to Y_1\right)+\mathcal{L}\left(X\to Y_2|Y_1\right),$$

$$\mathcal{L}\left(X\to Y^n\right)\le \sum _{i=1}^n\mathcal{L}\left(X\to Y_i|Y^{i-1}\right).$$

$$I(X;Y)=I_1(X,Y)\le I_{\alpha }(X,Y)\le I_{\infty }(X,Y)=\mathcal{L}\left(X\to Y\right).$$

$$L_{P_Z}(w)={\mathbb{E}}_P[\ell (w,Z)]$$

$$L_S(w)=\frac{1}{n}\sum _{i=1}^n\ell (w,Z_i).$$

$${\text{gen-err}}_{\mathcal{P}}(\mathcal{A},S)=L_{\mathcal{P}}(\mathcal{A}(S))-L_S(\mathcal{A}(S)).$$

$$\left|\mathbb{E}[{\text{gen-err}}_{\mathcal{P}}(\mathcal{A},S)]\right|\le \sqrt{\frac{2{\sigma }^{2}I(S;\mathcal{A}(S))}{n}}.$$

$$\mathbf{Pr}(|{\text{gen-err}}_P(\mathcal{A},S)|\ge t)\le 2\exp \left(-\frac{\alpha -1}{\alpha }\left(\frac{n{t}^2}{2{\sigma }^2}-I_{\alpha }(S,\mathcal{A}(S))\right)\right),$$

$$\mathbf{Pr}(|{\text{gen-err}}_P(\mathcal{A},S)|\ge t)\le 2\exp \left(-\left(\frac{n{t}^2}{2{\sigma }^2}-\mathcal{L}\left(S\to \mathcal{A}(S)\right)\right)\right).$$

$$W_t=g(W_{t-1})-{\eta }_{t}F(W_{t-1},Z_t)+{\xi }_t,\forall t\ge 1,$$

$$P(Z_{t+1}|Z^t,W^t,S)=P(Z_{t+1}|Z^t,S).$$

$$P(W_{t+1}|W^t,Z^T,S)=P(W_{t+1}|W_t,Z_{t+1}).$$

$$I(S;W)\le \frac{d}{2}\sum _{t=1}^T\log \left(1+\frac{{\eta }_t^2{L}^2}{d{\sigma }_t^2}\right).$$

$$h(d,p,f,r):=\underset{\overline{{\mathcal{B}}_p^d}(0,r)}{\int }\underset{x\in {\mathcal{B}}_p^d(0,r)}{\sup }f(w-x)\mathrm{d}w.$$

$$A_d:=\{w\in {\mathbb{R}}^d:w_i\ge 0,\text{ for all }i\in \{1,2,\dots ,d\}\}.$$

$$h_{+}(d,p,f,r):=\underset{\overline{{\mathcal{B}}_p^d}(0,r)\cap A_d}{\int }\underset{x\in {\mathcal{B}}_p^d(0,r)}{\sup }f(w-x)\mathrm{d}w.$$

$$\mathcal{L}\left(S\to W\right)\le \sum _{t=1}^T\log \left(f_t(0)V_p(d,{\eta }_{t}L)+h(d,p,f_t,{\eta }_{t}L)\right),$$

$$\mathcal{L}\left(S\to W\right)\le \mathcal{L}\left(Z^T\to W^T\right)\le \sum _{t=1}^T\mathcal{L}\left(Z^T\to W_t|W^{t-1}\right)=\sum _{t=1}^T\mathcal{L}\left(Z_t\to W_t|W_{t-1}\right),$$

$$\exp \left\{\mathcal{L}\left(Z_t\to W_t|W_{t-1}=w_{t-1}\right)\right\}$$

$$={\int }_{{\mathbb{R}}^d}\underset{P_{z_t}}{ess-sup}{f}_t\left(w_t-g(w_{t-1})+{\eta }_{t}F(w_{t-1},Z_t)\right)\mathrm{d}{w}_t,$$

$$={\int }_{{\mathbb{R}}^d}\underset{P_{z_t}}{ess-sup}{f}_t\left(w_t+{\eta }_{t}F(w_{t-1},Z_t)\right)\mathrm{d}{w}_t,$$

$$\exp \left\{\mathcal{L}\left(Z_t\to W_t|W_{t-1}=w_{t-1}\right)\right\}$$

$$\le {\int }_{{\mathbb{R}}^d}\underset{x_t\in {\mathcal{B}}_p(0,{\eta }_{t}L)}{\sup }{f}_t\left(w_t+x_t\right)\mathrm{d}{w}_t$$

$$=\underset{{\mathcal{B}}_p(0,{\eta }_{t}L)}{\int }\underset{x_t\in {\mathcal{B}}_p(0,{\eta }_{t}L)}{\sup }{f}_t\left(w_t+x_t\right)\mathrm{d}{w}_t+\underset{\overline{{\mathcal{B}}_p}(0,{\eta }_{t}L)}{\int }\underset{x_t\in {\mathcal{B}}_p(0,{\eta }_{t}L)}{\sup }{f}_t\left(w_t+x_t\right)\mathrm{d}{w}_t$$

$$=f_t(0)V_p(d,{\eta }_{t}L)+\underset{\overline{{\mathcal{B}}_p}(0,{\eta }_{t}L)}{\int }\underset{x_t\in {\mathcal{B}}_p(0,{\eta }_{t}L)}{\sup }{f}_t\left(w_t-x_t\right)\mathrm{d}{w}_t,$$

$$\mathcal{L}\left(S\to W\right)\le \sum _{t=1}^T\log \left(\frac{V_2(d,{\eta }_{t}L)}{(2\pi {\sigma }_t^2{)}^{d/2}}+\frac{1}{\Gamma \left(\frac{d}{2}\right)}\sum _{i=0}^{d-1}\left(\genfrac{}{}{0ex}{}{d-1}{i}\right)\Gamma \left(\frac{i+1}{2}\right){\left(\frac{{\eta }_{t}L}{{\sigma }_t\sqrt{2}}\right)}^{d-1-i}\right),$$

$$\mathcal{L}\left(S\to W\right)$$

$$=\sum _{t=1}^T\log \left(\frac{V_2(d,{\eta }_{t}L)}{(2\pi {\sigma }_t^2{)}^{\frac{d}{2}}}+\underset{\overline{{\mathcal{B}}_2}(0,{\eta }_{t}L)}{\int }\underset{x_t\in {\mathcal{B}}_2(0,{\eta }_{t}L)}{\sup }\frac{1}{(2\pi {\sigma }_t^2{)}^{\frac{d}{2}}}\exp \left\{-\frac{\Vert w_t-x_t{\Vert }_2^2}{2{\sigma }_t^2}\right\}\mathrm{d}{w}_t\right).$$

$$\Vert w_t-x_t\Vert \ge \Vert w_t-\frac{{\eta }_{t}L}{\Vert w_t\Vert }{w}_t\Vert =\Vert w_t\Vert -{\eta }_{t}L.$$

$$h(d,2,f_t,{\eta }_{t}L)$$

Full text

1 Introduction

One of the key challenges in machine learning research concerns the “generalization” behavior of learning algorithms. That is: if a learning algorithm performs well on the training set, what guarantees can one provide on its performance on new samples?

While the question of generalization is understood in many settings (Bousquet et al., 2003; Shalev-Shwartz and Ben-David., 2014), existing bounds and techniques provide vacuous expressions when employed to show the generalization capabilities of deep neural networks (DNNs) (Bartlett et al., 2017, 2019; Jiang et al., 2020; Zhang et al., 2021). In general, classical measures of model expressivity (such as Vapnik-Chervonenkis (VC) dimension (Vapnik and Chervonenkis, 1991), Rademacher complexity (Bartlett and Mendelson, 2003), etc.) fail to explain the generalization abilities of DNNs due to the fact that they are typically over-parameterized models with less training data than model parameters. A novel approach was introduced by (Russo and Zou, 2016), and (Xu and Raginsky, 2017) (further developed by Steinke and Zakynthinou (2020); Bu et al. (2020); Esposito et al. (2021); Esposito and Gastpar (2022) and many others), where information-theoretic techniques are used to link the generalization capabilities of a learning algorithm to information measures. These quantities are algorithm-dependent and can be used to analyze the generalization capabilities of general classes of updates and models e.g., noisy iterative algorithms such as the Stochastic Gradient Langevin Dynamics (SGLD) (Pensia et al., 2018; Wang et al., 2021), which can thus be applied to deep learning settings. Moreover, it has been shown that information-theoretic bounds can be non-vacuous and reflect the real generalization behavior even in deep learning settings (Dziugaite and Roy, 2017; Zhou et al., 2018; Negrea et al., 2019; Haghifam et al., 2020).

In this work we adopt and expand the framework introduced by Pensia et al. (2018), but instead of focusing on the mutual information between the input and output of an iterative algorithm, we compute the maximal leakage (Issa et al., 2020). Maximal leakage, together with other information measures of the Sibson/Rényi family (maximal leakage can be shown to be Sibson Mutual information of order infinity (Issa et al., 2020)), have been linked to high-probability bounds on the generalization error (Esposito et al., 2021). In particular, given a learning algorithm $\mathcal{A}$ trained on data-set $S$ (made of $n$ samples), one can provide the following guarantee in the case of the $0-1$ loss:

[TABLE]

where $\mathcal{L}\left(S\!\!\to\!\!\mathcal{A}(S)\right)$ is defined in equation (2) below. This deviates from much of the literature in which the focus is on bounding the expected generalization error instead (Xu and Raginsky, 2017; Steinke and Zakynthinou, 2020). Consequently, if one can guarantee that for a class of algorithms, the maximal leakage between the input and the output is bounded, then one can provide an exponentially decaying (in the number of samples $n$) bound on the probability of having a large generalization error. This is in general not true for mutual information, which can typically only guarantee a linearly decaying bound on the probability of the same event (Bassily et al., 2018). Moreover, a bound on maximal leakage implies a bound on mutual information (cf. Equation 6) and, consequently, a bound on the expected generalization error of $\mathcal{A}$ (exploiting the link between mutual information and expected generalization error (Xu and Raginsky, 2017)). The main advantage of maximal leakage lies in the fact that it depends on the distribution of the samples only through its support. It is thus naturally independent from the distribution over the samples and particularly amenable to analysis, especially in additive noise settings.

The contributions of this work can be summarized as follows:

• •

we derive novel bounds on $\mathcal{L}\left(S\!\!\to\!\!\mathcal{A}(S)\right)$ whenever $\mathcal{A}$ is a noisy, iterative algorithm (SGLD-like), which then implies the first bounds showing generalization with high-probability of said mechanisms;

• •

we leverage the analysis to extrapolate to optimize the type of noise to be added (in the sense of minimizing the induced maximal leakage), based on the assumptions imposed on the algorithm. In particular, if one assumes the $L_{\infty}$ norm of the gradient to be bounded, then adding uniform noise minimizes the maximal leakage upper bound. Hence, the analysis and computation of maximal leakage can also be used to inform the design of novel noisy, iterative algorithms.

1.1 Related Work

The line of work exploiting information measures to bound the expected generalization started in (Russo and Zou, 2016; Xu and Raginsky, 2017) and was then refined with a variety of approaches considering Conditional Mutual Information (Steinke and Zakynthinou, 2020; Haghifam et al., 2020), the Mutual Information between individual samples and the hypothesis (Bu et al., 2019) or improved versions of the original bounds (Issa et al., 2019; Hafez-Kolahi et al., 2020). Other approaches employed the Kullback-Leibler Divergence with a PAC-Bayesian approach (McAllester, 2013; Zhou et al., 2018). Moreover, said bounds were then characterized for specific SGLD-like algorithms, denoted as “noisy, iterative algorithms” and used to provide novel, non-vacuous bounds for Neural Networks (Pensia et al., 2018; Negrea et al., 2019; Haghifam et al., 2020; Wang et al., 2023) as well as for SGD algorithms (Neu et al., 2021). Recent efforts tried to provide the optimal type of noise to add in said algorithms and reduce the (empirical) gap in performance between SGLD and SGD (Wang et al., 2021). All of these approaches considered the KL-Divergence or (variants of) Shannon’s Mutual Information. General bounds on the expected generalization error leveraging arbitrary divergences were given in (Esposito and Gastpar, 2022; Lugosi and Neu, 2022). Another line of work considered instead bounds on the probability of having a large generalization error (Bassily et al., 2018; Esposito et al., 2021; Hellström and Durisi, 2020) and focused on large families of divergences and generalizations of the Mutual Information (in particular of the Sibson/Rényi-family, including conditional versions).

2 Preliminaries, Setup, and a General Bound

2.1 Preliminaries

2.1.1 Information Measures

The main building block of the information measures considered in this work is the Rényi’s $\alpha$-divergence between two measures $P$ and $Q$, $D_{\alpha}(P\|Q)$ (which can be seen as a parametrized generalization of the Kullback Leibler-divergence) (van Erven and Harremoës, 2014, Definition 2). Starting from Rényi’s Divergence and the geometric averaging that it involves, Sibson built the notion of Information Radius (Sibson, 1969) which can be seen as a special case of the following quantity (Verdú, 2015): $I_{\alpha}(X,Y)=\min_{Q_{Y}}D_{\alpha}(P_{XY}\|P_{X}Q_{Y}).$ Sibson’s $I_{\alpha}(X,Y)$ represents a generalization of Shannon’s mutual information, indeed one has that: $\lim_{\alpha\to 1}I_{\alpha}(X,Y)=I(X;Y)=\mathbb{E}_{P_{XY}}\left[\log\left(\frac{dP_{XY}}{dP_{X}P_{Y}}\right)\right].$ Differently, when $\alpha\to\infty$, one gets:

[TABLE]

where $\mathcal{L}\left(X\!\!\to\!\!Y\right)$ denotes the maximal leakage from $X$ to $Y$, a recently defined information measure with an operational meaning in the context of privacy and security (Issa et al., 2020). Maximal leakage represents the main quantity of interest for the scope of this paper, as it is amenable to analysis and has been used to bound the generalization error (Esposito et al., 2021). As such, we will bound the maximal leakage between the input and output of generic noisy iterative algorithms.

To that end, we mention a few useful properties of $\mathcal{L}\left(X\!\!\to\!\!Y\right)$. If $X$ and $Y$ are jointly continuous random variables, then (Issa et al., 2020, Corollary 4)

[TABLE]

where $f_{Y|X}$ is the conditional pdf of $Y$ given $X$. Moreover, maximal leakage satisfies the following chain rule (the proof of which is given in Appendix A):

Lemma 2.1**.**

Given a triple of random variables $(X,Y_{1},Y_{2})$, then

[TABLE]

where the conditional maximal leakage $\mathcal{L}\left(X\!\!\to\!\!Y_{2}|Y_{1}\right)=\operatornamewithlimits{ess-sup}_{P_{Y_{1}}}\mathcal{L}\left(X\!\!\to\!\!Y_{2}|Y_{1}=y_{1}\right)$, where the latter term is interpreted as the maximal leakage from $X$ to $Y_{2}$ with respect to the distribution $P_{XY_{2}|Y_{1}=y_{1}}$. Consequently, for random variables $(X,(Y_{i})_{i=1}^{n})$,

[TABLE]

Moreover, one can relate $\mathcal{L}\left(X\!\!\to\!\!Y\right)$ to $I(X;Y)$ through $I_{\alpha}$. Indeed, an important property of $I_{\alpha}$ is that it is non-decreasing in $\alpha$, hence for every $\infty>\alpha>1$:

[TABLE]

For more details on Sibson’s $\alpha$-MI we refer the reader to (Verdú, 2015), as for maximal leakage the reader is referred to (Issa et al., 2020).

2.1.2 Learning Setting

Let $\mathcal{Z}$ be the sample space, $\mathcal{W}$ be the hypothesis space, and $\ell:\mathcal{W}\times\mathcal{Z}\rightarrow\mathbb{R}_{+}$ be a loss function. Say $\mathcal{W}\subseteq\mathbb{R}^{d}$. Let $S=(Z_{1},Z_{2},\ldots,Z_{n})$ consist of $n$ i.i.d samples, where $Z_{i}\sim P$, with $P$ unknown. A learning algorithm $\mathcal{A}$ is a mapping $\mathcal{A}:\mathcal{Z}^{n}\to\mathcal{W}$ that given a sample $S$ provides a hypothesis $W=\mathcal{A}(S)$. $\mathcal{A}$ can be either a deterministic or a randomized mapping and undertaking a probabilistic (and information-theoretic) approach one can then equivalently consider $\mathcal{A}$ as a family of conditional probability distributions $P_{W|S=s}$ for $s\in\mathcal{Z}^{n}$ i.e., an information channel. Given a hypothesis $w\in\mathcal{W}$ the true risk of $w$ is denoted as follows:

[TABLE]

while the empirical risk of $w$ on $S$ is denoted as follows:

[TABLE]

Given a learning algorithm $\mathcal{A}$, one can then define its generalization error as follows:

[TABLE]

Since both $S$ and $\mathcal{A}$ can be random, $\text{gen-err}_{\mathcal{P}}(\mathcal{A},S)$ is a random variable and one can then study its expected value or its behavior in probability. Bounds on the expected value of the generalization error in terms of information measures are given in Xu and Raginsky (2017); Issa et al. (2019); Bu et al. (2019); Steinke and Zakynthinou (2020) stating different variants of the following bound (Xu and Raginsky, 2017, Theorem 1): if $\ell(w,Z)$ is $\sigma^{2}$-sub-Gaussian111A [math]-mean random variable $X$ is said to be $\sigma^{2}$-sub-Gaussian if $\log\mathbb{E}[\exp(\lambda X)]\leq\sigma^{2}\lambda^{2}/2$ for every $\lambda\in\mathbb{R}$. then

[TABLE]

Thus, if one can prove that the mutual information between the input and output of a learning algorithm $\mathcal{A}$ trained on $S$ is bounded (ideally, growing less than linearly in $n$) then the expected generalization error of $\mathcal{A}$ will vanish with the number of samples. Alternatively, Esposito et al. (2021) demonstrate high-probability bounds, involving different families of information measures. One such bound, which is relevant to the scope of this paper is the following (Esposito et al., 2021, Corollary 2): assume $\ell(w,Z)$ is $\sigma^{2}$-sub-Gaussian and let $\alpha>1$, then

[TABLE]

taking the limit of $\alpha\to\infty$ in (11) leads to the following (Esposito et al., 2021, Corollary 4):

[TABLE]

Thus, in this case, if one can prove that the maximal leakage between the input and output of a learning algorithm $\mathcal{A}$ trained on $S$ is bounded, then the probability of the generalization error of $\mathcal{A}$ being larger than any constant $t$ will decay exponentially fast in the number of samples $n$.

2.2 Problem Setup

We consider iterative algorithms, where each update is of the following form:

[TABLE]

where $Z_{t}\subseteq S$ (sampled according to some distribution), $g:\mathbb{R}^{d}\rightarrow\mathbb{R}^{d}$ is a deterministic function, $F(W_{t-1},Z_{t})$ computes a direction (e.g., gradient), $\eta_{t}$ is the step-size, and $\xi_{t}=(\xi_{t1},\ldots,\xi_{td})$ is random noise. We will assume for the remainder of this paper that $\xi_{t}$ has an absolutely continuous distribution. Let $T$ denote the total number of iterations, $W^{t}=(W_{1},W_{2},\ldots W_{t})$, and $Z^{t}=(Z_{1},Z_{2},\ldots,Z_{t})$. The algorithms under consideration further satisfy the following two assumptions

• •

Assumption 1 (Sampling): The sampling strategy is agnostic to parameter vectors:

[TABLE]

• •

Assumption 2 ($\mathbf{L_{p}}$-Boundedness): For some $p>0$ and $L>0$,  $\sup_{w,z}\|F(w,z)\|_{p}\leq L$.

As a consequence of the first assumption and the structure of the iterates, we get:

[TABLE]

The above setup was proposed by Pensia et al. (2018), who specifically studied the case $p=2$. Denoting by $W$ the final output of the algorithm (some function of $W^{T}$), they show that

Theorem 2.2** ((Pensia et al., 2018, Theorem 1)).**

If the boundedness assumption holds for $p=2$ and $\xi_{t}\sim\mathcal{N}(0,\sigma^{2}_{t}I_{d})$, then

[TABLE]

By virtue of inequality (10), this yields a bound on the expected generalization error.

In this work, we derive bounds on the maximal leakage between $\mathcal{L}\left(S\!\!\to\!\!W\right)$ for iterative noisy algorithms, which leads to high-probability bounds on the generalization error (cf. equation (12)). We consider different scenarios in which $F$ is bounded in $L_{1}$, $L_{2}$, or $L_{\infty}$ norm, and the added noise is Laplace, Gaussian, or Uniform. It is worth noting that the bounds we derive depend on $F$ only through the boundedness assumption (Assumption 2 above). Considering $F$ to be a gradient yields the most (practically) interesting scenario in which our results hold, as it represents a widely used family of learning algorithms. However, we do not leverage any structure that is particular to gradients (beyond the boundedness assumption).

2.3 Notation

Given $d\in\mathbb{N}$, $w\in\mathbb{R}^{d}$, and $r>0$, let $\mathcal{B}_{p}^{d}(w,r)=\{x\in\mathbb{R}^{d}:\|x-w\|_{p}\leq r\}$ denote the $L_{p}$-ball of radius $r$ and center $w$, and let $V_{p}(d,r)$ denote its corresponding volume. When the dimension $d$ is clear from the context, we may drop the superscript and write $\mathcal{B}_{p}(w,r)$. Given a set $S$, we denote its complement by $\overline{S}$. The $i$-th component of $w_{t}$ will be denoted by $w_{ti}$.

We denote the pdf of the noise $\xi_{t}$ by $f_{t}:\mathbb{R}^{d}\rightarrow\mathbb{R}$. The following functional will be useful for our study: given $d\in\mathbb{N}$, $p>0$, a pdf $f:\mathbb{R}^{d}\rightarrow\mathbb{R}$, and an $r\geq 0$, define

[TABLE]

We denote the “positive octant” by $A_{d}$, i.e.,

[TABLE]

Since we will mainly consider pdfs that are symmetric (Gaussian, Laplace, uniform), the $h$ functional “restricted” to $A_{d}$ will be useful:

[TABLE]

2.4 General Bound

Proposition 2.3**.**

Suppose $f_{t}:\mathbb{R}^{d}\rightarrow\mathbb{R}$ is maximized for $x=0$. If Assumptions 1 and 2 hold for some $p>0$, then

[TABLE]

where $h$ is defined in equation (17).

The above bound is appealing as it implicitly poses an optimization problem: given a constraint on the noise pdf $f_{t}$ (say, a bounded variance), one may choose $f_{t}$ as to minimize the upper bound in equation (20). Moreover, despite its generality, we show that it is tight in several interesting cases, including when $p=2$ and $f_{t}$ is the Gaussian pdf.

In the next section, we consider several scenarios for different values of $p$ and different noise distributions. As a testament to the tractability of maximal leakage, we derive exact semi-closed form expressions for the bound of Proposition 2.3. Finally, it is worth noting that the form of the bound allows us to choose different noise distributions at different time steps, but these examples are outside the scope of this paper.

Proof 2.4**.**

We proceed as in the work of Pensia et al. (2018):

[TABLE]

where the first inequality follows from Lemma 2 of Pensia et al. (2018) and the data processing inequality for maximal leakage (Issa et al., 2020, Lemma 1), the second inequality follows Lemma 2.1, and the equality follows from (15). Now,

[TABLE]

where the last equality follows from a change of a variable ${w_{t}}\leftarrow w_{t}-g(w_{t-1})$. Finally, since $\eta_{t}F(w_{t-1},z_{t})\in\mathcal{B}_{p}(0,\eta_{t}L)$ by assumption, we can further upper-bound the above by:

[TABLE]

where the last equality follows from the assumptions on $f_{t}$.

3 Boundedness in $L_{2}$-Norm

Considering the case where $F$ computes a gradient, then boundedness in $L_{2}$-norm is a common assumption. It is commonly enforced, for instance, using gradient clipping (Abadi et al., 2016a, b; Chen et al., 2020).

Theorem 3.1**.**

If the boundedness assumption holds for $p\leq 2$ and $\xi_{t}\sim\mathcal{N}(0,\sigma^{2}_{t}I_{d})$, then

[TABLE]

*where $\displaystyle V_{2}(d,r)=\frac{\pi^{d/2}}{\Gamma\left(\frac{d}{2}+1\right)}r^{d}$. *

Note that even if the parameter $L$ is large (e.g., Lipschitz constant of a neural network (Negrea et al., 2019)), it appears in (29) normalized by $\Gamma(d/2)$ so its effect is significantly dampened (as $d$ is also typically very large).

Finally, note that the bound in Proposition 2.3 is increasing in $p$: this can be seen from line (26), where the supremum over $\mathcal{B}_{p}$ can be further upper-bounded by a supremum over $\mathcal{B}_{p^{\prime}}$ for $p^{\prime}>p$. Therefore for $q\leq p$, the bound induced by Proposition 2.3 is smaller. The bound in Theorem 3.1 corresponds to $p=2$ and goes to 0 (as $d$ grows), hence the bound induced by Proposition 2.3 goes to 0 for all $q\leq p=2$.

Proof 3.2**.**

The conditions of Proposition 2.3 are satisfied, thus it is sufficient to prove the bound for $p=2$ (cf. discussion above):

[TABLE]

Hence, it remains to show that the second term inside the $\log$ matches that of equation (29). To that end, note that the point in $\mathcal{B}_{2}(0,\eta_{t}L)$ that minimizes the distance to $w_{t}$ is given $\frac{\eta_{t}L}{\|w_{t}\|}w_{t}$. So we get

[TABLE]

Then,

[TABLE]

To evaluate this integral, we use spherical coordinates (details in Appendix B). Then,

[TABLE]

Combining equations (31) and (35) yields (29).

Remark 3.3**.**

One could also derive a semi-closed form bound for the case in which the added noise is uniform.

4 Boundedness in $L_{\infty}$-Norm

The bound in Proposition 2.3 makes minimal assumptions about the pdf $f_{t}$. In many practical scenarios we have more structure we could leverage. In particular, we make the following standard assumptions in this section:

• •

$\xi_{t}$ is composed of i.i.d components. Let $f_{t0}$ be the pdf of a component, then $\displaystyle f_{t}(x_{t})\ =\prod_{i=1}^{d}f_{t0}(x_{ti})$.

• •

$f_{t0}$ is symmetric around 0 and non-increasing over $[0,\infty).$

In this setting, Proposition 2.3 reduces to a very simple form for $p=\infty$:

Theorem 4.1**.**

Suppose $f_{t}$ satisfies the above assumptions. If Assumptions 1 and 2 hold for $p=\infty$, then

[TABLE]

Note that the bounded-$L_{\infty}$ assumption is weaker than the bounded $L_{2}$-norm assumption. Moreover, the assumption of having a bounded $L_{\infty}$-norm is satisfied in Pichapati et al. (2019) where the authors clipped the gradient in terms of the $L_{\infty}$-norm, thus “enforcing” the assumption. On the other hand, the theorem has an intriguing form as, under standard assumptions, the bound depends on $f_{t0}$ only through $f_{t0}(0)$. This naturally leads to an optimization problem: given a certain constraint on the noise, which distribution $f^{\star}$ minimizes $f(0)$? The following theorem shows that, if the noise is required to have a bounded variance, then $f^{\star}$ corresponds to the uniform distribution:

Theorem 4.2**.**

Let $\mathcal{F}$ be the family of probability densities (over $\mathbb{R}$) satisfying for each $f\in\mathcal{F}$:

$f$* is symmetric around 0.* 2. 2.

$f$* is non-increasing over $[0,\infty)$.* 3. 3.

${\rm\mathbf{E}}_{f}[X^{2}]\leq\sigma^{2}$.

Then, the distribution minimizing $f(0)$ over $\mathcal{F}$ is the uniform distribution $\mathcal{U}(-\sigma\sqrt{3},\sigma\sqrt{3})$.

That is, uniform noise is optimal in the sense that it minimizes the upper bound in Theorem 4.1 under bounded variance constraints. The proof of Theorem 4.2 is deferred to Appendix D.

4.1 Proof of Theorem 4.1

Since the assumptions of Proposition 2.3 hold, then

[TABLE]

It remains to show that $h(d,\infty,f_{t},\eta_{t}L)$ (i.e., the second term inside the $\log$ in Equation 17) is equal to $(1+2\eta_{t}Lf_{t0}(0))^{d}-(2\eta_{t}Lf_{t0}(0))^{d}$. We will derive a recurrence relation for $h$ in terms of $d$. To simplify the notation, we drop the subscript $t$ and ignore the dependence of $h$ on $p=\infty$, $f_{t}$, and $\eta_{t}L$, so that we simply write $h(d)$ (and correspondingly, $h_{+}(d)$, cf. Equation 19).

By symmetry, $h(d)=2^{d}h_{+}(d)$. Letting $w^{d-1}:=(w_{1},\ldots,w_{d-1})$, we will decompose the integral over $\overline{\mathcal{B}_{\infty}^{d}}(0,\eta_{t}L)$ into two disjoint subsets: 1) $w^{d-1}\notin\mathcal{B}_{\infty}^{d-1}(0,\eta_{t}L)$, in which case $w_{d}$ can take any value in $\mathbb{R}$, and 2) $w^{d-1}\in\mathcal{B}_{\infty}^{d-1}(0,\eta_{t}L)$, in which case $w_{d}$ must satisfy $|w_{d}|>\eta_{t}L$.

[TABLE]

The innermost integral of line (40) is independent of $w^{d-1}$ so that the outer integral is equal to $h_{+}(d-1)$. Similarly, the innermost integral of line (39) is independent of $w^{d-1}$, and the supremum in the outer integral yields $f(0)$ for every $i$. Hence, we get

[TABLE]

the detailed proof of which is deferred to Appendix C. Finally, it is straightforward to check that $h(1)=1$, hence $h(d)=(1+2\eta_{t}Lf(0))^{d}-(2\eta_{t}Lf(0))^{d}$.

5 Boundedness in $L_{1}$-Norm

In this section, we consider the setting where Assumption 2 holds for $p=1$. By Proposition 2.3, any bound derived for $p=2$ holds for $p=1$ as well; in particular, Theorem 3.1 applies. Nevertheless, it is possible to compute a semi-closed form directly for $p=1$ (cf. Theorem 5.3 below).

We also consider the case in which the additive noise is Laplace, i.e., “matching” the $L_{1}$ constraint on the update function. Interestingly, we show that in this case the limit of maximal leakage, as $d$ goes to infinity, is finite.

5.1 Bound for Laplace noise

We say $X$ has a Laplace distribution, denoted by $X\sim\mathrm{Lap}(\mu,1/\lambda)$, if its pdf is given by $f(x)=\frac{\lambda}{2}e^{-\lambda|x-\mu|}$ for $x\in\mathbb{R}$, for some $\mu\in\mathbb{R}$ and $\lambda>0$. The corresponding variance is given by $2/\lambda^{2}$.

Theorem 5.1**.**

If the boundedness assumptions holds for $p=1$ and $\xi_{t}$ is composed of i.i.d components, each of which is $\sim\mathrm{Lap}(0,\frac{\sqrt{2}}{\sigma_{t}})$, then

[TABLE]

where $\displaystyle V_{1}(d,r)=\frac{(2r)^{d}}{d!}$. Consequently, for fixed $T$,

[TABLE]

Proof 5.2**.**

We give a high-level description of the proof (as similar techniques have been used in proofs of earlier theorems) and defer the details to Appendix E. Since the multivariate Laplace distribution (for i.i.d variables) depends on the $L_{1}$-norm of the corresponding vector of variables, we need to solve the following problem: given $R>0$ and $w\notin\mathcal{B}_{1}(0,R)$, compute

[TABLE]

The closest element in $\mathcal{B}_{1}(0,R)$ will lie on the hyperplane defining $\mathcal{B}_{1}$ that is in the same octant as $w$, so the problem reduces to projecting a point on a hyperplane in $L_{1}$-distance (the proof in the appendix does not follow this argument but arrives at the same conclusion). Then, we need to compute $h(d,1,f_{t},\eta_{t}L)$. We use a similar approach as in the proof of Theorem 4.1, that is, we split the integral and derive a recurrence relation.

5.2 Bound for Gaussian noise

Finally, we derive a bound on the induced leakage when the added noise is Gaussian:

Theorem 5.3**.**

If the boundedness assumptions holds for $p=1$ and $\xi_{t}\sim\mathcal{N}(0,\sigma^{2}_{t}I_{d})$, then

[TABLE]

In order to prove Theorem 5.3 one has to solve a problem similar to the one introduced in Theorem 5.1 (cf. equation (44)). However, in this case a different norm is involved: i.e., given $R>0$ and $w\notin\mathcal{B}_{1}(0,R)$, one has to compute

[TABLE]

Again, one can argue that the point achieving the infimum lies on the hyperplane defining $\mathcal{B}_{1}$ that is in the same octant as $w$. In other words, the minimizer $x^{\star}$ is such that the sign of each component is the same sign as the corresponding component of $w$ (and lies on the boundary of $\mathcal{B}_{1}$). Thus, we are projecting a point on the corresponding face of the $L_{1}$-ball. The length of the projection is then appropriately lower-bounded and the induced integral is solved by an opportune choice of change of variables. The details of the proof are given in Appendix F.

Acknowledgment

The work in this manuscript was supported in part by the Swiss National Science Foundation under Grant 200364 and by the University Research Board at the American University of Beirut (Beirut, Lebanon).

Appendix A Proof of Lemma 2.1

Recall the definition of maximal leakage and conditional maximal leakage:

Definition A.1** (Maximal Leakage (Issa et al., 2020, Definition 1)).**

Given two random variables $(X,Y)$ with joint distribution $P_{XY}$,

[TABLE]

where $U$ takes values in a finite, but arbitrary, alphabet, and $\hat{U}(Y)$ is the optimal estimator (i.e., MAP) of $U$ given $Y$.

Similarly,

Definition A.2** (Conditional Maximal Leakage (Issa et al., 2020, Definition 6)).**

Given three random variables $(X,Y,Z)$ with joint distribution $P_{XYZ}$,

[TABLE]

where $U$ takes values in a finite, but arbitrary, alphabet, and $\hat{U}(Y,Z)$ and $\hat{U}(Z)$ are the optimal estimators (i.e., MAP) of $U$ given $(Y,Z)$ and $U$ given $Z$, respectively.

It then follows that

[TABLE]

where the last inequality follows from the fact that $U-X-(Y_{1},Y_{2})$ implies $U-X-Y_{2}|Y_{1}$.

The fact that

[TABLE]

has been shown for discrete alphabets in Theorem 6 of (Issa et al., 2020). The extension to continuous alphabets is similar (with integrals replacing sums, and pdfs replacing pmfs, where appropriate).

Finally, it remains to show equation (5). We proceed by induction. The case $n=2$ has already been shown above. Assume the inequality is true up to $n-1$ variables, then

[TABLE]

where the second inequality follows from the induction hypothesis.

Appendix B Proof of equation (35)

To evaluate the integral in line (34), we write it in spherical coordinates:

[TABLE]

Now, note that for any $n\in\mathbb{N}$, $\displaystyle\int_{0}^{\pi}\sin^{n}(x)\mathrm{d}x=2\int_{0}^{\pi/2}\sin^{n}(x)\mathrm{d}x$, and

[TABLE]

where (a) follows from the change of variable $u=\sin x$, (b) follows from the change of variable $t=u^{2}$, (c) follows from the definition of the Beta function: $\displaystyle\mathrm{Beta}(s_{1},s_{2})=\int_{0}^{1}t^{s_{1}-1}(1-t)^{s_{2}-1}$, and the last equality is a known property of the Beta function ($\Gamma(1/2)=\sqrt{\pi}$). Consequently,

[TABLE]

To evaluate the innermost integral, the following identity will be useful:

[TABLE]

where the first equality follows from the change of variable $t=x^{2}$. Then,

[TABLE]

where (a) follows from the change of variable $t=\rho/(\sigma\sqrt{2})$, and (b) follows from (61).

Finally, combining equations (58), (60), and (65), we get

[TABLE]

Appendix C Proof of equation (41)

The innermost integral of line (40) evaluates to

[TABLE]

where the first equality follows from the monotonicity assumptions, the second from a change of variable, and the third from the symmetry assumption. Similarly, the innermost integral of line (39) evaluates to

[TABLE]

Combining equations (40), (68), and (71), we get

[TABLE]

where the second equality follows from the fact that $f$ is maximized at 0, and ${\mathcal{B}^{d-1}_{\infty}}(0,\eta_{t}L)\cap A_{d-1}$ is a $(d-1)$-dimensional hypercube of side $\eta_{t}L$ (with volume $(\eta_{t}L)^{d-1}$). Now,

[TABLE]

Appendix D Proof of Theorem 4.2

Consider any $f\in\mathcal{F}$, and let

[TABLE]

Then

[TABLE]

where the second equality follows from the symmetry assumption. Note that $2f_{+}$ is a valid probability density over $[0,\infty)$, and let $X_{+}\sim f_{+}$. Then, by previous equation,

[TABLE]

Hence,

[TABLE]

which is achieved by the uniform distribution $\mathcal{U}(-\sigma\sqrt{3},\sigma\sqrt{3})$. $\blacksquare$

Appendix E Proof of Theorem 5.1

First, we show that the limit of the right-hand side of equation (42) is given by the right-hand side of equation (43). Note that

[TABLE]

On the other hand,

[TABLE]

Since $T$ is finite, the limit and the sum are interchangeable, so that the above two equations yield the desired limit.

We now turn to the proof of inequality (42). For notational convenience, set $\lambda_{t}=\frac{\sigma_{t}}{\sqrt{2}}$ (so that $f_{t0}(x)=\frac{\lambda_{t}}{2}e^{-\lambda|x|}$ for all $x\in\mathbb{R}$) and $R_{t}=\eta_{t}L$. Since the noise satisfies the assumptions of Proposition 2.3, we get

[TABLE]

Recall $h(d,p,f_{t},R_{t})$ (cf. equation (17)) is defined to be the second term inside the $\log$. Similarly to the strategy adopted in the proof of Theorem 4.1, we will derive a recurrence relation for $h$ in terms of $d$, as such we will again suppress the dependence on $p$, $f_{t}$, and $R_{t}$ in the notation, and write $h(d)$ only (and correspondingly $h_{+}(d)$).

Lemma E.1**.**

Given $w\in\overline{\mathcal{B}_{1}^{d}}(0,R)\cap A_{d}$ ($A_{d}$ defined in equation (18)),

[TABLE]

Proof E.2**.**

Since we are minimizing a continuous function over a compact set, then the infimum can be replaced with a minimum.

Claim:* There exists a minimizer $x^{\star}$ such that for all $i$, $x^{\star}_{i}\leq w_{i}$.*

Proof of Claim:* Consider any $x\in\mathcal{B}_{1}(0,R)$ such that there exists $j$ satisfying $x_{j}>w_{j}$. Note that $w_{j}\geq 0$ by assumption. Now define $x^{\prime}=(x_{1},\ldots,x_{j-1},w_{j},x_{j+1},\ldots,x_{d})$. Then $\|x^{\prime}\|_{1}<\|x\|_{1}$ so that $x^{\prime}\in\mathcal{B}_{1}(0,R)$. Moreover, $\|w-x^{\prime}\|_{1}\leq\|w-x\|_{1}$ as desired. $\blacksquare$*

Now,

[TABLE]

Given the above lemma, we will derive the recurrence relation by decomposing the integral over $\overline{\mathcal{B}_{1}^{d}}(0,R_{t})$ into two disjoint subsets: 1) $w^{d-1}\notin\mathcal{B}_{1}^{d-1}(0,R_{t})$, in which case $w_{d}$ can take any value in $\mathbb{R}$, and 2) $w^{d-1}\in\mathcal{B}_{1}^{d-1}(0,R_{t})$, in which case $w_{d}$ must satisfy $|w_{d}|>R_{t}-\|w^{d-1}\|_{1}$.

[TABLE]

Hence,

[TABLE]

It is easy check that $h(1)=1$, and hence

[TABLE]

satisfies the base case and the recurrence relation. Re-substituting $\eta_{t}L$ and $\sigma_{t}/\sqrt{2}$ for $R_{t}$ and $\lambda_{t}$, respectively, yields the desired result in equation (42).

Appendix F Proof of Theorem 5.3

Let $R_{t}=\eta_{t}L$. Since the noise satisfies the assumptions of Proposition 2.3, we get

[TABLE]

Consider

[TABLE]

First we solve $\displaystyle\inf_{x_{t}\in\mathcal{B}_{1}(0,R_{t})}\|w_{t}-x_{t}\|_{2}$. If $w_{t}\in A_{d}$, then the infimum is achieved for $x^{\star}_{t}\in A_{d}$ as well (one can simply flip the sign of any negative component, which cannot increase the distance). In the subspace $A_{d}$, the boundary of the $L_{1}$ ball is defined by the hyperplane $\sum_{i=1}^{d}x_{ti}=R_{t}$. As such, finding the minimum distance corresponds to projecting the point $w$ to the given hyperplane:

[TABLE]

Now,

[TABLE]

For notational convenience, we drop the $t$ subscript in the following. We perform a change of variable as follows: $\tilde{w}_{d}=\sum_{i=1}^{d}w_{i}$. Hence, for $w\notin\mathcal{B}_{1}(0,R)$, $\tilde{w}_{d}\geq R$. Since $w_{d}\geq 0$, then $\sum_{i=1}^{d-1}w_{i}\leq\tilde{w}_{d}$. For $x\in\mathbb{R}$, define $S(x):=\{w^{d-1}\in\mathbb{R}^{d-1}:\sum_{i=1}^{d-1}w_{i}\leq x\}$. Then,

[TABLE]

where (a) follows from the fact that the innermost integral corresponds to the volume of a scaled probability simplex (scaled by $\tilde{w}_{d}$), and (b) follows from the same computations as in Equations 62 to 65 (with $\tilde{\sigma}=\sigma\sqrt{d}$). Noting that $h(d)=2^{d}h_{+}(d)$ yields the desired the term in Equation 45.