Mitigating Information Leakage in Image Representations: A Maximum Entropy Approach

TL;DR

This paper introduces a maximum entropy adversarial approach to learn image representations that balance task performance with privacy by minimizing information leakage about sensitive attributes.

Contribution

It formulates a novel non-zero sum game for privacy-preserving image representation learning and analyzes its stability and convergence properties.

Findings

Effective reduction of sensitive information leakage.

High task accuracy maintained in learned representations.

Outperforms likelihood-based methods in experiments.

Abstract

Image recognition systems have demonstrated tremendous progress over the past few decades thanks, in part, to our ability of learning compact and robust representations of images. As we witness the wide spread adoption of these systems, it is imperative to consider the problem of unintended leakage of information from an image representation, which might compromise the privacy of the data owner. This paper investigates the problem of learning an image representation that minimizes such leakage of user information. We formulate the problem as an adversarial non-zero sum game of finding a good embedding function with two competing goals: to retain as much task dependent discriminative image information as possible, while simultaneously minimizing the amount of information, as measured by entropy, about other sensitive attributes of the user. We analyze the stability and convergence dynamics of the proposed formulation using tools from non-linear systems theory and compare to that of the corresponding adversarial zero-sum game formulation that optimizes likelihood as a measure of information content. Numerical experiments on UCI, Extended Yale B, CIFAR-10 and CIFAR-100 datasets indicate that our proposed approach is able to learn image representations that exhibit high task performance while mitigating leakage of predefined sensitive information.

Tables

Table 1. Table 1 : Illumination Invariant Face Classification (%)

Method	$s$ (lighting)	$t$ (identity)
LR	96	78
NN + MMD [13]	-	82
VFAE [14]	57	85
ML-ARL [18]	57	89
Maxent-ARL	40	89

Table 2. Table 2 : Main classes and Superclasses in CIFAR-100

Superclass	Main Class
aquatic mammals	beaver, dolphin, otter, seal, whale
fish	aquarium fish, flatfish, ray, shark, trout
flowers	orchids, poppies, roses, sunflowers, tulips
food containers	bottles, bowls, cans, cups, plates
fruit and vegetables	apples, mushrooms, oranges, pears, sweet peppers
household electrical devices	clock, computer keyboard, lamp, telephone, television
household furniture	bed, chair, couch, table, wardrobe
insects	bee, beetle, butterfly, caterpillar, cockroach
large carnivores	bear, leopard, lion, tiger, wolf
large man-made outdoor things	bridge, castle, house, road, skyscraper
large natural outdoor scenes	cloud, forest, mountain, plain, sea
large omnivores and herbivores	camel, cattle, chimpanzee, elephant, kangaroo
medium-sized mammals	fox, porcupine, possum, raccoon, skunk
non-insect invertebrates	crab, lobster, snail, spider, worm
people	baby, boy, girl, man, woman
reptiles	crocodile, dinosaur, lizard, snake, turtle
small mammals	hamster, mouse, rabbit, shrew, squirrel
trees	maple, oak, palm, pine, willow
vehicles 1	bicycle, bus, motorcycle, pickup truck, train
vehicles 2	lawn-mower, rocket, streetcar, tank, tractor

Table 3. (a) No Privacy

Target Accuracy (%)	97.75	97.73	97.68
Adversary Accuracy (%)	23.44	23.09	22.68

Table 4. (a) No Privacy

Target Accuracy (%)	97.75	97.73	97.68
Adversary Accuracy (%)	23.44	23.09	22.68

Table 5. (b) ML-ARL

Target Accuracy (%)	97.52	97.44	97.35	91.52	91.15	60.00
Adversary Accuracy (%)	20.83	20.77	20.64	19.68	14.27	10.00

Table 6. (c) MaxEnt-ARL

Target Accuracy (%)	97.78	97.74	97.53	96.79	95.01	92.34	61.17
Adversary Accuracy (%)	23.44	22.91	21.17	21.14	19.05	12.00	10.64

Table 7. (a) No Privacy

Target Accuracy (%)	97.75	97.73	97.71
Adversary Entropy (nats)	1.65	1.65	1.67

Table 8. (a) No Privacy

Target Accuracy (%)	97.75	97.73	97.71
Adversary Entropy (nats)	1.65	1.65	1.67

Table 9. (b) ML-ARL

Target Accuracy (%)	97.52	97.50	96.58	95.97	60.00
Adversary Entropy (nats)	1.65	1.66	1.80	2.16	2.30

Table 10. (c) MaxEnt-ARL

Target Accuracy (%)	97.78	97.74	97.58	97.53	97.14	96.79	95.76	92.34	61.17
Adversary Entropy (nats)	1.65	1.66	1.78	2.11	2.26	2.26	2.27	2.27	2.29

Table 11. (a) No Privacy

Target Accuracy (%)	71.99	71.56
Adversary Accuracy (%)	30.69	30.59

Table 12. (a) No Privacy

Target Accuracy (%)	71.99	71.56
Adversary Accuracy (%)	30.69	30.59

Table 13. (b) ML-ARL

Target Accuracy (%)	71.32	70.52	70.43	69.98	69.42	24.66	22.22	5.00
Adversary Accuracy (%)	15.43	15.09	14.84	14.60	14.41	6.81	6.72	1.00

Table 14. (c) MaxEnt-ARL

Target Accuracy (%)	71.17	70.80	70.50	67.63	63.81	61.98	60.03	59.11	5.37	5.00
Adversary Accuracy (%)	16.88	16.60	16.43	13.23	8.38	5.02	3.80	2.81	1.23	1.00

Table 15. (a) No Privacy

Target Accuracy (%)	71.99
Adversary Entropy (nats)	2.09

Table 16. (a) No Privacy

Target Accuracy (%)	71.99
Adversary Entropy (nats)	2.09

Table 17. (b) ML-ARL

Target Accuracy (%)	71.32	64.90	56.99	54.46	24.66	22.22	5.00
Adversary Entropy (nats)	2.50	2.51	2.68	2.88	3.77	3.88	4.60

Table 18. (c) MaxEnt-ARL

Target Accuracy (%)	71.17	71.05	70.80	67.63	67.38	65.71	63.81	61.98	59.11	56.32	5.37	5.00
Adversary Entropy (nats)	2.27	2.28	2.31	2.91	3.01	3.24	4.14	4.56	4.57	4.57	4.59	4.60

Equations

$$\underset{{\mathbf{\theta }}_E,{\mathbf{\theta }}_T}{\min }\underset{{\mathbf{\theta }}_D}{\max }{J}_1({\mathbf{\theta }}_E,{\mathbf{\theta }}_T)-\alpha J_2({\mathbf{\theta }}_E,{\mathbf{\theta }}_D)$$

$$J_1({\mathbf{\theta }}_E,{\mathbf{\theta }}_T)$$

$$J_2({\mathbf{\theta }}_E,{\mathbf{\theta }}_D)$$

$$\underset{{\mathbf{\theta }}_D}{\min }$$

$$\underset{{\mathbf{\theta }}_E,{\mathbf{\theta }}_T}{\min }$$

$$V_1({\mathbf{\theta }}_E,{\mathbf{\theta }}_D)$$

$$V_2({\mathbf{\theta }}_E,{\mathbf{\theta }}_T)$$

$$V_3({\mathbf{\theta }}_E,{\mathbf{\theta }}_D)$$

$$\underset{{\mathbf{\theta }}_E}{\min }\text{mbox}{\mathbb{E}}_{\mathbf{x},t}\left[-\log q_T(t|E(\mathbf{x};{\mathbf{\theta }}_E);{\mathbf{\theta }}_T^{\ast })\right]+\log m$$

$$+\alpha {\mathbb{E}}_{\mathbf{x}}\left[\sum _{i=1}^m{q}_D(s_i|E(\mathbf{x};{\mathbf{\theta }}_E);{\mathbf{\theta }}_D^{\ast })\log q_D(s_i|E(\mathbf{x};{\mathbf{\theta }}_E);{\mathbf{\theta }}_D^{\ast })\right]$$

$$\underset{{\mathbf{\theta }}_E}{\min }\text{mbox}$$

$$+\alpha {\mathbb{E}}_{\mathbf{x},s}\left[\log q_D(s|E(\mathbf{x};{\mathbf{\theta }}_E);{\mathbf{\theta }}_D^{\ast })\right]$$

$$\dot{{\mathbf{\theta }}_D}$$

$$\dot{{\mathbf{\theta }}_T}$$

$$\dot{{\mathbf{\theta }}_E}$$

$$\dot{\mathbf{\theta }}=\mathbf{J}\mathbf{\theta }$$

$$V_1({\mathbf{\theta }}_E,{\mathbf{\theta }}_D)$$

$$={\mathbb{E}}_{(\mathbf{z},t,s)\sim p(\mathbf{z},t,s)}-\log q_D(s|\mathbf{z};{\mathbf{\theta }}_D)$$

$$=-\sum _{\mathbf{x},t,s}p(\mathbf{x},t,s)\log q_D(s|\mathbf{z};{\mathbf{\theta }}_D)$$

$$\sum _s{q}_D(s|\mathbf{z};{\mathbf{\theta }}_D)=1,\forall \mathbf{z}$$

$$q_D(s|\mathbf{z};{\mathbf{\theta }}_D)\ge 0,\forall \mathbf{z}$$

$$L=V_1({\mathbf{\theta }}_E,{\mathbf{\theta }}_D)+$$

$$\frac{\partial L}{\partial q_D(s|\mathbf{z};{\mathbf{\theta }}_D^{\ast })}$$

$$\Rightarrow -\frac{{\sum }_{t}p(\mathbf{z},t,s)}{q_D(s|\mathbf{z};{\mathbf{\theta }}_D^{\ast })}+\lambda (\mathbf{z})$$

$$\Rightarrow \lambda (\mathbf{z})q_D(s|\mathbf{z};{\mathbf{\theta }}_D^{\ast })$$

$$\lambda (\mathbf{z})=p(\mathbf{z})$$

$$q_D(s|\mathbf{z};{\mathbf{\theta }}_D^{\ast })=\frac{p(\mathbf{z},s)}{p(\mathbf{z})}=p(s|\mathbf{z})$$

$$q_D(s|E(\mathbf{x};{\mathbf{\theta }}_E);{\mathbf{\theta }}_D^{\ast })=p(s|E(\mathbf{x};{\mathbf{\theta }}_E))$$

$$V_2({\mathbf{\theta }}_E,{\mathbf{\theta }}_T)$$

$$={\mathbb{E}}_{(\mathbf{z},t,s)\sim p(\mathbf{z},t,s)}-\log q_T(t|\mathbf{z};{\mathbf{\theta }}_T)$$

$$=-\sum _{\mathbf{x},t,s}p(\mathbf{x},t,s)\log q_T(t|\mathbf{z};{\mathbf{\theta }}_T)$$

$$\sum _t{q}_T(t|\mathbf{z};{\mathbf{\theta }}_T)=1,\forall \mathbf{z}$$

$$q_T(t|\mathbf{z};{\mathbf{\theta }}_T)\ge 0,\forall \mathbf{z}$$

$$L=V_2({\mathbf{\theta }}_E,{\mathbf{\theta }}_T)+$$

$$\frac{\partial L}{\partial q_T(t|\mathbf{z};{\mathbf{\theta }}_T^{\ast })}$$

$$\Rightarrow -\frac{{\sum }_{s}p(\mathbf{z},t,s)}{q_T(t|\mathbf{z};{\mathbf{\theta }}_T^{\ast })}+\lambda (\mathbf{z})$$

$$\Rightarrow \lambda (\mathbf{z})q_T(t|\mathbf{z};{\mathbf{\theta }}_T^{\ast })$$

$$\lambda (\mathbf{z})=p(\mathbf{z})$$

$$q_T(t|\mathbf{z};{\mathbf{\theta }}_T^{\ast })=\frac{p(\mathbf{z},t)}{p(\mathbf{z})}=p(t|\mathbf{z})$$

$$q_T(t|E(\mathbf{x};{\mathbf{\theta }}_E);{\mathbf{\theta }}_T^{\ast })=p(t|E(\mathbf{x};{\mathbf{\theta }}_E))$$

Full text

1 Introduction

Current day machine learning algorithms based on deep neural networks have demonstrated impressive progress across multiple domains such as image classification, speech recognition etc. By stacking together multiple layers of linear and non-linear operations deep neural networks have been able to learn and identify complex patterns in data. As a by-product of these capabilities, deep neural networks have also become powerful enough to inadvertently identify sensitive information or features of data even in the absence of any additional side information. For example, consider a scenario where a user enrolls their facial image in a face recognition system for the purpose of access control. During enrollment, a feature vector is extracted from the image and stored in a database. Apart from the identity of the user, this feature vector potentially contains information that is sensitive to the user, such as the age, information that the user may never have expressly consented to provide. More generally, learned data representations could leak auxiliary information that the participants may never have intended to release. Information obtained in this manner can be used to compromise the privacy of the user or to be biased and unfair to the user. Therefore, it is imperative to develop representation learning algorithms that can intentionally and permanently obscure sensitive information while retaining task dependent information. Addressing this problem is the central aim of this paper.

A few recent attempts have been made to study related problems, such as learning censored [3], fair [14], or invariant [18] representations of data. The central idea of these approaches, collectively referred to as Adversarial Representation Learning (ARL), is to learn a representation of data in an adversarial setting. These approaches couple together (i) an adversarial network that seeks to classify and extract sensitive information from a given representation, and (ii) an embedding network that is tasked with extracting a compact representation of data while preventing the adversarial network from succeeding at leaking sensitive information. To achieve their respective goals, the adversary is optimized to maximize the likelihood of the sensitive information, while the encoder is optimized to minimize the same likelihood i.e., adversary’s likelihood of the sensitive information, thereby leading to a zero-sum game. We will henceforth refer to this formulation as Maximum Likelihood Adversarial Representation Learning (ML-ARL).

The zero-sum game formulation of optimizing the likelihood, however, is practically sub-optimal from the perspective of preventing information leakage. As an illustration consider a problem where the sensitive attribute has three categories. Let there be two instances where the adversary’s probability distribution of the sensitive label is (0.33, 0.17, 0.5) and (0.33., 0.33., 0.33.) and let the correct label be class 1 for both of them. In each of these cases the likelihood of the discriminator is the same i.e., $\log 0.33$ but the former instance is more informative than the latter. Moreover, the potential of this formulation to prevent information leakage is predicated upon: (i) the existence of an equilibrium, and (ii) the ability of practical optimization procedures to converge to such an equilibrium. As we will show, in practice, the conditions necessary for convergence may not be satisfied. Therefore, when the optimization does not reach the equilibrium, a probability distribution with the minimum likelihood is the distribution that is most certain with the potential to leak the most amount of information. In contrast, the second instance is a uniform distribution over the sensitive labels and provides no information to the adversary. This solution corresponds to the maximum entropy distribution over the sensitive labels.

Contributions: Building on the observations above, we propose a framework, dubbed Maximum Entropy Adversarial Representation Learning (MaxEnt-ARL), which optimizes an image representation with two major objectives, (i) maximally retain information pertinent to a given target attribute, and (ii) minimize information leakage about a given sensitive attribute. We pose the learning problem in an adversarial setting as a non-zero sum three player game between an encoder, a predictor and a discriminator (proxy adversary) where the encoder tries to maximize the entropy of the discriminator on the sensitive attribute and maximizes the likelihood of the predictor on the target attribute.

We analyze the equilibrium and convergence properties of the ML-ARL as well as the proposed MaxEnt-ARL formulation using tools from non-linear systems theory. We compare and evaluate the numerical performance of ML-ARL and MaxEnt-ARL for fair classification tasks on the UCI dataset, illumination invariant classification on the Extended Yale B dataset and two fabricated tasks on the CIFAR-10 and CIFAR-100 datasets. On a majority of these tasks MaxEnt-ARL outperforms all other baselines.

2 Related Work

Adversarial Representation Learning: In the context of image classification, adversarial learning has been utilized to learn representations that are invariant across domains [4, 5, 17], thereby enabling us to train classifiers on a source domain and utilize on a target domain.

The entire body of work devoted to learning fair and unbiased representations of data share many similarities to the adversarial representation learning problem. Early work on this topic did not involve an explicit adversary but shared the goal of learning representations with competing objectives. The concept of learning fair representations was first introduced by Zemel et al [19], where the goal was to learn a representation of data by “fair clustering” while maintaining the discriminative features of the prediction task. Building upon this work many approaches have been proposed to learn an unbiased representation of data while retaining its effectiveness for a prediction task. To remove influence of “nuisance variables” Louizos et al [14] proposed variational fair autoencoder (VFAE), a joint optimization framework for learning an invariant representation and a prediction task. In order to improve fairness in the representation, they regularized the marginal distribution $p(z|s)$ through Maximum Mean Discrepancy (MMD).

More recent approaches [3, 20, 1, 18] have used explicit adversarial networks to measure information content of sensitive attributes. These problems are set up as a minimax game between the encoder and the adversary. The encoder is setup to achieve fairness by maximizing the loss of the adversary i.e. minimizing negative log-likelihood of sensitive variables as measured by the adversary. Among these approaches, our proposed MaxEnt-ARL formulation is most directly related to the Adversarial Invariant Feature Learning introduced by Xie et al. [18].

Optimization Theory for Adversarial Learning: The formulation of adversarial representation learning poses unique challenges from an optimization perspective. The parameters of the models in ARL are typically optimized through stochastic gradient descent, either jointly [3, 15] or alternatively [4]. The former is, however, more commonly used in practice and is a generalization of gradient descent. While the convergence properties of gradient descent and its variants are well understood, there is relatively little work on the convergence and stability of simultaneous gradient descent in adversarial minimax problems. Recently, Mescheder *et al. * [15] and Nagarajan *et al. * [16] both leveraged tools from non-linear systems theory [9] to analyze the convergence properties of simultaneous gradient descent in the context of GANs. They show that without the introduction of additional regularization terms to the objective of the zero-sum game, simultaneous gradient descent does not converge. Our convergence analysis of ML-ARL and MaxEnt-ARL also leverages the same non-linear systems theory tools and show the conditions under which they converge.

3 Adversarial Representation Learning

The Adversarial Representation Learning setup involves observational input $\bm{x}$, a target attribute with $n$ classes $\mathcal{Y}=\{y_{1},\dots,\allowbreak y_{n}\}$ and a sensitive attribute with $m$ classes $\mathcal{S}=\{s_{1},\ldots,s_{m}\}$. In this paper, we restrict ourselves to attributes over a discrete space with multiple labels. Our goal is to learn an embedding function that maps $\bm{x}$ to $\bm{z}$ from which we can predict a target attribute $\mathcal{Y}$, while also minimizing information leakage about a known sensitive attribute $\mathcal{S}$ i.e. class labels of attribute $\mathcal{S}$.

3.1 Problem Setting

The Adversarial Representation Learning problem is formulated as a game among three players, encoder $E$, a target predictor $T$, and a discriminator $D$ that serves as a proxy for an unknown adversary $A$. After $E$ is learned and fixed, we train and evaluate an adversary $A$ with the aim of leaking information of the sensitive attribute that we sought to protect. Since the adversary $A$ is unknown to encoder at training, the encoder $E$ is trained against the discriminator $D$, which thereby acts as a proxy for the unknown $A$. An illustration of this setting is shown in Fig. 1. The encoder is modeled as a deterministic function, $\bm{z}=E(\bm{x};\bm{\theta}_{E})$, the target predictor models the conditional distribution $p(t|\bm{x})$ via $q_{T}(t|\bm{z};\bm{\theta}_{T})$ and the discriminator models the conditional distribution $p(s|\bm{x})$ via $q_{D}(s|\bm{z};\bm{\theta}_{D})$, where $p(t|\bm{x})$ and $p(s|\bm{x})$ are the ground truth labels for a given target and sensitive labels $t$ and $s$, respectively.

3.2 Background

In existing formulations of ARL, the goal of the encoder is to maximize the likelihood of the target attribute, as measured by the target predictor $T$, while minimizing the likelihood of the sensitive attribute, as measured by the discriminator $D$. This problem (henceforth referred to as ML-ARL) was formally defined by Xie et al. [18] as a three player zero-sum minimax game:

[TABLE]

where $\alpha$ is a parameter that allows us to trade-off between the two competing objectives for the encoder and,

[TABLE]

where the $KL(\cdot\|\cdot)$ terms reduce to the log-likelihood if the label distributions are ideal categorical distributions.

3.3 Maximum Entropy Adversarial Representation Learning

In the MaxEnt-ARL formulation the goal of the encoder is to maximize the likelihood of the target attribute, as measured by the target predictor, while maximizing the uncertainty in the sensitive attribute, as measured by the entropy of the discriminator’s prediction. Formally, we define the MaxEnt-ARL optimization problem as a three player non-zero sum game:

[TABLE]

where $\alpha$ allows us to trade-off between the two competing objectives for the encoder and,

[TABLE]

where $U$ is the uniform distribution. The crucial difference between the MaxEnt-ARL formulation and the ML-ARL formulation is the fact that while the encoder and the discriminator have competing objectives, in ML-ARL they directly compete against each other on the same metric (likelihood of sensitive attribute), while in MaxEnt-ARL they are optimizing competing metrics that are related but not the exact same metric.

Optimizing the embedding function to maximize the entropy of the discriminator instead of minimizing its likelihood has one crucial practical advantage. Entropy maximization inherently does not need class labels for training. This is advantageous in settings where it is either, (i) Undesirable for the embedding function to have access to the sensitive label, potentially for privacy reasons., or (ii) Sensitive labels for the data points are unknown. For instance consider, a semi-supervised scenario where only the desired label is known while the sensitive label is unknown. The embedding function can learn from such data by obtaining gradients from the entropy of the discriminator.

4 Theoretical Analysis

In this section we analyze the properties of the MaxEnt-ARL formulation and compare it to the ML-ARL formulation, both in terms of equilibrium as well as convergence dynamics under simultaneous gradient descent.

4.1 Equilibrium

Theorem 1**.**

Given a fixed encoder $E$, the optimal discriminator is $q_{D}(s|E(\bm{x};\bm{\theta}_{E});\bm{\theta}_{D}^{*})=p(s|E(\bm{x};\bm{\theta}_{E}))$ and the optimal predictor is $q_{T}(t|E(\bm{x};\bm{\theta}_{E});\bm{\theta}_{T}^{*})=p(t|E(\bm{x};\bm{\theta}_{E}))$.

Proof.

The proof uses the fact that, given a fixed encoder $E$, the objective is convex w.r.t. each distribution. Thus we can obtain the stationary point for $q_{D}(s|E(\bm{x};\bm{\theta}_{E});\bm{\theta}_{D})$ and $q_{T}(s|E(\bm{x};\bm{\theta}_{E});\bm{\theta}_{T})$ as a function of $p(s|E(\bm{x};\bm{\theta}_{E}))$ and $p(t|E(\bm{x};\bm{\theta}_{E}))$, respectively. The detailed proof is included in the supplementary material. ∎

Therefore, both the optimal distributions $q_{D}(s|E(\bm{x};\bm{\theta}_{E});\bm{\theta}_{D}^{*})$ and $q_{T}(t|E(\bm{x};\bm{\theta}_{E});\bm{\theta}_{T}^{*})$ are functions of the encoder parameters $\bm{\theta}_{E}$. The objective for optimizing the encoder now reduces to:

[TABLE]

where the first term is minimizing the uncertainty (negative log-likelihood) of the true target attribute label and the second term is maximizing unpredictability (as measured by entropy) across all the classes in the discriminator distribution, thereby, preventing leakage of any information about the sensitive attribute label. In contrast the corresponding objective of the ML-ARL problem is [18],

[TABLE]

where the first term is minimizing the uncertainty (negative log-likelihood) of the true target attribute label, while the second term is maximizing uncertainty (log-likelihood) of only the true sensitive attribute label. However, by doing so, the encoder inadvertently becomes more certain about the other labels, and can still be informative to an adversary.

**Equilibrium when $s\perp\!\!\!\perp t$: ** When the target and sensitive attributes are independent with respect to each other (e.g., age and gender), the two terms in the encoder optimization can both reach their optima simultaneously. Furthermore, the problem reduces to a non-zero sum two player game between the encoder and the discriminator in the MaxEnt-ARL case and to a zero-sum two player game between the same players in the case of ML-ARL.

Corollary 1.1**.**

When $s\perp\!\!\!\perp t$, let the optimum discriminator and predictor for an encoder $E$ be $q_{D}(s|E(\bm{x};\bm{\theta}_{E});\bm{\theta}_{D}^{*})$ and $q_{T}(t|E(\bm{x};\bm{\theta}_{E});\bm{\theta}_{T}^{*})$ respectively. The optimal encoder $E(\cdot)$ in the MaxEnt-ARL formulation induces a uniform distribution in the discriminator $q_{D}(s|E(\bm{x};\bm{\theta}_{E}^{*});\bm{\theta}_{D}^{*})$ over the classes of the sensitive attribute.

Proof.

The proof uses the fact that, given a fixed optimal discriminator $D$, $q_{T}(t|E(\bm{x};\bm{\theta}_{E});\bm{\theta}_{T}^{*})$ is independent of $q_{D}(s|E(\bm{x};\bm{\theta}_{E});\bm{\theta}_{D}^{*})$ when $s\perp\!\!\!\perp t$. The detailed proof is included in the supplementary material. ∎

**Equilibrium when $s\not\perp\!\!\!\perp t$: ** When the target and sensitive attributes are related to each other (e.g., beard and gender), the two terms in the encoder optimization cannot reach their optima simultaneously. In both the formulations, ML-ARL and MaxEnt-ARL, the relative optimality of the two objectives depends on the trade-off factor $\alpha$.

4.2 Convergence Dynamics

We analyze the standard algorithm (simultaneous stochastic gradient descent) for finding the equilibrium solution of such adversarial games. That is, we take simultaneous gradient steps in $\bm{\theta}_{E}$, $\bm{\theta}_{D}$ and $\bm{\theta}_{T}$, which can be expressed as differential equations of the form:

[TABLE]

where the gradients $f(\bm{\theta})=(f_{D}(\bm{\theta}),f_{T}(\bm{\theta}),f_{E}(\bm{\theta}))$ define a vector field over $\bm{\theta}=\left(\bm{\theta}_{D},\bm{\theta}_{T},\bm{\theta}_{E}\right)$.

The qualitative behavior of the aforementioned non-linear system near any equilibrium point can be determined via linearization with respect to that point [9]. Restricting our attention to a sufficiently small neighborhood of the equilibrium point, the non-linear state equations in (3) can be approximated by a linear state equation:

[TABLE]

where, $\left.\bm{J}=\begin{bmatrix}\frac{\partial f_{D}(\bm{\theta})}{\partial\bm{\theta}_{D}}&\frac{\partial f_{D}(\bm{\theta})}{\partial\bm{\theta}_{T}}&\frac{\partial f_{D}(\bm{\theta})}{\partial\bm{\theta}_{E}}\\ \frac{\partial f_{T}(\bm{\theta})}{\partial\bm{\theta}_{D}}&\frac{\partial f_{T}(\bm{\theta})}{\partial\bm{\theta}_{T}}&\frac{\partial f_{T}(\bm{\theta})}{\partial\bm{\theta}_{E}}\\ \frac{\partial f_{E}(\bm{\theta})}{\partial\bm{\theta}_{D}}&\frac{\partial f_{E}(\bm{\theta})}{\partial\bm{\theta}_{T}}&\frac{\partial f_{E}(\bm{\theta})}{\partial\bm{\theta}_{E}}\end{bmatrix}\right\rvert_{\bm{\theta}=\bm{\theta}^{*}}$ is the Jacobian of the vector field evaluated at the chosen equilibrium point $\bm{\theta}^{*}=(\bm{\theta}^{*}_{D},\bm{\theta}^{*}_{T},\bm{\theta}^{*}_{E})$. For small neighborhoods around an equilibrium, the trajectories of the non-linear system in (3) is expected to be “close” to the trajectories of the linear approximate system in (4).

Theorem 2** (Linearization).**

Let $\bm{x}=\bm{0}$ be an equilibrium point for the non-linear system, $\dot{\bm{x}}=f(\bm{x)}$, where $f:\mathcal{D}\rightarrow\mathbb{R}^{n}$ is continuously differentiable and $\mathcal{D}$ is a neighborhood of the origin. Let, $\left.\bm{J}=\frac{\partial f}{\partial\bm{x}}\right\rvert_{\bm{x}=\bm{0}}$. Then,

• •

The origin is asymptotically stable if Re$(\lambda_{i})<0$ for all eigenvalues of $\bm{J}$.

• •

The origin is unstable if Re$(\lambda_{i})\geq 0$ for one or more of the eigenvalues of $\bm{J}$.

Proof.

See Theorem 4.7 of [9]. ∎

5 Numerical Experiments

In this section we will evaluate the efficacy of the proposed Maximum Entropy Adversarial Representation Learning model and compare it with other Adversarial Representation Learning baselines.

5.1 Three Player Game: Linear Case

As an illustrative example we analyze the convergence of both ML-ARL and MaxEnt-ARL under the same setting. The encoder, discriminator and predictor are linear models with multiplicative weights $w_{1},w_{2}$ and $w_{3}$, respectively. We limit our model to this three variable setting for ease of analysis and visualization. Both predictor and the discriminator are optimizing cross-entropy loss on binary $\{0,1\}$ labels. To observe the game between the three players we provide same data sample $x=1$ yet with different target and sensitive labels i.e., 4 samples with $\{00,01,10,11\}$ for target and sensitive labels. Loss is calculated as the average over all samples and corresponding vector field values are also computed. The stationary point of this game, for both ML-ARL and MaxEnt-ARL, is at $(w_{1}=0,w_{2}=0,w_{3}=0)$ and the gradient of the loss functions are zero at this point. We consider a small ($30\times 30\times 30$ grid) neighborhood around the stationary point in the range $[-0.01,0.01]$ for weights $w_{1},w_{2},w_{3}$ and visualize trajectories by following the vector field of the game.

Figure 3 shows streamline plots of the vector field around $(0,0,0)$ for a point starting at the green location. In the ML-ARL case, we observe that when the predictor is fixed at $w_{3}=0$, the trajectory for the encoder and the discriminator does not converge and rotates around the stationary point. In contrast, for the MaxEnt-ARL method converges to the stationary point. When $w_{1}=0$, the streamlines for both ML-ARL and MaxEnt-ARL converge to $(0,0)$. For an alternate formulation, where the discriminator is of the form $D=z^{2}+b_{2}$, we found convergent behavior for both ML-ARL and MaxEnt-ARL.

5.2 Mixture of Gaussians

In this experiment we seek to visualize and compare the representation learned by MaxEnt-ARL and ML-ARL. We consider a mixture of 4 Gaussians with means $\mu$ at $((1,1),(2,1.5),(1.5,2.5),(2.5,3))$ and variance $\sigma=0.3$ in each case. Our model is a neural network with 2 hidden layer with 2 neuron in each layer. Each data sample has two attributes, color and shape. We setup the ARL problem with shape as the target attribute and color as the sensitive attribute. The encoder is a neural network with one hidden layer, mapping the 2-D shape into another 2-D embedding, and both the predictor and discriminator are logistic regression classifiers. The trade-off parameter is set to $\alpha=0.1$ and the parameters are learned using the Adam optimizer with learning rate of $10^{-4}$. After learning the embedding function, we freeze its parameters and learn a logistic classifier as the adversary. The test accuracy of the adversary is 63% for MaxEnt-ARL and 70% for ML-ARL. Therefore, by optimizing the entropy instead of the likelihood MaxEnt-ARL is able to leak less information about the sensitive label compared to ML-ARL. Figure 4 shows the data and the learned embeddings.

5.3 Fair Classification

We consider the setting of fair classification on two datasets from the UCI ML-repository [2], (a) The German credit dataset with 20 attributes for 1000 instances with target label being classifying bank account holders with good or bad credit and gender being the sensitive attribute, (b) The Adult income dataset has 45,222 instances with 14 attributes. The target is a binary label of annual income more or less than \50,000$, while gender is the sensitive attribute. For both ML-ARL and MaxEnt-ARL, the encoder is a NN with one hidden layer, discriminator is a NN with 2 hidden layers, and target predictor is linear logistic regression. Following ML-ARL [18] we choose 64 units in each hidden layer. We compare both ARL formulations with state-of-the-art baselines LFR (Learning Fair Representations [19]), VAE (Variational Auto-encoder [11]) and VFAE (Variational Fair Auto-encoder [14]). For MaxEnt-ARL, after learning the embedding, we again learn an adversary to extract the sensitive attribute.

Figure 5 show the results for the German and Adult datasets, for both the target and sensitive attributes. For German data, MaxEnt-ARL’s prediction accuracy is 86.33% which is close to that of the original data (87%). Other models such as, LFR, VAE, VFAE and ML-ARL have target accuracies of 72.3%, 72.5%, 72.7% and 74.4% respectively. On the other hand, for the sensitive attribute, the MaxEnt-ARL adversary’s accuracy is 72.7%. Other models reveal much more information with adversary accuracies of 80%, 80.5%, 79.5%, 79.7% and 80.2% for the original data, LFR, VAE, VFAE and ML-ARL, respectively. For the adult income dataset, the target accuracy for original data, ML-ARL and MaxEnt-ARL is 85%, 84.4% and 84.6%, respectively, while the adversary’s performance on the sensitive attribute is 67.7% and 65.5% for ML-ARL and MaxEnt-ARL, respectively.

5.4 Illumination Invariant Face Classification

We consider the task of face classification under different illumination conditions. We used the Extended Yale B dataset [6] comprising of face images of 38 people under different lighting conditions (directions of the light source) : upper right, lower right, lower left, upper left, or the front. Our target task is to identify one of the 38 people in the dataset with the direction of the light source being the sensitive attribute. We follow the experimental setup of Xie et al. [18] and Louizos et al. [14] using the same train/test split strategy and no validation set. $38\times 5=190$ samples are used for training and the rest of the 1,096 data samples are used for testing. Following the model setup in  [18], the encoder is a one layer neural network, target predictor is a linear layer and the discriminator has two hidden layers where each hidden layer consists of 100 units. The parameters are trained using Adam [10] with a learning rate of $10^{-4}$ and weight decay of $5\times 10^{-2}$.

We report baseline  [13, 14, 18] results for this experiment in Table 1 and compare with the proposed MaxEnt-ARL framework. Louizos et al. [14] regularize their neural networks via Maximum Mean Discrepancy to remove lighting conditions from data whereas Xie et al. [18] use the ML-ARL framework. The MaxEnt-ARL achieves an accuracy of 89% for identity classification (same as ML-ARL) while outperforming MMD (82%) and VFAE (85%). In terms of protecting sensitive attribute i..e, illumination direction, adversary’s classification accuracy reduces from 57% for ML-ARL to 40.2% for MaxEnt-ARL. It is clear from the table that, MaxEnt-ARL is able to remove more information from the image compared to the baselines.

5.5 CIFAR-10

We create a new binary target classification problem on the CIFAR-10 dataset[12]. The CIFAR-10 dataset consists of 10 basic classes, namely, (‘airplane’, ‘automobile’, ‘bird’, ‘cat’, ‘deer’, ‘dog’, ‘frog’, ‘horse’, ‘ship’, ‘truck’). We divide the classes into two groups: living and non-living objects. We expect the living objects to have visually discriminative properties like smooth shapes compared to regular geometric shapes of non-living objects. The target task is binary classification of an image into these two supersets with the underlying class label being the sensitive attribute. For example, the task of classifying an object as living (‘dog’ or ‘cat’) or non-living (‘ship’ or ‘truck’) should not reveal any information about its underlying identity (‘dog’, ‘cat’, ‘truck’ or ‘ship’). But as we will see, this is a challenging problem and the image representation might not be able to prevent leakage of the sensitive label.

Implementation Details: We adopt the ResNet-18 [8] architecture as the encoder, and the discriminator and adversary are 2-layered neural networks with 256 and 64 neurons, respectively. The encoder and the target predictor are trained using SGD with momentum of 0.9, learning rate of $10^{-3}$ and weight-decay of $10^{-3}$ for the prediction task. Both the discriminator and the adversary, however, are trained using Adam with a learning rate of $10^{-4}$ and weight-decay of $10^{-3}$ for 300 epochs.

Experimental Results: We evaluate performance of the predictor and adversary as we vary the trade-off parameter $\alpha$. We first note that, ideally, the desired predictor accuracy is 100%, adversary accuracy is 10% (random chance for 10 classes) and adversary entropy is 2.3 nats (uniform distribution for 10 classes). Figure 6 (a)-(b) shows the trade-off achieved between predictor and adversary along with the corresponding normalized hyper-volume (HV). For the predictor and adversary accuracy, the HV corresponds to area above the trade-off curve, while for the predictor accuracy and adversary entropy the HV is the area under the curve.

We obtain these results by repeating all the experiments five times and retaining the non-dominated solutions i.e., a solution that is no worse than any other solution in both the objectives. From these results, we observe that without privacy considerations, the representation achieves the best target accuracy but also leaks significant information. In contrast adversarial learning of the representation achieves a better trade-off between utility and information leakage. Among ARL approaches, we observe that MaxEnt-ARL is able to obtain a better trade-off compared to ML-ARL. Furthermore, among all possible solutions, MaxEnt-ARL achieves the solution closest to the ideal desired point.

5.6 CIFAR-100

We formulate a new privacy problem on the CIFAR-100 dataset. The dataset consists of 100 classes and are grouped into 20 superclasses (Table 2). Each image has a “fine” (the class to which it belongs) and a “coarse” (the superclass to which it belongs) label. We treat the “coarse” (superclass) and “fine” (class) labels as the target and sensitive attribute, respectively. So the encoder is tasked to learn features of the super-classes while not revealing the information of the underlying classes. We adopt ResNet-18 as the encoder while the predictor, discriminator and adversary are all 2-layered fully connected networks. The adversarial game is trained for 150 epochs, followed by training the adversary for 100 epochs while the parameters of the encoder are frozen.

Just as in the case of CIFAR-10, we report the trade-off achieved between predictor and adversary along with the corresponding normalized hyper-volume (HV) in Fig. 6 (c)-(d). Here we note that, ideally, we desire predictor accuracy of 100%, adversary accuracy of 1% (random chance for 100 classes) and adversary entropy of $\ln{100}=4.61$ nats (uniform distribution for 100 classes). We make the following observations from the results. Firstly, the performance of the different approaches suggest that this task is significantly harder than the CIFAR-10 task, with much lower achievable target accuracy and much higher adversary accuracy. Secondly, representation learning without privacy considerations leaks significant amount of information. Thirdly, MaxEnt-ARL is able to significantly outperform ML-ARL on this task, achieving trade-off solutions that are far better, both in terms of adversary accuracy and entropy of adversary.

6 Conclusion

This paper introduced a new formulation of Adversarial Representation Learning called Maximum Entropy Adversarial Representation Learning (MaxEnt-ARL) for mitigating information leakage from learned representations under an adversarial setting. In this model, the encoder is optimized to maximize the entropy of the adversary’s distribution of a sensitive attribute as opposed to minimizing the likelihood (ML-ARL) of the true sensitive label. We analyzed the equilibrium and convergence properties of the ML-ARL and MaxEnt-ARL. Numerical experiments on multiple datasets suggests that MaxEnt-ARL is a promising framework for preventing information leakage from image representations, outperforming the baseline minimum likelihood objective.

7 Appendix

In this appendix we include proof of Theorem 1 in Section 7.1, Corollary 1.1 in Section 7.2 and finally provide the numerical values of the trade-off fronts in the CIFAR-10 and CIFAR-100 experiment in Section 7.3.

7.1 Proof of Theorem 1

Theorem 3**.**

Given a fixed encoder $E$, the optimal discriminator is $q_{D}(s|E(\bm{x};\bm{\theta}_{E});\bm{\theta}_{D}^{*})=p(s|E(\bm{x};\bm{\theta}_{E}))$ and the optimal predictor is $q_{T}(t|E(\bm{x};\bm{\theta}_{E});\bm{\theta}_{T}^{*})=p(t|E(\bm{x};\bm{\theta}_{E}))$.

Proof.

Let, $\bm{z}$ be the fixed encoder output from input $\bm{x}$ i.e. $\bm{z}=E(\bm{x};\bm{\theta}_{E})$. Let, $p(\bm{x},t,s)$ be the true joint distribution of the variables, i.e. input $\bm{x}$, target label $t$ and sensitive label $s$. The fixed encoder is a deterministic transformation of $\bm{x}$ and generates an implicit distribution $p(\bm{z},t,s)$.

Discriminator: The objective of the discriminator is,

[TABLE]

The Lagrangian dual of the problem can be written as

[TABLE]

Now we take partial derivative of $L$ w.r.t. $q_{D}(s|\bm{z};\bm{\theta}_{D}^{*})$, the distribution of optimal discriminator. Therefore, the optimal discriminator satisfies,

[TABLE]

where we used the fact that, $\sum_{t}p(\bm{z},t,s)=p(\bm{z},s)$. Now summing w.r.t. to variable $s$ on the both sides of last line and using the fact that $\sum_{s}q_{D}(s|\bm{z};\bm{\theta}_{D}^{*})=1$ we get,

[TABLE]

By substituting $\lambda(\bm{z})$ we obtain the solution for the optimal discriminator,

[TABLE]

Therefore,

[TABLE]

Target Predictor: The objective of the predictor is,

[TABLE]

The Lagrangian dual of the problem can be written as

[TABLE]

Now we take partial derivative of $L$ w.r.t. $q_{T}(t|\bm{z};\bm{\theta}_{T}^{*})$, the distribution of optimal predictor. The optimal predictor satisfies the equation.

[TABLE]

where we used the fact that, $\sum_{s}p(\bm{z},t,s)=p(\bm{z},t)$. Now summing w.r.t. to variable $t$ on the both sides of last line and using the fact that $\sum_{t}q_{T}(t|\bm{z};\bm{\theta}_{T}^{*})=1$ we get,

[TABLE]

By substituting $\lambda(\bm{z})$ we obtain the solution of the optimal discriminator

[TABLE]

Therefore,

[TABLE]

∎

7.2 Proof of Corollary 1.1

Corollary 3.1**.**

When $s\perp\!\!\!\perp t$, let the optimum discriminator and predictor for an encoder $E$ be $q_{D}(s|E(\bm{x};\bm{\theta}_{E});\bm{\theta}_{D}^{*})$ and $q_{T}(t|E(\bm{x};\bm{\theta}_{E});\bm{\theta}_{T}^{*})$ respectively. The optimal encoder $E(\cdot)$ in the MaxEnt-ARL formulation induces a uniform distribution in the discriminator $q_{D}(s|E(\bm{x};\bm{\theta}_{E}^{*});\bm{\theta}_{D}^{*})$ over the classes of the sensitive attribute.

Proof.

Here we will prove that, when discriminator is fixed, then the encoder learns a representation of data $\bm{x}$ such that $q_{D}(s|E(\bm{x};\bm{\theta}_{E}^{*});\bm{\theta}_{D}^{*})=1/m$. First we note that although the discriminator is fixed, the discriminator probability $q_{D}(s|E(\bm{x};\bm{\theta}_{E});\bm{\theta}_{D}^{*})$ can change by changing the encoder parameters $\bm{\theta}_{E}$. Optimization of the encoder in MaxEnt-ARL is formulated as:

[TABLE]

The Lagrangian dual of the problem can be written as,

[TABLE]

Here $\lambda$ is a Lagrangian multiplier and is assumed to be a constant in the absence of any further information. Since $s\perp\!\!\!\perp t$, we have $q_{T}(t|E(\bm{x};\bm{\theta}_{E});\bm{\theta}_{T}^{*})$ is independent of $q_{D}(s|E(\bm{x};\bm{\theta}_{E});\bm{\theta}_{D}^{*})$ given $E(\bm{x};\bm{\theta}_{E})$ from Theorem 3. Therefore, if we take derivative of $L$ w.r.t. $q_{D}(s_{i}|E(\bm{x};\bm{\theta}_{E});\bm{\theta}_{D}^{*})$ and set it to zero we have:

[TABLE]

Using the first (non-trivial) constraint, we have

[TABLE]

Hence, the probability distribution of the discriminator after the encoder’s parameters $\bm{\theta}_{E}$ are optimized is $q_{D}(s_{i}|E(\bm{x};\bm{\theta}_{E}^{*});\bm{\theta}_{D}^{*})=1/m$. Thus, when the optimum discriminator parameters are fixed, the encoder optimizes the representation such that the discriminator does not leak any information, i.e., it induces a uniform distribution. ∎

7.3 CIFAR Trade-Off

We report the numerical values of the target accuracy and adversary accuracy trade-off results on the CIFAR-10 and CIFAR-100 experiments in Table 3 and Table 5, respectively. Similarly, we report the numerical values of the target accuracy and adversary entropy trade-off results on the CIFAR-10 and CIFAR-100 experiments in Table 4 and Table 6, respectively.