Protecting the Protected Group: Circumventing Harmful Fairness

TL;DR

This paper introduces Welfare-Equalizing fairness constraints to address discrimination in ML, showing how they can improve protected group welfare and providing algorithms for optimal classifiers in self-interested scenarios.

Contribution

It proposes a unified Welfare-Equalizing fairness framework, generalizing existing notions like Demographic Parity and Equal Opportunity, and analyzes conditions for aiding disadvantaged groups.

Findings

Welfare-Equalizing constraints can improve protected group welfare.

The paper characterizes optimal classifiers under these fairness constraints.

Algorithms for computing optimal classifiers are provided.

Abstract

Machine Learning (ML) algorithms shape our lives. Banks use them to determine if we are good borrowers; IT companies delegate them recruitment decisions; police apply ML for crime-prediction, and judges base their verdicts on ML. However, real-world examples show that such automated decisions tend to discriminate against protected groups. This potential discrimination generated a huge hype both in media and in the research community. Quite a few formal notions of fairness were proposed, which take a form of constraints a "fair" algorithm must satisfy. We focus on scenarios where fairness is imposed on a self-interested party (e.g., a bank that maximizes its revenue). We find that the disadvantaged protected group can be worse off after imposing a fairness constraint. We introduce a family of \textit{Welfare-Equalizing} fairness constraints that equalize per-capita welfare of protected groups, and include \textit{Demographic Parity} and \textit{Equal Opportunity} as particular cases. In this family, we characterize conditions under which the fairness constraint helps the disadvantaged group. We also characterize the structure of the optimal \textit{Welfare-Equalizing} classifier for the self-interested party, and provide an algorithm to compute it. Overall, our \textit{Welfare-Equalizing} fairness approach provides a unified framework for discussing fairness in classification in the presence of a self-interested party.

Equations

$$R(c)=\mathbb{E}\left[c(X,A)\left({\alpha }_{+}(X)Y-{\alpha }_{-}(X)(1-Y)\right)\right].$$

$$t(x):=\frac{{\alpha }_{-}(x)}{{\alpha }_{+}(x)+{\alpha }_{-}(x)},$$

$$r(x,a):=\left({\alpha }_{+}(x)+{\alpha }_{-}(x)\right)\left(p(x,a)-t(x)\right);$$

$$R(c)$$

$$=\mathbb{E}\left[r(X,A)c(X,A)\right].$$

$$W_{v,c}(a)=\mathbb{E}\left[v(X,A,Y)c(X,A)\mid A=a\right].$$

$$W_{u,c}(0)=W_{u,c}(1),$$

$$\mathbb{E}\left[c(X,A)\mid Y=1,A=0\right]=\mathbb{E}\left[c(X,A)\mid Y=1,A=1\right].$$

$$\overline{u}(x,a)$$

$$=u(x,a,1)p(x,a)+u(x,a,0)(1-p(x,a)).$$

$$R_a^{\ast }(w)=\underset{\{c:\mathcal{X}\to [0,1]\mid W_{u,c}(a)=w\}}{\max }\mathbb{E}\left[r(X,A)\cdot c(X)\mid A=a\right],$$

$$c_{\mathrm{WE}(u)}^{\ast }(x,a)=\{\begin{array}{ll}1& r(x,a)>{\lambda }_a\overline{u}(x,a)\\ 0& r(x,a)<{\lambda }_a\overline{u}(x,a)\\ {\tau }_a(x)& r(x,a)={\lambda }_a\overline{u}(x,a)\end{array}.$$

$$W_{v,c_{\text{unc}}^{\ast }}(a)<W_{v,c_{\text{unc}}^{\ast }}(1-a),$$

$$\begin{array}[]{c|ccc}&x=0&x=1&x=2\\ \hline\cr a=0&3/4&3/4&1/4\\ a=1&1&0&0\end{array}.$$

$$W_{u,c_{\text{unc}}^{\ast }}(0)=\mathbb{E}\left[c_{\text{unc}}^{\ast }(X,A)\mid Y=1,A=0\right]=\frac{6}{7}.$$

$$W_{u,c_{\text{unc}}^{\ast }}(1)=\mathbb{E}\left[c_{\text{unc}}^{\ast }(X,A)\mid Y=1,A=1\right]=1.$$

$$\{\begin{array}{l}{W}_{u,c_{\text{unc}}^{\ast }}(a)<W_{u,c_{\text{unc}}^{\ast }}(1-a)\\ W_{v,c_{\text{unc}}^{\ast }}(a)<W_{v,c_{\text{unc}}^{\ast }}(1-a)\end{array}⟹W_{v,c_{\text{unc}}^{\ast }}(a)\le W_{v,c_{\mathrm{WE}(u)}^{\ast }}(a).$$

$$W_{u,c_{\text{unc}}^{\ast }}(a)<W_{u,c_{\text{unc}}^{\ast }}(1-a)\Rightarrow \{\begin{array}{l}{W}_{u,c_{\text{unc}}^{\ast }}(a)\le W_{u,c_{\mathrm{WE}(u)}^{\ast }}(a)\\ W_{u,c_{\text{unc}}^{\ast }}(1-a)\ge W_{u,c_{\mathrm{WE}(u)}^{\ast }}(1-a)\end{array}.$$

$$W_{u,c_{\text{unc}}^{\ast }}(a)<W_{u,c_{\text{unc}}^{\ast }}(1-a)⟹c_{\text{unc}}^{\ast }(x,a)\le c_{\mathrm{WE}(u)}^{\ast }(x,a).$$

$$\epsilon =2\sqrt{6\left(\frac{1}{\mathbb{P}(A=0)}+\frac{1}{\mathbb{P}(A=1)}\right)\max \{{\eta }_u,{\eta }_r\}}$$

$$O\left(\frac{1}{\max \{{\eta }_u,{\eta }_r\}}\left(\log \frac{1}{\delta }+\log \log \frac{1}{\max \{{\eta }_u,{\eta }_r\}}\right)\right).$$

$$\max_{c\in\mathrm{WE}(u)}R(c)=\max_{w\in\mathbb{R}}\max_{{\begin{array}[]{c}\scriptstyle{c:\,\mathcal{X}\times A\to[0,1]}\\ \scriptstyle{W_{u,c}(0)=W_{u,c}(1)=w}\end{array}}}\mathbb{P}(A=0)R_{0}(c)+\mathbb{P}(A=1)R_{1}(c).$$

$$w^{\ast }\in \arg \underset{w}{\max }\{\mathbb{P}(A=0)R_0^{\ast }(w)+\mathbb{P}(A=0)R_1^{\ast }(w)\}$$

$$R_a(c)-{\lambda }_a{W}_{u,c}(a)\le R_a(c_{a,w^{\ast }}^{\ast })-{\lambda }_a{W}_{u,c_{a,w^{\ast }}^{\ast }}(a).$$

$$R_a(c)-{\lambda }_a{W}_{u,c}(a)=\mathbb{E}\left[\left(r(X,A)-{\lambda }_a\overline{u}(X,A)\right)c(X,A)\mid A=a\right],$$

$$c_{\text{parity}}(x,a)=\{\begin{array}{ll}1& p(x,a)>t(x,a)+\frac{{\lambda }_a}{{\alpha }_{+}(x)+{\alpha }_{-}(x)}\\ 0& p(x,a)<t(x,a)+\frac{{\lambda }_a}{{\alpha }_{+}(x)+{\alpha }_{-}(x)}\end{array},$$

$$c_{\text{opportunity}}^{\ast }(x,a)=\{\begin{array}{ll}1& p(x,a)\left(1-\frac{{\lambda }_a}{{\alpha }_{+}(x)+{\alpha }_{-}(x)}\right)>t(x,a)\\ 0& p(x,a)\left(1-\frac{{\lambda }_a}{{\alpha }_{+}(x)+{\alpha }_{-}(x)}\right)<t(x,a)\end{array}.$$

$$\begin{array}[]{c|cc}&x=0&x=1\\ \hline\cr a=0&0.4&0.6\\ a=1&0&1\end{array}.$$

$$\begin{array}[]{c|cc}&x=0&x=1\\ \hline\cr a=0&2/3&1/3\\ a=1&1/3&2/3\end{array}.$$

$$W_{v,c_{\text{unc}}^{\ast }}(0)\le \alpha W_{u,c_{\text{unc}}^{\ast }}(0)<\frac{1}{\alpha }{W}_{u,c_{\text{unc}}^{\ast }}(1)\le W_{v,c_{\text{unc}}^{\ast }}(1).$$

$$\text{maximize}\sum _{x,a}\mathbb{P}(X=x,A=a)\cdot r(x,a)\cdot c(x,a)$$

$$\text{subject to}\sum _x\mathbb{P}(X=x\mid A=0)\cdot \overline{u}(x,0)\cdot c(x,0)$$

Full text

1 Introduction

At first glance, algorithms may seem free of human biases such as sexism or racism. However, in many situations, they are not: the automated recruiting tool used by Amazon was favoring men (Doleac and Hansen 2016); judges in the US use the COMPAS algorithm to estimate the probability that the defendant will re-offend while this algorithm was accused of being biased against black people (Larson et al. 2016). See O’Neill (2016) for many more examples. These challenges call for imposing fairness constraints on algorithm design and, in particular, on machine-learned classifiers, which are the subjects of this paper. As a running example, consider a bank that gives loans to potential borrowers and is regulated by a policy-maker. The bank learns a decision rule (namely, a classifier) from historical data to decide for whom to approve or decline the loan to maximize its revenue that is increasing with the number of repaid loans. As repeatedly observed in the past, the resulting classifier may be biased against a protected group (e.g., ethnic minority). Hence, the regulator may wish to impose a fairness constraint on the bank.

Bias is deemed unjust. Beyond that, it affects the welfare of protected groups, as borrowers have preferences towards the different outcomes of the classification, captured by utility functions.111We use the term utility for the well-being of a single individual and welfare for the aggregated well-being of groups of individuals. Consequently, the goal of imposing fairness constraints is to improve the welfare of the disadvantaged group (the originally-discriminated one). A natural approach by which the regulator can achieve this goal is by assuming a utility function and requiring the bank’s classifier to equalize the protected groups’ welfare. The two most popular fairness constraints, Demographic Parity (Agarwal et al. 2018; Dwork et al. 2012) and Equal Opportunity (Hardt et al. 2016) (henceforth DP and EO, respectively) are special cases of this approach for particular utility functions. For instance, DP is recovered by assuming that every agent gets a utility of one when receiving the loan, and zero otherwise.

The possibility that fairness may harm the well-being of those it is designed to protect, i.e., that it can harm the disadvantaged group’s welfare, seems counter-intuitive. However, it is well known both theoretically and empirically that the most intuitive fairness constraint of Unawareness (which forbids using the sensitive attribute in classification) can be harmful (Corbett-Davies and Goel 2018; Dwork et al. 2018; Ustun, Liu, and Parkes 2019; Doleac and Hansen 2016). For example, Doleac and Hansen (2016) show that the “ban the box” policy, adopted by the United States and preventing employers from seeing applicants’ criminal background, decreased the welfare of discriminated minorities (the chances of getting a job). Additionally, Liu et al. (2018) discovered that DP and EO are not free of the same flaw if we consider long-term consequences. They show that fairness constraints may force the bank to give loans to those members of the disadvantaged group who, otherwise, would not have received the loans due to the high probability of default. Therefore, such unqualified borrowers are likely to have problems with paying back the loan. This increased default ratio would harm the disadvantaged group’s average credit score, thereby harming its welfare in the long run.

1.1 Our contribution

In this paper, we uncover another mechanism underlying harmful fairness even in static settings:

Imposing a fairness constraint can make the disadvantaged group worse off if the fairness constraint and the utilities of the population mismatch.

Following the recent trends of fair ML literature, we assume that agents may have different preferences over classification outcomes, which are captured by utility functions. For example, borrowers may differ in their value for getting the loan depending on their access to alternative sources of money and on the purpose of borrowing. With utilities in hand, we can use social welfare to evaluate a group’s well-being for any given classifier. To talk about the mismatch between utilities and fairness constraints, the latter has to be defined in utilitarian terms. As we described above, fairness constraints like DP and EO are naturally cast as equalizing some welfare functions of the protected groups. However, the difficulty in applying any welfare-based fairness constraint is that the utilities must be known to the designer, while these utilities represent individuals’ private tastes. Hence, even if domain experts determine the utilities used in a fairness constraint (henceforth, assumed utilities), they can only approximate the actual utilities of the population (underlying utilities in what follows). This discrepancy leads to the following conclusion.

In practice, the mismatch between the underlying utilities of the population and the utilities assumed by the regulator is unavoidable.

Together with the observation that the mismatch can make fairness harmful, this becomes a serious caution for regulators that design fairness constraints for a certain industry, e.g., banking. We complement this caution with a positive message. Naturally, a small discrepancy between underlying and assumed utilities must be innocuous. However, we characterize a much more applicable and promising connection; we show that

Fairness constraints help the disadvantaged group whenever the utilities assumed by the regulator and the underlying utilities of the population agree on which group is disadvantaged.

Finally, we suggest additional ways to deal with the mismatch if the underlying utilities can be approximated from data.

1.2 Related Work on Economic Ideas in Fair Classification

Welfare-Equalizing, our approach to fairness, has a long history in normative economics (Pazner and Schmeidler 1978; Roemer 1986) (where it is known under the name of egalitarianism) and political philosophy (Rawls 2009); it was used for fair resource allocation without money transfers (Li and Xue 2013), in the field of cooperative games (Dutta and Ray 1989) and bargaining problems (Kalai and Smorodinsky 1975). In contrast to recent papers on the utilitarian approach to fair classification (Heidari et al. 2018; Heidari, Gummadi, and Krause 2019), which suggest maximizing the minimal welfare among the protected groups, we strengthen this desideratum by making it a normative requirement: the welfare must be equal among the subgroups defined by a sensitive attribute. This normative condition allows one to separate the fairness constraint (which may be imposed by a regulator) from the selfish objective of the decision-maker (a revenue-maximizing bank in our running example) and thus allows one to analyze how decisions change after imposing the fairness constraint. Another advantage of the Welfare-Equalizing concept is the simple threshold structure of the optimal fair classifier (similar to the one for Demographic Parity or Equal Opportunity (Corbett-Davies et al. 2017)), which makes it efficiently computable.

This work joins recent attempts (Rambachan et al. 2020a, b; Hu and Chen 2020; Elzayn and Fish 2020; Hossain, Mladenovic, and Shah 2020) to bring better economic understanding to fairness in ML; we address some of them here and refer the reader to Finocchiaro et al. (2020) for a comprehensive survey. Hu and Chen (2020) propose an optimization framework for fair classification and welfare analysis. In their modeling, a learner executes a soft-margin SVM with the additional constraint of group fairness: limiting the two groups’ welfare discrepancy to a predefined quantity. They provide a sensitivity analysis, showing that applying stricter fairness constraints (decreasing the allowed discrepancy) can worsen welfare outcomes for both groups. Their findings are in line with ours, but our analysis is fundamentally different; in particular, they do not address the utility mismatch issue. Imposing fairness constraints on profit-maximizing entities, as we do in this paper, is an understudied point of view, as noted recently by Elzayn and Fish (2020).

There are also several recent attempts to harness economic principles to fair classification. For example, Gölz, Kahng, and Procaccia (2019) treat fair classification as an allocation of goods, where there is a fixed amount of resources to distribute. They examine the compatibility (or lack thereof) of Equalized Odds with axioms of fairness from the economic literature on fair division; see (Brandt et al. 2016) for a survey. Envy-freeness, the dominant fairness concept in economics, plays a crucial role in several recent papers at the intersection of economics and AI (Caragiannis et al. 2019; Cohler et al. 2011; Benade et al. 2018; Guruswami et al. 2005; Gal et al. 2016; Plaut and Roughgarden 2018), including several works on fair classification (Zafar et al. 2017; Balcan et al. 2019; Ustun, Liu, and Parkes 2019; Hossain, Mladenovic, and Shah 2020). However, these papers on fair classification focus on sample complexity and generalization (Balcan et al. 2019), or asserting that users favor treatment disparity (Ustun, Liu, and Parkes 2019; Zafar et al. 2017) in health applications.

The work most related to ours is the paper by Hossain, Mladenovic, and Shah (2020). Concurrently to and independently of our work, Hossain, Mladenovic, and Shah (2020) argue for group equability in fair classification, which coincides with our Welfare-Equalizing fairness constraint. They, too, show that their concept subsumes previously suggested fairness notions. However, there is a significant difference between the two works. First, Hossain, Mladenovic, and Shah (2020) are interested in learning the best classifier from data, and hence address issues of generalization from samples and differentiability. In contrast, we devote our paper to societal considerations of fair classification, and thus consider fairness as a post-processing step similarly to (Corbett-Davies et al. 2017; Hardt et al. 2016). Additionally, in contrast to Hossain, Mladenovic, and Shah (2020), our analysis focuses on the mismatch between the utilities assumed by the regulator and the actual, underlying utilities.

1.3 Paper structure

In Section 2, we present our formal model, define the Welfare-Equalizing fairness framework, and prove structural results for optimal fair classifiers in Subsection 2.2. Section 3 deals with the implications of a mismatch between the population’s underlying utilities and the utilities assumed by the regulator. Finally, Section 4 describes how to compute the bank-optimal fair classifier if the assumed utilities approximate the underlying utilities well enough.

2 Model

We consider a general classification problem, where agents have an ex-ante non-observable “quality” correlated with observable attributes. We keep using the metaphor of a bank that predicts the reliability of the population and makes lending decisions; however, the same setting captures student admissions, recruiting, assessing the recidivism risk for a criminal, etc.

There are three parties in the model: a heterogeneous population of potential borrowers; a bank that makes lending decisions based on the observable attributes of borrowers and cares only about its revenue; and a regulator that cares about fairness and can restrict the set of lending policies available to the bank by imposing a fairness constraint. We now present these parties formally.

Borrowers

We assume that each potential borrower (henceforth borrower) is associated with a pair of observable attributes $(X,A)\in\mathcal{X}\times\{0,1\}$. Here $A$ is a binary222The assumption of the dichotomy of $A$ is made for simplicity. Extending our results to the non-binary case (ethnicity) is straightforward. sensitive attribute (e.g., gender) and $X\in\mathcal{X}$ encodes all other characteristics of a borrower, e.g., employment history, salary, education, assets and so on. We do not impose any assumptions on $\mathcal{X}$. By $\{A=0\}$ and $\{A=1\}$ we denote the groups of all borrowers with the sensitive attribute equal to [math] or $1$, respectively; we call $\{A=a\}$ a protected group. Furthermore, in addition to the observable attributes $X$ and $A$, every borrower is also associated with an unobservable variable $Y\in\{0,1\}$, which describes whether that borrower will pay back the loan or not. For brevity, we call borrowers with $Y=1$ and $Y=0$, good and bad, respectively. The statistical characteristics of the population are described by a probability space $(\Omega,\mathcal{F},\mathbb{P})$; so $X=X(\omega)$, $A=A(\omega)$ and $Y=Y(\omega)$ are random variables on $\omega\in\Omega$. By small letters $(x,a,y)$ we denote realizations of $X$, $A$, and $Y$, i.e., generic elements of $\mathcal{X}\times\{0,1\}\times\{0,1\}$.

Each borrower $(x,a,y)$ obtains a utility when receiving the loan. This utility can depend on $x$ and $a$ in many ways, but what is more critical is that it must depend on the non-observable quality of the borrower, namely $v=v(x,a,y)$. To simplify the presentation, we assume that the utility from a rejected application is zero and that the average utility of a borrower with given $x$ and $a$ is non-negative,333Zero utility for not getting a loan is a normalization-condition: before borrowing money, everybody is at zero utility level. Non-negativity of $v$ can be regarded as a rationality assumption on borrowers: no rational agent would apply for a loan if she/he expects that getting the loan brings negative utility while not getting gives [math]. i.e., $\mathbb{E}[v(X,A,Y)\mid X=x,A=a]\geq 0$. However, both assumptions can be relaxed. We refer to $v$ as the underlying utility of the population.

The bank

We assume that the bank knows the joint distribution of $(X,A,Y)$ from historical data. In particular, it knows the exact conditional probability of being a good borrower given the observable attributes; we denote it by $p(x,a)=\mathbb{P}(Y=1\mid X=x,A=a)$.444Indeed, this is aligned with previous works that consider fairness as a post-processing step (Corbett-Davies et al. 2017; Hardt et al. 2016).

The bank makes lending decisions based on $X$ and $A$ but without observing $Y$. It uses a classifier $c:\mathcal{X}\times\{0,1\}\to[0,1]$ where $c(x,a)$ is the probability of giving a loan to a population of borrowers with $X=x$ and $A=a$. Each loan given to a good borrower brings a revenue555In contrast to the rest of the literature, we allow the bank’s revenue to depend on the non-sensitive attribute $X$. This becomes important if $X$ also encodes the type of loan a client is applying for, e.g., different borrowers may need a different amount of money and thus bring a different revenue/loss. of $\alpha_{+}(X)>0$ to the bank while each bad borrower leads to a loss of $\alpha_{-}(X)>0$; we assume that $\alpha_{\pm}(x)$ are bounded functions of $x\in\mathcal{X}$. The bank’s revenue depends on the choice of a classifier $c$, and is defined by

[TABLE]

To ease notation, we define $t(x)$ and $r(x,a)$ such that for every $a\in\{0,1\},x\in\mathcal{X}$

[TABLE]

hence, we can rewrite Equation (1) by taking a conditional expectation with respect to $A,X$ as

[TABLE]

The goal of the bank is to maximize $R(c)$ over the set of feasible classifiers, i.e., classifiers that satisfy the regulator’s constraints.

The regulator

The regulator evaluates the well-being of a group using its welfare: the expected utility of its members. For an underlying utility function $v$ and a classifier $c$, the welfare of the subgroup $\{A=a\}$ is given by

[TABLE]

The regulator aims to equalize welfare among the protected groups. However, as we discuss in the introduction, the underlying utility $v$ is unknown to the regulator; thus, the regulator is forced to use a certain substitute $u$ instead. We refer to $u$ as the assumed utility; ideally, the assumed utility must be an approximation to the underlying one.

The objective of the regulator is captured by the following $u$-Welfare-Equalizing constraint ($u$-WE for abbreviation).

Definition 1** ($u$-WE classifier).**

Given a utility-function $u$, a classifier $c$ is $u$-Welfare-Equalizing if

[TABLE]

i.e., if $c$ equalizes the welfare among the two protected groups. The set of all such classifiers is denoted by $\mathrm{WE}(u)$.

Note that Welfare-Equalizing constraint is defined with respect to the assumed utility.

2.1 Special cases of Welfare-Equalizing fairness

The framework of Welfare-Equalizing fairness allows one to analyze existing fairness constraints in a unified manner. For instance,

• •

The fairness constraint of Demographic Parity (DP) (e.g., (Agarwal et al. 2018; Dwork et al. 2012)) requires that the fraction of those who receive loans in the two groups must be the same. Formally, a classifier $c$ satisfies DP if $\mathbb{E}[c(X,A)\mid A=0]=\mathbb{E}[c(X,A)\mid A=1]$. It is a special case of WE fairness with $u(x,a,y)\equiv 1$ for any triplet $(x,a,y)$.

• •

Motivated by drawbacks of DP, Hardt et al. (2016) concluded that good and bad borrowers within protected groups must be treated separately and introduced the concept of Equal Opportunity (EO). Under this fairness constraint, the fraction of good borrowers who get loans must be the same in the two subgroups. Formally, a classifier $c$ satisfies EO if

[TABLE]

We recover EO by setting $u(x,a,y)=y\cdot\beta_{a}$. The coefficients $\beta_{a}=\nicefrac{{1}}{{\mathbb{E}[Y\mid A=a]}}$ normalize the maximal possible welfare in each group to $1$. Such a rescaling is known under the name of “relative welfare” and is commonly used in economics to make welfare or utilities among groups comparable (Kalai and Smorodinsky 1975).

• •

Borrowers can differ in the amount of money $m$ they need. We can assume that information about $m$ is encoded in $X$, so $m=m(X)$. Then, a straightforward generalization of EO is the following concept of Heterogeneous-EO given by $u(x,a,y)=y\cdot m(x)\cdot\beta_{a}$ with $\beta_{a}=\nicefrac{{1}}{{\mathbb{E}[m(x)\mid A=a]}}$. We can capture any other heterogeneity similarly (e.g., different interest rates, time-period, and payment schedules).

2.2 Structural properties of the bank-optimal classifiers

In this subsection, we analyze classifiers that are optimal for the bank. We first state the result of Corbett-Davies et al. (2017), who characterize the structure of the bank-optimal unconstrained classifier. Then, we build upon their results for the constrained case. Namely, we assume that the regulator imposes the $u$-WE fairness constraint on the bank and explore the structure of the bank-optimal classifiers. We show that the optimal classifiers have a generalized threshold structure, a fact that is extensively used in Sections 3 and 4.

Unconstrained classifier $c_{\textnormal{unc}}^{*}$

If the regulator imposes no constraint on the bank, i.e., the bank is free to choose any classifier, then the revenue-maximizing classifier has a simple form (Corbett-Davies et al. 2017). Only borrowers with $r(x,a)>0$ are profitable for the bank, which is equivalent to the probability of paying back $p(x,a)$ being greater than $t(x)$ (recall that $t$ and $r$ are defined by Equations (2) and (3)). Consequently, the optimal lending policy is given by the following threshold classifier $c_{\textnormal{unc}}^{*}$: all borrowers with $p(x,a)>t(x)$ get loans ($c_{\textnormal{unc}}^{*}(x,a)=1$) and all borrowers with $p(x,a)\leq t(x)$ are rejected ($c_{\textnormal{unc}}^{*}(x,a)=0$).666For definiteness, we assume that if the bank finds the two decisions equally profitable (the knife-edge case $p(x,a)=t(x)$), it chooses the one with fewer loans given (e.g., this policy minimizes paperwork).

Constrained classifier $c^{*}_{\mathrm{WE}(u)}$

Consider the general, constrained case, where the regulator imposes $u$-WE fairness on the bank. For a fixed and given assumed utility $u$, we denote by $c^{*}_{\mathrm{WE}(u)}$ the classifier that maximizes the bank’s revenue $R(c)$ (see Equation (1)) among all $u$-WE classifiers $c\in\mathrm{WE}(u)$. The set of $u$-WE classifiers is non-empty since $0\in\mathrm{WE}(u)$ and, therefore, the bank’s optimization problem is well-defined. To ease notation, we denote by $\overline{u}(x,a)$ the assumed utility of a borrower associated with $(x,a)$ averaged over the possible values of $Y$,

[TABLE]

Further, we denote by $R_{a}^{*}(w)$ the maximal revenue that the bank could extract from the group $\{A=a\}$ at the (assumed) welfare level $W_{u,c}(a)=w$. Formally,

[TABLE]

where $r$ is given by Equation (3). The following Proposition 2 shows that the bank-optimal constrained classifier always exists and reveals its structure.

Proposition 2**.**

The bank-optimal $u$-WE classifier $c^{*}_{\mathrm{WE}(u)}$ exists. Furthermore, each optimal classifier has the following form:

[TABLE]

The group-dependent thresholds $\lambda_{a}$, $a\in\{0,1\}$ belong to the super-gradient777For a concave function $f=f(t),\ t\in[t_{0},t_{1}]$, the super-gradient $\partial_{t}f$ is the set of all $q\in\mathbb{R}$ such that $f(t^{\prime})\leq f(t)+q(t^{\prime}-t)$ for all $t^{\prime}$. If $f$ is continuous, then for any $t$ the super-gradient is non-empty, see Rockafellar (2015). of the subgroup-optimal revenue $R_{a}^{*}(w)$ (a concave function of $w$) computed at the welfare level $w^{*}$ maximizing the total bank’s revenue $\mathbb{P}(A=0)R_{0}^{*}(w)+\mathbb{P}(A=1)R_{1}^{*}(w)$. The functions $\tau_{a}:\,\mathcal{X}\to[0,1]$ are arbitrary888In particular, there always exists $c^{*}_{\mathrm{WE}(u)}$ with constant $\tau_{a}$, i.e., independent of $x$. up to the constraint that $c^{*}_{\mathrm{WE}(u)}$ provides the desired welfare level $w^{*}$ for both groups, $w^{*}=W_{u,c^{*}_{\mathrm{WE}(u)}}(0)=W_{u,c^{*}_{\mathrm{WE}(u)}}(1)$.

Proof sketch of Proposition 2.

The revenue maximization over $c\in\mathrm{WE}(u)$ can be represented as a two-stage procedure. In the first stage, we find the revenue-maximizing classifier in each of the subgroups $\{A=a\}$ given the welfare level $w$; in the second stage, we optimize over $w$. The welfare constraint in the first stage can be internalized using the Lagrangian approach; the corresponding Lagrange multipliers $\lambda_{a}$ are equal to the “shadow prices”, i.e., the derivatives of the value functions $R_{a}^{*}(w)$ with respect to $w$. This internalization reduces finding the subgroup optimal classifier to the unconstrained problem; thus, the optimal classifier has a threshold structure similarly to $c_{\textnormal{unc}}^{*}$. This structure is inherited by $c^{*}_{\mathrm{WE}(u)}$. Since the resulting linear program is infinite-dimensional and $R_{a}^{*}(w)$ may be non-differentiable, the formal proof requires some functional-analytic arguments presented in the appendix. ∎

For the special cases of Demographic Parity and Equal Opportunity, the explicit form of the optimal classifiers was obtained by Corbett-Davies et al. (2017). Their result becomes an immediate corollary of Proposition 2; see Subsection A.3 in the appendix.

3 Mismatch of Fairness and Underlying Utilities

As we discussed in the introduction, the regulator aiming to equalize welfare among protected groups unavoidably assumes a certain approximation $u$ of the underlying utilities $v$. For example, $u$ can be determined by the domain experts, while $v$ reflects the private tastes of the population and hence is not observable directly. We refer to the fact that $u$ is different from $v$ as a mismatch. This section explores how imposing the Welfare-Equalizing fairness constraint with respect to $u$ affects the underlying welfare, which is measured by $v$.

We use the situation that exists before imposing the fairness constraint as the benchmark. We say that a group $\{A=a\}$ is $v$-disadvantaged if under the bank-optimal unconstrained classifier, the welfare of $\{A=a\}$ is lower than the welfare of the other group $\{A=1-a\}.$ Formally, the group $\{A=a\}$ is $v$-disadvantaged if

[TABLE]

where $W$ is defined in Equation (5) and $c_{\textnormal{unc}}^{*}$ is the bank-optimal unconstrained classifier from Subsection 2.2. When the underlying utility $v$ is clear from the context, we say that the group is disadvantaged and omit the dependence on $v$.

Ideally, imposing WE-fairness (or any other fairness constraint) should improve the welfare of the disadvantaged group. However, as we show next, this is not always the case. We say that a fairness constraint harms the group $\{A=a\}$ if $W_{v,c^{*}}(a)<W_{v,c_{\textnormal{unc}}^{*}}(a)$, where $c^{*}$ is the bank-optimal classifier after imposing that fairness constraint. Put differently, the fairness constraint harms the group $\{A=a\}$ if the welfare of the group (measured with respect to underlying utilities) decreases after imposing the fairness constraint.

Harmful mismatch

We now demonstrate that the mismatch between assumed and underlying utilities can make the fairness constraint harmful to the disadvantaged group.

Example 3*.*

Let the underlying utility-function be $v(x,a,y)\equiv 1$, i.e., all borrowers equally benefit from receiving loans. However, the regulator does not know the underlying utilities and decides to impose the fairness constraint of EO. Equivalently, the regulator assumes the utility $u(x,a,y)=y\cdot\beta_{a}$, for normalizing coefficients $\beta_{a}$ (as we explained in Subsection 2.1). Notice that the underlying utility is the one associated with DP, and the regulator’s assumed utility is the one associated with EO. Since $v\neq u$, there is a mismatch.

Suppose that $\mathcal{X}=\{0,1,2\}$, and all the combinations of $(x,a)$ have the same probability of $\frac{1}{6}$. Furthermore, assume that the fraction $p(x,a)$ of good borrowers is given by the table

[TABLE]

In addition, let the revenue of the bank from a paid-back loan be $\alpha_{+}(x)=1$, and the loss from a borrower’s default be $\alpha_{-}(x)=2$ for every $x\in\mathcal{X}$.

We first want to determine which group is disadvantaged. The threshold for the bank-optimal unconstrained classifier $c_{\textnormal{unc}}^{*}$ equals $t(x)=\frac{2}{3}$. Hence, under $c_{\textnormal{unc}}^{*}$, only borrowers with $x=0$ receive loans in the group $\{A=1\}$. In $\{A=0\}$, borrowers with $x\in\{0,1\}$ receive loans since in such cases $t(x)=\frac{2}{3}<\frac{3}{4}=p(x,0)$; however, loans are not given to borrowers with $x=2$ since $t(2)=\frac{2}{3}>\frac{1}{4}=p(2,0)$. Consequently, $\{A=1\}$ is disadvantaged: $W_{v,c_{\textnormal{unc}}^{*}}(0)=\frac{2}{3}$ compared to $W_{v,c_{\textnormal{unc}}^{*}}(1)=\frac{1}{3}$.

Next, let us examine how imposing the fairness constraint of EO changes the outcome of the bank-optimal classifier. The proportion of loans given by $c_{\textnormal{unc}}^{*}$ to good borrowers in $\{A=0\}$ is equal to the welfare of this group with respect to the assumed utility $u$, namely

[TABLE]

In contrast, for $\{A=1\}$ we have

[TABLE]

By imposing $u$-WE-fairness, the regulator requires the bank to equalize these two quantities. To do so, the bank-optimal constrained classifier can either increase the amount of loans given to $\{A=0\}$ by approving some applications of $x=2$ or decrease the number of loans given to $\{A=1\}$. However, giving loans to $x=2$ in $\{A=0\}$ is too costly: the cost $\frac{3}{4}\alpha_{-}-\frac{1}{4}\alpha_{+}$ is not compensated by the benefit $1\cdot\alpha_{+}$ from giving the same amount of loans to good borrowers with $x=0$ in $\{A=1\}$. Therefore, the bank-optimal constrained classifier coincides with $c_{\textnormal{unc}}^{*}$ in $\{A=0\}$ and gives fewer loans to $x=0$ in the group $\{A=1\}$: $c^{*}_{\mathrm{WE}(u)}(0,1)=\frac{6}{7}$. Consequently, the bank equalizes the proportion of loans given to good borrowers in both groups: $W_{u,c^{*}_{\mathrm{WE}(u)}}(0)=W_{u,c^{*}_{\mathrm{WE}(u)}}(1)=\frac{6}{7}$. However, since the underlying utility is measured by $v$, not $u$, we get that $W_{v,c^{*}_{\mathrm{WE}(u)}}(1)=\frac{2}{7}<W_{v,c_{\textnormal{unc}}^{*}}(1)=\frac{1}{3}$ and the disadvantaged group is harmed by imposing the fairness constraint.

3.1 Can WE-fairness help the disadvantaged group despite the mismatch?

In this subsection, we examine when imposing WE-fairness could help the disadvantaged group, even in the presence of a mismatch.999We remind the reader that “helping” and “harming” is always with respect to the actual underlying utility. A natural observation is that if the assumed and underlying utilities “almost” match, the results of imposing $u$-WE-fairness or $v$-WE-fairness should be roughly the same. We postpone such quantitative statements to the end of this subsection and first address easy-to-check general conditions guaranteeing that fairness is not harmful. Theorem 4 below shows that only a lenient condition is required to assure that imposing WE-fairness benefits the disadvantaged group.

Theorem 4**.**

If $v$ and $u$ agree on which group is disadvantaged, then the $u$-WE classifier weakly increases $v$-welfare of the disadvantaged group, i.e.,

[TABLE]

At first glance, this theorem may look rather intuitive. However, the claim is non-trivial even if there is no mismatch, i.e., when $v\equiv u$. To see this, recall that the WE-fairness constraint is imposed on the bank: a self-interested party, which is going to find the revenue-optimal way to satisfy the constraint. One possible way to achieve welfare equality is to give no loans to both protected groups thus harming both of them. As we show in Section C in the appendix, such an undesired behavior is possible when Unawareness is imposed. However, it never happens under the WE-fairness; we use this inherent property of WE-fairness as a tool to prove Theorem 4. Momentarily, let us assume that there is no mismatch, i.e., that the assumed utility and the underlying one are exactly the same. In such a case, the following Lemma 5 suggests that not only imposing WE-fairness always improves the welfare of the disadvantaged group, but also that every individual in the disadvantaged group is weakly better off.

Lemma 5** (Matching utilities).**

The bank-optimal $u$-WE classifier makes the $u$-disadvantaged protected group $\{A=a\}$ weakly better off at the expense of the advantaged group. Formally,

[TABLE]

Moreover, any borrower from the $u$-disadvantaged group who receives a loan under the unconstrained classifier, receives it under the bank-optimal $u$-WE one.101010While our paper is focused on group notions of fairness, we stress that this result provides stronger “individual” guarantees. Formally, for every $x\in\mathcal{X}$ it holds that

[TABLE]

Proof of Lemma 5.

By Proposition 2, the welfare level $w^{*}=W_{u,c_{\mathrm{WE}(u)}^{*}}(0)=W_{u,c_{\mathrm{WE}(u)}^{*}}(1)$ achieved by the $u$-WE classifier maximizes the revenue $\mathbb{P}(A=0)R_{0}^{*}(w)+\mathbb{P}(A=1)R_{1}^{*}(w)$ as a function of welfare level $w$. The sub-group revenues $R_{a}^{*}(w)$, $a\in\{0,1\}$ are concave functions; thus the welfare level $w^{*}$ lies between their maxima. These maxima are attained at the welfare levels of the bank-optimal unconstrained classifier; therefore, the welfare level $w^{*}$ is between $W_{u,c_{\textnormal{unc}}^{*}}(a)$ and $W_{u,c_{\textnormal{unc}}^{*}}(1-a)$. See Figure 1 for illustration.

The second part of the lemma, the individual guarantees, follow from the threshold structure of the bank-optimal constrained classifier $c_{\mathrm{WE}(u)}^{*}$, which we identified in Proposition 2. Since the welfare level $w^{*}$ is above the maximum of $R_{a}^{*}(w)$ (for $\{A=a\}$ being the disadvantaged group), its super-gradient contains $\lambda_{a}\leq 0$; therefore, $c_{\textnormal{unc}}^{*}(x,a)$, which corresponds to $\lambda_{a}=0$, is below $c_{\mathrm{WE}(u)}^{*}(x,a).$ ∎

Equipped with Lemma 5, we are ready to prove Theorem 4.

Proof of Theorem 4.

We apply Lemma 5 to the assumed utility function $u$. By the second part of the lemma, after imposing the ${u}$-WE constraint every borrower $x\in\mathcal{X}$ from the ${u}$-disadvantaged group $\{A=a\}$ who received loans under the unconstrained classifier still receives them. Namely, $c_{\textnormal{unc}}^{*}(x,a)\leq c_{\mathrm{WE}({u})}^{*}(x,a)$ for all $x\in\mathcal{X}$. Multiplying both sides by the actual underlying utility $v(x,a,y)$, substituting $X=x$ and $Y=y$, and taking expectation, we get $W_{v,c_{\textnormal{unc}}^{*}}(a)<W_{v,c_{\mathrm{WE}({u})}^{*}}(a).$ In other words, the bank-optimal classifier $c_{\mathrm{WE}({u})}^{*}$ for the utilities ${u}$ assumed by the regulator improves the welfare of the group $\{A=a\}$ with respect to the underlying utilities $v$. Since $v$ and $u$ agree on the disadvantaged group’s identity, this classifier improves the disadvantaged group’s underlying welfare. ∎

In the presence of a mismatch, it is easy to see that a weak converse to Theorem 4 holds, i.e., fairness always makes the disadvantaged group weakly worse off. To avoid a mismatch, one can use a simple quantitative criterion. We say that $u$ is an $\alpha$-approximation of $v$ for some $\alpha\geq 1$ if for all $x,a$, and $y$, $\frac{1}{\alpha}\leq\frac{v(x,a,y)}{{u}(x,a,y)}\leq\alpha.$ Theorem 4 implies the following quantitative criterion on how well the regulator should approximate the underlying utilities to help the disadvantaged group.

Corollary 6**.**

If the utility ${u}$ assumed by the regulator is an $\alpha$-approximation of the underlying utility $v$ and the gap between the welfare of the groups with respect to ${u}$ is big enough, namely, $\frac{W_{u,c_{\textnormal{unc}}^{*}}(0)}{W_{u,c_{\textnormal{unc}}^{*}}(1)}\in\left(0,\frac{1}{\alpha^{2}}\right)\cup\left(\alpha^{2},+\infty\right)$, then ${u}$-WE classifier helps the $v$-disadvantaged group.

4 Computing Bank-Optimal Welfare-Equalizing Classifiers

In this section, we provide evidence for the applicability of WE-fairness by developing tools for computing bank-optimal WE classifiers. Our goal is to show how the bank can use the assumed utility proposed by the regulator to compute approximately optimal classifiers. We first discuss the case where the underlying and assumed utilities match (i.e., $v\equiv u$) and the other objects are fully known (revenues and losses $\alpha_{\pm}(x)$, and the probability $p(x,a)$ of paying back). Later on, we relax this assumption. Due to space considerations and our desire to focus on the conceptual assets of the paper, we defer most of the analysis to the appendix, as well as an elaborated version of formal statements.

Consider the case where $u\equiv v$, and both $\alpha_{\pm}$ and $p$ are known. If the data is tabular, i.e., $\mathcal{X}$ is relatively small (say several thousand different borrower types), we can compute the bank-optimal $u$-WE-classifier $c^{*}_{\mathrm{WE}(u)}$ explicitly by standard LP-methods. For large sets of attributes, e.g., multidimensional or continuous, the size of the LP “explodes”, and a different approach should be taken. In this case, we use the structural insights from Proposition 2: the bank-optimal WE-classifier $c_{\mathrm{WE}(u)}^{*}$ is parameterized by the two thresholds $\lambda_{a}$ for $a\in\{0,1\}$; therefore, to compute it we can restrict our attention to a finite-dimensional parametric family of classifiers. Due to the large-scale nature of the problem, we shall seek efficient algorithms for computing approximately bank-optimal WE classifiers, where these approximated solutions are defined as follows.

Definition 7**.**

A classifier $c$ is $(\varepsilon,\varepsilon^{\prime})$ bank-optimal $u$-WE if $R(c)\geq R(c^{*}_{\mathrm{WE}(u)})-\varepsilon$ and $|W_{u,c}(0)-W_{u,c}(1)|\leq\varepsilon^{\prime}$.

Notice that such classifiers are doubly approximated: they approximate the revenue of the (exact) bank-optimal WE classifier, and also approximately equalize the welfare of the two classes. In the appendix, we demonstrate how the bank can efficiently find a classifier that approximates the revenue and (exactly) equalizes the welfare. Namely, we show how one can apply the ternary search method (as in Hardt et al. (2016)) to find $(\varepsilon,0)$ $u$-WE-classifier in $O\left(\log^{2}\left(\frac{1}{\varepsilon}\right)\right)$ run-time.

Next, we get rid of the full information assumption. Recent papers (Balcan et al. 2019; Hossain, Mladenovic, and Shah 2020) propose convex relaxations for imposing fairness constraints in settings like ours, which includes generalization bounds. However, since an extensive body of literature deals with estimating real-valued functions (ranging from linear regression to deep learning), we take a different approach. We suggest that the bank employs the assumed utility given by the regulator, and describe its performance guarantees in terms of the “quality” of $u$. This perspective has been adopted recently for several other ML problems (Medina and Vassilvitskii 2017; Lykouris and Vassilvitskii 2018; Purohit, Svitkina, and Kumar 2018).

For simplicity, we assume that $|r(x,a)|$, $u(x,a,y)$, and $v(x,a,y)$ are all upper-bounded by $1$.

Proposition 8**.**

Fix a small $\delta>0$ and assume that the bank has access to a sample of $(X,A,Y,\alpha_{\pm},v)$ and to estimators ${u}$ and $\hat{r}$ such that $\mathbb{E}\left[\left|{{u}}-v\right|\right]\leq\eta_{u}$ and $\mathbb{E}\left[\left|\hat{r}-r\right|\right]\leq\eta_{r}$ for small enough $\eta_{u}$ and $\eta_{r}$. Then, a $(\varepsilon,\varepsilon)$ bank-optimal $v$-WE classifier with

[TABLE]

can be computed with probability $1-\delta$ on a sample of size

[TABLE]

5 Conclusions

Our paper draws on the economic approach to fair classification. It initiates the discussion of the impact that the regulator’s misconception about the population characteristics may have on the protected groups’ well-being. Beyond that, we believe that our WE-fairness can serve as an anchor for grounding other fairness stances.

We have prioritized clarity over generality and focused on binary classification. Nevertheless, our results are more general. The key technical Proposition 2 can be extended to multiple classes (e.g., several loans with different periods and interest rates).111111As in the binary case, there is one threshold $\lambda_{a}$ per group $a$, but now the revenue $r$ and the utilities $\overline{u}$ depend on the class. The optimal WE-classifier selects a class $c$ with the maximal $r-\lambda_{a}\cdot\overline{u}$. All other results (the mismatch analysis and the algorithms) are, essentially, corollaries of Proposition 2 and extend straightforwardly. Noisy utilities and revenues can be assumed for free if we interpret $u$, $v$, and $r$ in all formulas as the conditional expectations for a given triplet $(X,A,Y)$. Allowing for utilities with negative expectations also does not alter the statements. We see considerable scope for follow-up work. One prominent direction is to understand how the “price of fairness” is distributed among the parties, e.g., by how much the bank’s revenue and the advantaged group’s welfare drop.

Acknowledgements

We are grateful to Lily Hu, Nikita Kalinin, Alexander Nesterov, Margarita Niyazova, and Ivan Susin for insightful discussions and pointers to the relevant literature. We also thank the anonymous reviewers for their helpful comments and clarifications and Lillian Bluestein for proofreading.

The work of M. Tennenholtz is funded by the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement n$\degree$ 740435). F. Sandomirskiy is partially supported by the Lady Davis Foundation, by Grant 19-01-00762 of the Russian Foundation for Basic Research, and by Basic Research Program of the National Research University Higher School of Economics. This work was done while O. Ben-Porat and F. Sandomirskiy were at the Technion—Israel Institute of Science and were partially funded by the European Union’s Horizon 2020 research and innovation programme (grant agreement n$\degree$ 740435).

Appendix A Threshold Structure of the Bank-optimal WE-classifier

Proof of Proposition 2.

The revenue maximization under the WE constraint can be split into two subsequent maximization problems:

[TABLE]

First, the bank finds the revenue-maximizing classifier $c_{a,w}^{*}:\mathcal{X}\to[0,1]$ that maximizes the revenue $R_{a}(c)=\mathbb{E}[r(X,A)\cdot c(X)\mid A=a]$ in the subgroup $\{A=a\}$ given some welfare-level $W_{u,c}(a)=w$ (we refer to these classifiers as optimal marginal classifiers). Then, the bank finds the bank-optimal level $w^{*}$ of $w$ by maximizing the total revenue $\mathbb{P}(A=0)R_{0}^{*}(w)+\mathbb{P}(A=1)R_{1}^{*}(w)$, where as in the statement of the proposition, $R_{a}^{*}(w)$ denotes $R_{a}(c_{a,w}^{*})$.

Thus, the bank-optimal WE classifier $c^{*}_{\mathrm{WE}(u)}(x,a)$ equals $c_{a,w^{*}}^{*}(x)$ provided that the optimization problems for $c_{a,w}^{*}$ and $w^{*}$ have a solution. The threshold representation in Equation (7) will follow from a similar representation for the optimal marginal classifiers $c_{a,w}^{*}$. Existence and the threshold representation of $c_{a,w^{*}}^{*}$ is the subject of the next two subsections.

A.1 Existence of $c_{a,w}^{*}$ and $w^{*}$, concavity of $R_{a}^{*}(w)$

For a given welfare-level $w$ of a group $\{A=a\}$, the set of feasible marginal classifiers $F(w)=\{c:\mathcal{X}\to[0,1]\mid W_{u,c}(a)=w\}$ is non-empty if and only if $w\in[0,\mathbb{E}[u\mid A=a]]$. Indeed, for any $c$ we have $W_{u,c}(a)\in[0,\mathbb{E}[u\mid A=a]]$; therefore, the set $F(w)$ is empty outside of this interval. For any $w$ inside, the constant classifier $\frac{w}{\mathbb{E}[u\mid A=a]}\in F(w)$ and thus $F(w)$ is non-empty. Therefore, for $w\in[0,\mathbb{E}[u_{+}\mid A=a]]$, $R_{a}^{*}(w)=\sup_{c\in F(w)}R_{a}(c)$ is finite; outside this interval we assume $R_{a}^{*}(w)=-\infty$. Let us show that $R_{a}^{*}(w)$ is concave and continuous and that supremum is, in fact, maximum, i.e., that the bank-optimal (marginal) classifier $c_{a,w}^{*}$ exists.

For any $c^{\prime}\in F(w^{\prime})$ and $c^{\prime\prime}\in F(w^{\prime\prime})$ the convex combination $c=\beta c^{\prime}+(1-\beta)c^{\prime\prime}$ for $\beta\in[0,1]$ belongs to $F(w)$, where $w=\beta w^{\prime}+(1-\beta)w^{\prime\prime}$. Therefore, $R_{a}^{*}(w)\geq\beta R_{a}(c^{\prime})+(1-\beta)R_{a}(c^{\prime\prime})$. Taking supremum over $c^{\prime}\in F(w^{\prime})$ and $c^{\prime\prime}\in F(w^{\prime\prime})$ we obtain $R_{a}^{*}(w)\geq\beta R_{a}^{*}(w^{\prime})+(1-\beta)R_{a}^{*}(w^{\prime\prime})$. Thus $R_{a}^{*}(w)$ is concave in $w$.

Next, we prove that the maximum is attained. Consider $c^{(n)}\in F(w),n=1,2,...$, a sequence of classifiers such that $R_{a}(c^{(n)})\to R^{*}_{a}(w)$. This sequence is a bounded set in the Hilbert space $L^{2}(\mathcal{X},\mathbb{P}|_{(X\mid A=a)})$; thus, by the Banach-Alaoglu theorem (John 1985, Section 5), this sequence contains a weakly-convergent subsequence $c^{(n_{k})}\to c\in F(w)$. Since $R_{a}(c^{n_{k}})$ is a scalar product of $c$ and $r(\,\cdot\,,a)$ in $L^{2}(\mathcal{X},\mathbb{P}|_{(X\mid A=a)})$, we get $R_{a}(c)=\lim_{k\to\infty}R_{a}(c^{n_{k}})=R_{a}^{*}(w)$. Thus the maximum is attained.

A similar argument proves continuity of $R_{a}^{*}(w)$. By concavity, continuity on $[0,\mathbb{E}[u\mid A=a]]$ follows from upper-semi-continuity: $R_{a}^{*}(w)\geq\lim_{n\to\infty}R^{*}_{a}(w^{(n)})$ for any sequence $w^{(n)}\to w,\ n\to\infty.$ Consider the sequence of bank-optimal classifiers $c^{*}_{a,w^{(n)}},\ n=1,2,\dots$. There is a weakly-convergent subsequence $c^{*}_{a,w^{(n_{k})}}$ with some $c$ as the weak limit. We have $R_{a}(c)=\lim_{k\to\infty}R^{*}_{a}(w^{(n_{k})})$ and similarly $W_{u,c}(a)=\lim_{k\to\infty}w^{(n_{k})}=w$. Thus $c\in F(w)$ and $R_{a}^{*}(w)\geq\lim_{n\to\infty}R^{*}_{a}(w^{(n)})$.

The existence of the bank-optimal $w^{*}$, where

[TABLE]

follows from the continuity of $R^{*}_{a}(w)$ for $w\in[0,\mathbb{E}[u\mid A=a]]$.

A.2 Threshold structure of the bank-optimal marginal classifiers $c_{a,w^{*}}^{*}$

By concavity and continuity of $R^{*}_{a}(w)$, the super-gradient at $w^{*}$ is non-empty. Pick an element $\lambda_{a}$ from it.

By the definition of super-gradient, for any $w\in\mathbb{R}$ it holds that $R^{*}_{a}(w)\leq R^{*}_{a}(w^{*})+\lambda_{a}(w-w^{*})$. Equivalently, for the bank-optimal classifier $c_{a,w^{*}}^{*}\in F(w^{*})$ and an arbitrary classifier $c:\ \mathcal{X}\to[0,1]$ we have

[TABLE]

In other words, the bank-optimal classifier $c_{a,w^{*}}^{*}$ maximizes $R_{a}(c)-\lambda_{a}W_{u,c}(a)$ over all $c:\ \mathcal{X}\to[0,1]$. The converse is also true: any maximizer $c$ gives a bank-optimal classifier $c_{a,w^{*}}$ provided that it belongs to $F(w^{*})$.

Since

[TABLE]

the maximizer equals $1$ when $r(x,a)-\lambda_{a}\overline{u}(x,a)>0$ and equals [math] if the inequality has the strict opposite sign. The condition $c\in F(w^{*})$ imposes the constraint on otherwise arbitrary values of $c(x,a)=\tau_{a}(x)$ for $x$ with $r(x,a)-\lambda_{a}\overline{u}(x,a)=0$. ∎

A.3 Example: the bank-optimal classifiers for DP and EO

Proposition 2 immediately provides the bank-optimal classifiers under DP and EO: for the former, it is given by the “additive perturbation” of the optimal unconstrained classifier, while the latter is obtained by the “multiplicative perturbation.”

More precisely, for the optimal classifier $c_{\textnormal{parity}}^{*}$ satisfying DP we recover the result of Corbett-Davies et al. (2017): there exist group-dependent constants $\lambda_{a}\in\mathbb{R}$, $a\in\{0,1\}$ such that

[TABLE]

where $p(x,a)$ is the probability that a borrower with attributes $X=x$ and $A=a$ pays back, and the threshold $t(x)$ is defined by (2).

For the bank-optimal EO classifier $c_{\textnormal{opportunity}}^{*}$, there exist $\lambda_{a}\in\mathbb{R}$, $a\in\{0,1\}$ such that

[TABLE]

Appendix B Unconstrained bank-optimal classifier is unfair

Here we show that if the bank is free to pick any classifier (i.e., the regulator imposes no constraints), the bank-optimal unconstrained classifier (formally defined in Subsection 2.2) can discriminate against one of the groups even if the groups are of equal size and contain the same fraction of good borrowers. We illustrate this claim by the following example.

Example 9* (Bank-optimal unconstrained classifier is unfair).*

Let $A$ and $X$ be binary, and let all four combinations have the probability $\frac{1}{4}$. The probability $p(x,a)$ of being a good borrower is given by the matrix

[TABLE]

In the group $\{A=0\}$, the attribute $X$ poorly separates good and bad borrowers, while it is a perfect predictor of creditworthiness for $\{A=1\}$. If losses from a defaulting client are equal to the revenue from two borrowers paying back, e.g. $\alpha_{-}(x)=2\cdot\alpha_{+}(x)$, then the revenue-maximizing classifier has the following form: the bank gives the loans to those borrowers that have the probability of paying back above the threshold $t=\frac{\alpha_{-}(x)}{\alpha_{-}(x)+\alpha_{+}(x)}\equiv\frac{2}{3}\simeq 0.66$. Thus, the bank gives loans to applicants with $(X,A)=(1,1)$ only. As a result, no loans are given to the group $\{A=0\}$, although the prior distribution shows that in total $1/2$ of borrowers from each of the groups $\{A=0\}$ and $\{A=1\}$ are good. This contradicts the intuitive understanding of fairness. Moreover, for any utility function $u$, the welfare of the group $\{A=0\}$ is below the welfare of the group $\{A=1\}$, i.e., the optimal unconstrained classifier is not $u$-Welfare-Equalizing for any $u$.

Appendix C Unawareness Can Harm Both Groups and the Bank

Unawareness is perhaps the most intuitive fairness criterion. A classifier $c$ satisfies unawareness if $c(x,0)=c(x,1)$ for all $x\in\mathcal{X}$. Informally, to deliver a “fair” outcome to both groups $A=0$ and $A=1$, the classifier $c$ must ignore the protected attribute $A$.

This natural idea is not innocuous: $X$ and $A$ can be dependent and thus information contained in $X$ can be used as a proxy for $A$, making the disadvantaged group even worse off; see Doleac and Hansen (2016). Despite its flaws, unawareness is promoted by layers in the labor market, insurance market121212By the decision of the Court of Justice of the European Union, insurance premiums must be determined in a gender-blind way starting from December 2012. This initiative eventually increased the gap between premiums paid by females and males (Collinson 14 January 2017)., and also by GDPR.131313The General Data Protection Regulation is a law adopted by the European Union in 2016 that regulates data protection and privacy.

The bank-optimal unaware classifier $c_{\text{unaware}}^{*}$ can be constructed by the same logic as the optimal unconstrained one; see Subsection 2 and Corbett-Davies et al. (2017): an applicant receives money if the probability $\mathbb{P}(Y=1\mid X=x)$ of being a good borrower given the observed attribute $X=x$ is above the threshold $t(x)=\frac{\alpha_{-}(x)}{\alpha_{+}(x)+\alpha_{-}(x)}$. Formally, $c_{\text{unaware}}^{*}(x,a)=c_{\text{unaware}}^{*}(x)=1$ for borrowers $x$ such that $\overline{p}(x)=\mathbb{P}(A=0)p(x,0)+\mathbb{P}(A=1)p(x,1)\geq t(x)$ and $c_{\text{unaware}}^{*}(x)=0$ if $\overline{p}(x)<t(x)$.

We now show that Unawareness can make all three parties strictly worse off: both groups and the bank, regardless of the underlying utility function $v$. The following example shows this phenomenon when information losses caused by Unawareness are significant: the interpretation of the non-protected attribute $X$ depends on the protected141414For example, having children correlates with spending more time at work for men and has negative correlation for women (Parker and Wang 2013). attribute $A$ and without knowing $A$ the classifier cannot achieve good separation of good and bad borrowers; thus giving loans becomes too risky.

Example 10*.*

Suppose that $\mathcal{X}=\{0,1\}$ and all the four combinations of attributes $(x,a)$ have an equal probability of $\frac{1}{4}$. The fraction $p(x,a)$ of good borrowers is given by the following table

[TABLE]

Notice that the fraction of good and bad borrowers is the same in both groups. Further, $X=1$ is a positive signal about the quality of a borrower in the group $\{A=0\}$ and a negative one for $\{A=1\}$.

The bank-optimal unconstrained classifier $c_{\textnormal{unc}}^{*}$ gives loans to agents with $p(x,a)\geq t(x)$; thus, if the threshold $t(x)$ is between $\frac{1}{3}$ and $\frac{2}{3}$ (for example, $t(x)=\frac{3}{5}$ if the revenue $\alpha_{+}$ from giving money to a good borrower equals $\frac{2}{3}$ of the losses $\alpha_{-}$ from a bad one), then agents with $(X,A)$ equal to $(0,0)$ or $(1,1)$ get loans under $c_{\textnormal{unc}}^{*}$. So one half of the members in each group receives loans and the bank gets a positive revenue.

After imposing Unawareness, the bank-optimal classifier $c_{\text{unaware}}^{*}$ compares the threshold $t(x)$ with the average fraction of good borrowers for a given $X=x$, namely $\overline{p}(x)=\mathbb{P}(A=0)p(x,0)+\mathbb{P}(A=1)p(x,1)$. In our example, $\overline{p}(x)=\frac{1}{2}$ for every $x$, and thus no loans are given for $t(x)>1/2$. Thus, if $t(x)\in(1/2,2/3)$ for all $x\in\mathcal{X}$, Unawareness pushes the welfare of both groups to zero for any underlying utility $v$ as well as the bank’s revenue.

Appendix D Proof of Corollary 6

Proof of Corollary 6.

By Theorem 4, it is enough to show that, ${u}$ and $v$ agree on which group is disadvantaged. Without loss of generality, the group $\{A=0\}$ is $u$-disadvantaged. We shall show that $\{A=0\}$ is also $v$-disadvantaged. We get the following chain of inequalities:

[TABLE]

The first and the last steps follow from the definition of $\alpha$-approximation, and the second step from the condition in the corollary. Consequently, $\{A=0\}$ is $v$-disadvantaged, which completes the proof. ∎

Appendix E Algorithms for Approximately Bank-optimal WE-Classifiers

This section is devoted to finding approximately bank-optimal WE-classifiers. It contains a formal version of the results discussed in Section 4 together with their proofs.

We first assume complete information about the problem (the utilities $u\equiv v$, revenues/losses $\alpha_{\pm}$, and the probability distribution $\mathbb{P}$ are known), and then relax this assumption.

Throughout this section we assume that the bank-optimal unconstrained classifier provides non-zero utility to both groups. This positivity condition allows us to avoid degenerate cases.

E.1 Complete information

In this subsection we assume that the underlying utility $v$ coincides with the utility $u$ assumed by the regulator. Furthermore, these utilities are known to the bank as well as the distribution $\mathbb{P}$ and functions $\alpha_{+},\alpha_{-}$. In particular, the bank can compute $r(x,a)=\mathbb{E}[\alpha_{+}Y-\alpha_{-}(1-Y)\mid X=x,A=a]$ and $\overline{u}(x,a)=\mathbb{E}[u(X,A,Y)\mid X=x,A=a]$, namely, the average revenue and the average utility for a given $(x,a)$ pair.

LP-based approach

As we mentioned in Section 4, the bank-optimal $u$-WE-classifier is the solution to the linear program (LP) given in Figure 2. It maximizes the revenue under the constraint of equal welfare, and hence for a medium size set of attributes $\mathcal{X}$ (say several thousands) we can compute $c^{*}_{\mathrm{WE}(u)}$ explicitly by standard LP-methods.

Threshold-based approach

For large sets of attributes $\mathcal{X}$, e.g., multidimensional or continuous, solving the LP in Figure 2 is no longer feasible in practice. In this case, we harness the threshold structure of the bank-optimal classifier (see Proposition 2) and use binary search over the thresholds $\lambda_{a}$ to compute approximately optimal solutions. The approximation notion is captured by Definition 7.

The next proposition shows that the bank can efficiently compute a classifier that approximates the optimal revenue and achieves an exact equality of welfare. To simplify the presentation, in this subsection we assume that the group $\{A=0\}$ is disadvantaged.

Proposition 11**.**

Assume that $\alpha_{+},\alpha_{-}$, and $u$ are known. Further, assume that the bank has access to an oracle that computes expectations w.r.t. $\mathbb{P}$ in constant time. Then, for any positive $\varepsilon$ such that

[TABLE]

an ($\varepsilon\cdot\max_{x,a}|r(x,a)|$, 0) bank-optimal $u$-WE-classifier can be computed in $O\left(\log^{2}\left(\frac{1}{\varepsilon}\right)\right)$, uniformly over all other parameters.

As we show below, the assumption on the expectation oracle can be relaxed by sampling, which leads to an approximate welfare equality.

Proof of Proposition 11.

The high-level structure of the proof is as follows. Similarly to the proof of Proposition 2, we represent the optimization problem as finding an approximately bank-optimal marginal classifier for a given welfare level $w$ in each subgroup, and then determining the revenue-maximizing $w$. For a given welfare level $w$, the revenue-maximizing classifiers and the revenue $R^{*}(w)$ itself can be approximated by the binary search over thresholds $\lambda_{a}$. Since $R^{*}$ is a concave function, the revenue-maximizing $w$ can be determined by the Golden ratio or the ternary search as in Hardt et al. (2016). The technical difficulty is to bound the number of steps needed for the binary search to converge even though a small change in $\lambda_{a}$ can result in a massive change both in welfare and revenue.

Now we discuss all the steps in detail. To simplify formulas, we can assume w.l.o.g. that $\max_{x,a}\overline{u}(x,a)=\max_{x,a}|r(x,a)|=1$ by changing the scale.

Proposition 12 below guarantees that for each subgroup $\{A=a\}$ and welfare level $w$, a marginal classifier with welfare $w$ and revenue at least $R_{a}^{*}(w)-\frac{\varepsilon}{2}$ can be computed in $O\left(\log\left(\frac{1}{\varepsilon}\right)\right)$ time by binary search over thresholds $\lambda_{a}$. We verify below that the upper bound on $\varepsilon$ ensures that none of $\{\lambda_{0},\lambda_{1}\}$ exceeds $\frac{2}{\varepsilon}$ (a technical condition of Proposition 12).

Therefore, for a given $w$ we can also compute the optimal total revenue $R^{*}(w)=\mathbb{P}(A=0)R_{0}^{*}(w)+\mathbb{P}(A=1)R_{1}^{*}(w)$ up to $\frac{\varepsilon}{2}$ in $O\left(\log\left(\frac{1}{\varepsilon}\right)\right)$. The total revenue $R^{*}$ is a continuous concave function of the welfare level $w$ (see the proof of Proposition 2) and thus it can be approximately maximized by the Golden-ratio search or by the ternary search as in Hardt et al. (2016).

For the moment, assume that we can compute $R^{*}(w)$ exactly. Then $O\left(\log\frac{1}{\varepsilon}\right)$ iterations of the search will find $w$ such that $R^{*}(w)\geq R^{*}(w^{*})-\frac{\varepsilon}{2}$, where $w^{*}$ is the revenue-maximizing welfare level. Indeed, the function $R^{*}(w)$ is defined on the interval $0\leq w\leq b$, where $b=\min\{\mathbb{E}[u|A=0],\,\mathbb{E}[u|A=1]\}$, and takes values within $[-1,1]$ by the assumption on the magnitude of $r$. On any interval $[w,w^{\prime}]$ with $|w^{\prime}-w|\leq\frac{\varepsilon}{4}\cdot b$ containing the bank-optimal welfare level $w^{*}$, the value $R^{*}(w^{*})$ is within $\frac{\varepsilon}{2}$ of $\max\{R^{*}(w),R^{*}(w^{\prime})\}$ (otherwise, by convexity, there exists $w^{\prime\prime}\in[0,b]$ such that $R^{*}(w^{\prime\prime})<-1$); thus, $O\left(\log\frac{1}{\varepsilon}\right)$ steps of the search are enough.

If, along the way, the values of $R^{*}$ are computed with precision $\frac{\varepsilon}{2}$, the output $w$ of the Golden-ratio search satisfies $R^{*}(w)\geq R^{*}(w^{*})-{\varepsilon}$ and, therefore, provides an $\varepsilon$-bank-optimal WE-classifier. Since all $O\left(\log\frac{1}{\varepsilon}\right)$ iterations of the search algorithm involve computing $R^{*}$ at a new point, the total run-time is $O\left(\log\frac{1}{\varepsilon}\cdot\log\frac{1}{\varepsilon}\right)$.

In order to satisfy the technical condition of Proposition 12, we need one more twist: the search should start on the smaller interval

[TABLE]

where $w_{\lambda}(a)$ denotes a point $w$ such that the super-gradient of $R^{*}_{a}(w)$ contains $\lambda$ (a point $w_{\lambda}(a)$ can be easily computed as in the proof of Proposition 12). By the definition of $I$, any $w\in I$ satisfies the condition of Proposition 12 and it remains to check that the Golden-ratio search, restricted to $I$, still approximates the optimum $w^{*}$. Put differently, it is enough to show that $w^{*}\in I$. At $w^{*}$, First-order conditions imply $0\in\partial R^{*}(w^{*})=\mathbb{P}(A=0)\partial R^{*}_{0}(w^{*})+\mathbb{P}(A=1)\partial R^{*}_{1}(w^{*})$. Therefore, there exist $\lambda_{a}\in\partial R^{*}_{a}(w^{*})$ such that $\mathbb{P}(A=0)|\lambda_{0}|=\mathbb{P}(A=1)|\lambda_{1}|$. Pick $\lambda_{a}$ that is non-negative. Then $\lambda_{a}\cdot w^{*}\leq 2$; otherwise, $R^{*}_{a}(0)\leq R^{*}_{a}(w^{*})+\lambda_{a}(0-w^{*})<-1$ which contradicts the normalization of $r$. By Lemma 5, $w^{*}\geq W_{u,c_{\textnormal{unc}}^{*}}(0)$; hence, $\lambda_{a}\leq\frac{2}{W_{u,c_{\textnormal{unc}}^{*}}(0)}$. The upper bound on $\varepsilon$ implies that both $|\lambda_{0}|$ and $|\lambda_{1}|$ are bounded by $\frac{1}{3\varepsilon}$ and thus $w^{*}$ belongs to $I$. ∎

The next proposition is an auxiliary technical result used in the proof of Proposition 11. It shows that approximately bank-optimal marginal classifier for a group $\{A=a\}$ with a given welfare level $w$ can be found by binary search over thresholds. The threshold $\lambda_{a}$ can also be thought as a super-gradient of the marginal revenue $R^{*}_{a}$. The super-gradient of a concave function may “explode” if the point is close to the boundary. This is why we need an additional technical assumption that the super-gradient of $R^{*}_{a}$ at $w$ is not too big.

Proposition 12**.**

Under the assumptions of Proposition 11, for a given $a\in\{0,1\}$, $\varepsilon>0$, and w\in\big{[}0,\,\mathbb{E}[u|A=a]\big{]} such that the super-gradient $\partial R^{*}_{a}(w)$ contains $\lambda$ with $|\lambda|<\frac{1}{\varepsilon}$, a marginal classifier for a subgroup $\{A=a\}$ with welfare level $w$ and revenue of at least $R_{a}^{*}(w)-\varepsilon\cdot\max_{x,a}|r(x,a)|$ can be computed in $O\left(\log\left(\frac{1}{\varepsilon}\right)\right)$.

Proof of Proposition 12.

Similarly to the proof of Proposition 11, we can assume w.l.o.g. that $\max_{x,a}|r(x,a)|=1$. Consider the threshold classifier $c_{\lambda}$ such that $c_{\lambda}(x)=1$ if $r(x,a)\geq\lambda\overline{u}(x,a)$ and equals zero otherwise. For any $\lambda\in\mathbb{R}$, this classifier generates the optimal subgroup revenue among all the classifiers having the same welfare level $w_{\lambda}(a)=\mathbb{E}[\overline{u}(X,A)c_{\lambda}(X)\mid A=a]$, i.e., $R_{a}(c_{\lambda})$ equals $R_{a}^{*}(w_{\lambda})$; moreover, $\lambda$ belongs to the super-gradient of the concave function $R^{*}_{a}$ at $w_{\lambda}$ (see the proof of Proposition 2).

Note that $w_{\lambda}$ is a decreasing function of $\lambda$ and thus we can use binary search in order to find a $\lambda$ such that $w_{\lambda}$ is close to $w$. By the assumption on $w$ we can restrict the search to $\lambda\in\left[-\frac{1}{\varepsilon},\frac{1}{\varepsilon}\right]$ and thus after $O\left(\log\frac{1}{\varepsilon}\right)$ steps find $\lambda$ and $\lambda^{\prime}$ such that $w_{\lambda}\leq w\leq w_{\lambda^{\prime}}$ and $|\lambda-\lambda^{\prime}|\leq\varepsilon$. We can treat $w$ as the convex combination $\theta w_{\lambda}+(1-\theta)w_{\lambda^{\prime}}$, $\theta\in[0,1]$, and consider a classifier $c=\theta c_{\lambda}+(1-\theta)c_{\lambda^{\prime}}$. By the construction, $c$ has the right welfare level of $w$.

Let us check that $R_{a}(c)\geq R^{*}_{a}(w)-\varepsilon$. For any concave function $h=h(z)$, $z_{1},z_{2}\in\mathbb{R}$, and $\theta\in[0,1]$, it holds that

[TABLE]

where $g_{k}$ is any super-gradient of $h$ at $z_{k}$. The revenue of $c$ is given by the convex combination of values $\theta R^{*}_{a}(w_{\lambda})+(1-\theta)R^{*}(w_{\lambda^{\prime}})$, while $R^{*}_{a}(w)$ is the value at the convex combination $w=\theta w_{\lambda}+(1-\theta)w_{\lambda^{\prime}}$. Applying the above inequality, we get $R_{a}(c)\geq R^{*}_{a}(w)-|w_{\lambda}-w_{\lambda^{\prime}}|\cdot|\lambda-\lambda^{\prime}|$; hence, $R_{a}(c)\geq R^{*}_{a}(w)-\varepsilon$. ∎

E.2 Incomplete information

The assumption of expectation oracle from Proposition 11, as well as the exact knowledge of revenues/losses $\alpha_{+}$ and $\alpha_{-}$ can be relaxed. Moreover, in this subsection we allow for a mismatch between the unknown underlying utility function $v$ and the known utility $u$ assumed by the regulator. We assume that the mismatch is small and interpret $u$ as the estimator of $v$.

The following proposition mirrors Proposition 8.

Proposition 13** (Full version of Proposition 8).**

Fix $\delta>0$ and assume that the bank has access to a sample of $(X,A,Y,\alpha_{\pm},v)$ and to estimators $u={{u}}(x,a)$ of $v$ and $\hat{r}=\hat{r}(x,a)$ of $r$ such that $\mathbb{E}\left[\left|{{\overline{u}}}-\overline{v}\right|\right]\leq\eta_{u}\cdot\max_{x,a}\overline{v}(x,a)$ and $\mathbb{E}\left[\left|\hat{r}-r\right|\right]\leq\eta_{r}\cdot\max_{x,a}|{r}(x,a)|$. Let $\varepsilon$ be such that $\varepsilon>2\sqrt{6\left(\frac{1}{\mathbb{P}(A=0)}+\frac{1}{\mathbb{P}(A=1)}\max\{\eta_{u},\eta_{r}\}\right)}$ that also satisfies the upper bound in Inequality (8). Then, an \Big{(}\varepsilon\cdot\max_{x,a}|{r}(x,a)|,\ \ \varepsilon\cdot\max_{x,a}\overline{v}(x,a)\Big{)} bank-optimal $v$-WE classifier can be computed with probability of at least $1-\delta$ on a sample of size $O\left(\frac{1}{\varepsilon^{2}}\left(\log\frac{1}{\delta}+\log\log\frac{1}{\varepsilon}\right)\right)$.

The proof of the proposition is deferred to the end of the subsection, and it relies on three auxiliary propositions. First, in Proposition 16 we prove that having a sample access to the distribution is enough for constructing an $(\varepsilon,\varepsilon)$ bank-optimal $v$-WE classifier of a particular functional form.

Definition 14**.**

A classifier $c$ is a classifier in the reduced form151515Note that both bank-optimal and $\varepsilon$-bank-optimal classifiers from Propositions 2 and 11 have a reduced form. if $c=c(r(x,a),\,\overline{v}(x,a))$ (i.e., $c$ depends on $x$ and $a$ only through $r(x,a)$ and $\overline{v}(x,a)$).

Note that in order to compute the outcome of a reduced form classifier, one must know the exact values of $r$ and $v$; we use their approximations instead. However, the classifier from Proposition 16 turns out to be sensitive to small errors in $r$ and ${v}$. In particular, using the estimator $u$ instead of the exact underlying utility $v$ can dramatically change the outcome. In Proposition 17, we describe the smoothing technique that allows us to ensure that the outcome of classification is robust to small perturbations of $r$ and $v$ in the following sense.

Definition 15**.**

A reduced-form classifier $c$ is $\varkappa$-robust if

[TABLE]

for all numbers $s,s^{\prime}\in\mathbb{R}$ and $w,w^{\prime}\in\mathbb{R}_{+}$.

Finally, Proposition 18 bounds the losses in revenue and welfare of a $\varkappa$-robust classifier that relies on estimators of $v$ and $r$ instead of the exact values.

Proposition 16**.**

For $\delta>0$ and $\varepsilon>0$ satisfying Inequality (8), a reduced form of an $\left(\varepsilon\cdot\max_{x,a}|{r}(x,a)|,\varepsilon\cdot\max_{x,a}\overline{v}(x,a)\right)$ bank-optimal $v$-WE classifier can be computed with probability of at least $1-\delta$ on a sample of size $O\left(\frac{1}{\varepsilon^{2}}\left(\log\frac{1}{\delta}+\log\log\frac{1}{\varepsilon}\right)\right)$.

Proof of Proposition 16.

We mimic the algorithm from Proposition 11 but change all the exact expectations to empirical averages.

By re-scaling, we assume $\max_{x,a}|r(x,a)|=\max_{x,a}\overline{v}(x,a)=1$. The algorithm from Proposition 11 computes the following conditional expectations $O\left(\log^{2}\frac{1}{\varepsilon}\right)$ times: welfares $\mathbb{E}[{v}(X,A,Y)c_{\lambda}(X)\mid A=a]$ and revenues $\mathbb{E}[r(X,A)c_{\lambda}(X)\mid A=a]=\mathbb{E}[(\alpha_{+}(X)Y-\alpha_{-}(X)(1-Y))c_{\lambda}(X)\mid A=a]$. By the Chernoff bound, for i.i.d. random variables $\xi_{1},...,\xi_{K}$ in $[-1,1]$ we have $\mathbb{P}\left(\left|\mathbb{E}\xi_{1}-\frac{1}{K}\sum_{k=1}^{K}\xi_{k}\right|>\varepsilon\right)\leq 2\exp\left(-\frac{K\varepsilon^{2}}{2}\right)$. Hence, we ensure that each particular expectation is computed with accuracy $\varepsilon$ with probability $1-\delta^{\prime}$ using a sample of size $O\left(\frac{1}{\varepsilon^{2}}\log\frac{1}{\delta^{\prime}}\right)$ conditional on $\{A=a\}$. In order to have such a conditional sample, the unconditional sample must be $\frac{1}{\mathbb{P}(A=a)}$-times bigger. By the union bound, selecting $\delta^{\prime}$ such that $\delta^{\prime}\cdot O\left(\log^{2}\frac{1}{\varepsilon}\right)\leq\delta$ we guarantee that with probability of at least $1-\delta$ all $O\left(\log^{2}\frac{1}{\varepsilon}\right)$ expectations are computed with precision $\varepsilon$ throughout the run of the algorithm. ∎

Proposition 17**.**

The result of Proposition 16 remain true if we additionally ask the computed classifier to be $\frac{\varepsilon}{3}$-robust.

Proof of Proposition 17.

We describe the smoothing technique as usual assuming $\max_{x,a}|r(x,a)|=\max_{x,a}|\overline{v}(x,a)|=1$. To compute an $\frac{\varepsilon}{3}$-robust classifier $c$, we consider an auxiliary “smoothed” classification problem with $x^{\circ}=(x,\xi_{v},\xi_{r})$, where $\xi_{v},\xi_{r}$ are uniformly distributed on $[0,\varepsilon/3]$ and are independent from each other and from $(x,a)$, and define $\overline{v}^{\circ}(x^{\circ},a)=\overline{v}(x,a)+\xi_{v}$ and $r^{\circ}(x^{\prime},a)=r(x,a)+\xi_{r}$.

Next, we compute the reduced form of an $\left(\varepsilon/3,\varepsilon/3\right)$ bank-optimal ${v}^{\circ}$-WE classifier $c^{\circ}$ in the new problem and define a reduced-form classifier in the original problem (under $v$) by c(r,\overline{v})=\mathbb{E}_{(\xi_{v},\xi_{r})}c^{\circ}\big{(}r+\xi_{r},\overline{v}+\xi_{v}\big{)}. If we slightly perturb $r$ and $\overline{v}$, then $c^{\circ}$ is integrated over almost-coinciding squares in the $(r,\overline{v})$-plane and hence $|c(r,\overline{v})-c(r^{\prime},\overline{v}^{\prime})|\leq\frac{|r-r^{\prime}|+|\overline{v}-\overline{v}^{\prime}|}{\varepsilon/3}$, i.e., we get robustness. The constructed classifier $c$ equalizes welfare up to $2\varepsilon/3\leq\varepsilon$ since $v$ and $v^{\circ}$ differ by at most $\varepsilon/3$.

It remains to check that $c$ generates $\varepsilon$-optimal revenue. The bank-optimal classifier $c^{*}_{\mathrm{WE}(v)}$ in the original problem induces a classifier that equalizes welfare up to $\varepsilon/3$ and has revenue $R^{\circ}(c^{*}_{\mathrm{WE}(v)})\geq R(c^{*}_{\mathrm{WE}(v)})-\varepsilon/3$ in the new problem. We get $R^{\circ}(c^{\circ})+\varepsilon/3\geq R^{\circ}(c^{*}_{\mathrm{WE}(v)})\geq R(c^{*}_{\mathrm{WE}(v)})-\varepsilon/3$. Since $R(c)\geq R^{\circ}(c^{\circ})-\varepsilon/3$, we obtain that $c$ generates $\varepsilon$-optimal revenue in the original problem. ∎

Proposition 18**.**

Suppose we have estimators ${{u}}$ and $\hat{{r}}$ such that $\mathbb{E}\left[|{\overline{u}}-\overline{v}|\right]\leq\eta_{u}\cdot\max_{x,a}v(x,a)$ and $\mathbb{E}\left[|\hat{r}-r|\right]\leq\eta_{r}\cdot\max_{x,a}|r(x,a)|$. Further, let $c$ be a reduced form of a $\varkappa$-robust $(\varepsilon,\varepsilon^{\prime})$ bank-optimal WE classifier and define $c^{\prime}$ as the version of $c$ that uses the estimators ${{u}}$ and $\hat{r}$, i.e., $c^{\prime}(x,a)=c(\hat{r},{\overline{u}})$. Then, $c^{\prime}$ is $\left(\hat{\varepsilon},\hat{\varepsilon^{\prime}}\right)$ bank-optimal with

[TABLE]

and

[TABLE]

Proof of Proposition 18.

By $\varkappa$-robustness we get

[TABLE]

thus, by applying the triangle inequality we get

[TABLE]

The argument for the revenue is similar but simpler since we need unconditional expectations only. ∎

We are now ready to prove Proposition 13.

Proof of Proposition 13.

As usual assume $\max_{x,a}\overline{v}(x,a)=\max_{x,a}{r}(x,a)=1$. By Proposition 17, we can compute an $\frac{\varepsilon}{6}$-robust $\left(\frac{\varepsilon}{2},\frac{\varepsilon}{2}\right)$ bank-optimal WE classifier $c$ in the reduced form on a sample of required size. The lower bound on $\varepsilon$ is chosen in such a way that $\frac{6}{\varepsilon}(\eta_{u}+\eta_{r})\left(\frac{1}{\mathbb{P}(A=0)}+\frac{1}{\mathbb{P}(A=1)}\right)\leq\frac{\varepsilon}{2}$ and thus, by Proposition 18, the classifier $c(\hat{r}(x,a),{\overline{u}}(x,a))$ is an $(\varepsilon,\varepsilon)$ bank-optimal WE classifier. ∎