Privacy-preserving Crowd-guided AI Decision-making in Ethical Dilemmas

TL;DR

This paper introduces a novel privacy-preserving method for aggregating moral preferences in AI ethical decision-making, ensuring high accuracy while safeguarding voter privacy through differential privacy techniques.

Contribution

It proposes four new algorithms for privacy-preserving preference aggregation in ethical AI, balancing privacy protection with decision accuracy.

Findings

High accuracy in preference aggregation with privacy guarantees

Effective protection of individual voter privacy in ethical dilemmas

Applicable to both synthetic and real-world data sets

Abstract

With the rapid development of artificial intelligence (AI), ethical issues surrounding AI have attracted increasing attention. In particular, autonomous vehicles may face moral dilemmas in accident scenarios, such as staying the course resulting in hurting pedestrians or swerving leading to hurting passengers. To investigate such ethical dilemmas, recent studies have adopted preference aggregation, in which each voter expresses her/his preferences over decisions for the possible ethical dilemma scenarios, and a centralized system aggregates these preferences to obtain the winning decision. Although a useful methodology for building ethical AI systems, such an approach can potentially violate the privacy of voters since moral preferences are sensitive information and their disclosure can be exploited by malicious parties. In this paper, we report a first-of-its-kind privacy-preserving crowd-guided AI decision-making approach in ethical dilemmas. We adopt the notion of differential privacy to quantify privacy and consider four granularities of privacy protection by taking voter-/record-level privacy protection and centralized/distributed perturbation into account, resulting in four approaches VLCP, RLCP, VLDP, and RLDP. Moreover, we propose different algorithms to achieve these privacy protection granularities, while retaining the accuracy of the learned moral preference model. Specifically, VLCP and RLCP are implemented with the data aggregator setting a universal privacy parameter and perturbing the averaged moral preference to protect the privacy of voters' data. VLDP and RLDP are implemented in such a way that each voter perturbs her/his local moral preference with a personalized privacy parameter. Extensive experiments on both synthetic and real data demonstrate that the proposed approach can achieve high accuracy of preference aggregation while protecting individual voter's privacy.

Equations

$$\mathbb{P}\left[Y(D)\in \mathcal{Y}\right]\le e^{ϵ}\times \mathbb{P}\left[Y(D^{\prime })\in \mathcal{Y}\right],$$

$$\mathcal{L}({\mathbf{\beta }}_i,D_i):=\sum _{j=1}^n\ln \Phi ({\mathbf{\beta }}_i^{\top }(X_j^{(i)}-Z_j^{(i)})).$$

$${\overline{\mathbf{\beta }}}_i(D_i):=\underset{{\mathbf{\beta }}_i:\Vert {\mathbf{\beta }}_i{\Vert }_1\le B}{\mathrm{argmax}}\mathcal{L}({\mathbf{\beta }}_i,D_i).$$

$${\Delta }_{VLCP}$$

$${\Delta }_{RLCP}$$

$$\mathbb{P}\left[{\Vert {\mathbf{\beta }}^{\ast }-\overline{\mathbf{\beta }}\Vert }_{\infty }\ge \alpha \right]=\mathbb{P}\left[{\Vert \mathbf{R}\Vert }_{\infty }\ge \alpha \right]$$

$$=\mathbb{P}\left[\left(\underset{k\in \{1,2,\dots ,d\}}{\max }|R[k]|\right)\ge \alpha \right]\le \sum _{k\in \{1,2,\dots ,d\}}\mathbb{P}\left[|R[k]|\ge \alpha \right]$$

$$=d\cdot \mathbb{P}\left[|Lap(\frac{2B}{Nϵ})|\ge \alpha \right]=\gamma \text{ for α:=Nϵ2B⋅ln(d/γ)},$$

$${\Delta }_{VLDP}^{(i)}$$

$${\Delta }_{RLDP}^{(i)}$$

$$\mathcal{L}({\mathbf{\beta }}_i,V_j^{(i)}{|}_{j\in \{1,2,\dots ,n\}})={\sum }_{j=1}^n\ln \Phi ({\mathbf{\beta }}_i^{\top }{V}_j^{(i)}).$$

$$f(V_j,\mathbf{\beta })=\ln \Phi ({\mathbf{\beta }}^{\top }{V}_j).$$

$${\mathbf{\beta }}^o=\underset{\mathbf{\beta }}{\mathrm{argmax}}{\sum }_{j=1}^{n}f(V_j,\mathbf{\beta }).$$

$$f(V_j,\mathbf{\beta })={\sum }_{w=0}^W{\sum }_{\varphi \in {\Psi }_w}{\lambda }_{{\varphi }_{,V_j}}\varphi (\mathbf{\beta }),$$

$$g_1(V_j,\mathbf{\beta })={\mathbf{\beta }}^{\top }{V}_j\text{ and }{f}_1(z)=\ln \Phi (z).$$

$${\tilde{f}}_D(\mathbf{\beta })={\sum }_{j=1}^n{\sum }_{k=0}^{\infty }\frac{f_1^{(k)}(0)}{k!}({\mathbf{\beta }}^{\top }{V}_j{)}^k.$$

$${\hat{f}}_D(\mathbf{\beta }):={\sum }_{j=1}^n{\sum }_{k=0}^2\frac{f_1^{(k)}(0)}{k!}({\mathbf{\beta }}^{\top }{V}_j{)}^k.$$

$$\Delta$$

$$=2\cdot \underset{V\in D}{\max }\left(\frac{f_1^1(0)}{1!}{\sum }_{k=1}^{d}V[k]+\frac{f_1^2(0)}{2!}{\sum }_{1\le k,l\le d}V[k]V[l]\right)$$

$$\le 2\left(\sqrt{2d/\pi }+d/\pi \right),\text{which we denote by Δupper below.}$$

Full text

Artificial intelligence; Ethical decision making; Differential privacy

††ccs: Security and privacy Data anonymization and sanitization††ccs: Security and privacy Privacy protections††ccs: Security and privacy Human and societal aspects of security and privacy

1. Introduction

Artificial intelligence (AI) is becoming an integral part of our daily lives and critical infrastructures. With the widespread applications of AI, ethical issues surrounding AI have become an important socio-technical challenge (Wallach and Allen, 2008; Awad et al., 2018). One of the fundamental questions in AI ethics is how to allow humans to guide AI to make moral decisions when faced with ethical dilemmas. An ethical dilemma is a situation in which any decision violates certain aspects of ethics (Yu et al., 2018). The following is a concrete example of an ethical dilemma that autonomous vehicles encounter (Greene, 2016). A self-driving car has a sudden mechanical failure and cannot brake in time. If the car were to continue its current trajectory, it would kill pedestrians but the passenger would be safe. Alternatively, if it were to swerve into a wall, the pedestrians would be safe but the passenger would be killed. In such a situation, an ethical dilemma arises when the AI must choose between the two alternatives. This is just one example of many ethical dilemmas AI technologies face (Shariff et al., 2017). To enable AI to deal with such situations, it is useful to aggregate opinions from the human society.

To explore moral dilemmas faced by autonomous vehicles, Bonnefon et al. (2016) surveyed via Amazon Mechanical Turk whether people prefer to save many versus few lives, or the vehicle’s passengers versus pedestrians. Their findings suggest that participants prefer others to buy autonomous vehicles which sacrifice their passengers for the greater good, but prefer for themselves to ride in autonomous vehicles which protect passengers at all costs.

In a study similar to (Bonnefon et al., 2016), Awad et al. (2018) also gathered data about voters’ decisions in scenarios where autonomous vehicles face ethical dilemmas, but at a much larger scale and with each voter’s data containing more dimensions. Specifically, they built an online experimental platform named the Moral Machine (Mor, [n. d.]), which collected 39.61 million decisions in ten languages from 4 million people in 233 countries and territories. The data collected from each voter are high-dimensional, which include preferences of saving many versus few lives, passengers versus pedestrians, the young versus the elderly, humans versus pets, and pedestrians who cross legally versus pedestrians who jaywalk, etc. With such a high volume of crowdsourced data, Awad et al. (2018) summarized global and regional moral preferences as well as cross-cultural ethical variations.

Using the data collected from the Moral Machine, Noothigattu et al. (2018) built a global moral preference model so as to guide automated ethical decision-making. Specifically, each voter’s data are analyzed to infer her/his parameter of moral preference, and the moral preference of the society is obtained by averaging voters’ parameters. Each voter’s data consist of a number of records, with each record being the voter’s preference in a given ethical dilemma scenario. In each scenario, no matter how the autonomous vehicle decides to act, some undesirable outcome will happen. A voter’s preference in each scenario means that the voter prefers a decision made by the autonomous vehicle (e.g., staying the course and killing the crossing pedestrians) over the other alternative (e.g., swerving and killing the passengers).

As discussed above, existing studies on AI ethics (e.g., (Bonnefon et al., 2016; Awad et al., 2018; Noothigattu et al., 2018)) directly analyze voters’ data of moral decisions. However, such approaches may violate the privacy of voters. Since moral preferences are sensitive information and their disclosure may be exploited by malicious parties to produce negative consequences. One may wonder how the learned and aggregated moral preference model for the society can leak a voter’s sensitive choices about the moral dilemmas. A recent work (Fredrikson et al., 2015) has shown that an adversary can successfully use a learned deep neural network model to infer users’ sensitive facial information in the training dataset. Specifically, for an attacker which obtains the learned face recognition model, although it is difficult to infer all users’ faces, the attacker may recover an image which is close to one user’s face, resulting in a privacy breach of the particular user in the training dataset (Fredrikson et al., 2015).

In this paper, to prevent the learned preference model of the society from leaking individual voter’s sensitive information, we propose a differential privacy (DP)-based secure preference aggregation model to enable crowdsourced opinions to guide AI decision-making in ethical dilemmas without exposing sensitive privacy information. We adopt the formal notion of DP (Dwork and Roth, 2014; Dwork et al., 2006a) to quantify privacy. Intuitively, by incorporating some noise, the output of an algorithm under DP will not change significantly due to the presence or absence of one voter’s information in the dataset.

Contributions. With the proposed approach, we make the following contributions in this paper.

• •

We quantify four granularities of privacy protection by combing voter-/record-level privacy protection and centralized/distributed perturbation, which are denoted by VLCP, RLCP, VLDP, and RLDP. We further propose different algorithms to achieve them. Specifically, to achieve VLCP and RLCP, the aggregator adds Laplace noise to the average preference parameter of all voters. VLDP and RLDP are achieved by having each voter adding Laplace noise to her/his local moral preference parameter with a personalized privacy parameter. Moreover, we also propose to achieve RLDP by perturbing the objective function when learning each voter’s preference parameters, which achieves higher accuracy than the addition of Laplace noise.

• •

We conduct extensive experiments on both synthetic datasets and a real-world dataset extracted from the Moral Machine. The results demonstrate that our algorithms can achieve high accuracy while ensuring strong privacy protection.

To the best of our knowledge, this is the first research on privacy issues in studies of human-guided ethical AI decision-making. More specifically, we proposed privacy-preserving mechanisms to address voters’ privacy in computing a society’s moral preference.

Organization. The remainder of the paper is organized as follows. Section 2 reviews the related studies of AI ethics and privacy protection. In Section 3, we discuss the preliminaries of differential privacy and formalize the research problem. Section 4 presents the proposed privacy-preserving crowd-guided ethical AI decision-making algorithms. In Section 5, we conduct extensive experiments to evaluate the effectiveness of our algorithms. Section 6 provides discussions and future directions. Section 7 concludes the paper.

2. Related Work

The widespread adoption of AI has made it pertinent to address the ethical issues surrounding this technology.

Greene et al. (2016) advocated solving AI ethical issues via preference aggregation, in which each voter expresses her/his preferences over the possible decisions, and a centralized system aggregates these preferences to obtain the winning decision. Conitzer et al. (2017) discussed the idea of collecting a labeled dataset of moral dilemmas represented as lists of feature values, and then leveraging machine learning techniques to learn to classify actions as morally right or wrong. However, no actual data collection or analysis was presented. Using a large-scale dataset of voters’ ethical decisions collected by the Moral Machine (2018), Noothigattu et al. (2018) built a moral preference model for each voter and averaged these models to obtain the moral preference of the whole society. Hammond and Belle (2018) utilized tractable probabilistic learning to induce models of moral scenarios and blameworthiness automatically from datasets of human decision-making, and computed judgments tractably from the obtained models. Zhang and Conitzer (2019) showed that many classical results from Probably Approximately Correct (PAC) learning can be applied to the preference aggregation framework. For a comprehensive study of more related work on AI ethics, interested readers can refer to a recent survey by Yu et al. (2018).

The above studies significantly contributed to the emerging area of AI ethics, but unfortunately, none of them is designed to protect voters’ privacy in the data analysis process. Moral preferences of voters are sensitive information and should be well protected, since their disclosure can be exploited by malicious entities to have adverse consequences. A recent work (Fredrikson et al., 2015) has shown that even a learned deep neural network can leak sensitive information in the training dataset. Hence, it is critical to incorporate formal privacy protection into models for AI ethics guided by humans.

To quantify privacy, we use the rigorous notion of differential privacy (DP) (Dwork and Roth, 2014). Recently, differential privacy has been widely studied in many areas (Abadi et al., 2016). For general statistical data release problems, DP can be achieved by the Laplace mechanism (Dwork et al., 2006b) which injects Laplace noise into the released statistical results. Furthermore, for parameter estimation solved by an optimization problem, an alternative algorithm to achieve DP is the functional mechanism proposed by (Zhang et al., 2012), which perturbs the objective function of the optimization problem rather than the optimized parameters.

As a representative example, this paper adds voter privacy protection into the learned moral preference model of Noothigattu et al. (2018). Incorporating privacy protection into other studies are future directions. Although Jones et al. (2018) and Brubaker (2018) emphasized the importance of security and privacy protection in AI ethics, to the best of our knowledge, our paper is the first technical work to formally build privacy protection into the study of AI ethics.

3. Preliminaries

Differential privacy (DP) (Dwork et al., 2006a; Dwork and Roth, 2014) provides strong guarantees for the privacy of any individual in a query response, regardless of the adversary’s prior knowledge.

Definition 1 ($\epsilon$-Differential Privacy (DP) (Dwork and Roth, 2014; Dwork et al., 2006a)).

A randomized algorithm $Y$ satisfies $\epsilon$-differential privacy, if for any two neighboring datasets $D$ and $D^{\prime}$ which differ in only one tuple, and for any possible subset of outputs $\mathcal{Y}$ of $Y$, we have

[TABLE]

where ${\mathbb{P}}\left[{\cdot}\right]$ is the probability of an event. $\epsilon$ refers to the privacy parameter. Smaller $\epsilon$ means stronger privacy protection, but less utility as more randomness is introduced into $Y$.

There are two variants of DP: bounded DP and unbounded DP (Tramèr et al., 2015). In bounded DP, which we adopt in this paper, two neighboring datasets have the same sizes but different records at only one of all positions. In unbounded DP, the sizes of two neighboring datasets differ by one (i.e., one tuple is in one database, but not in the other).

The Laplace Mechanism (Dwork et al., 2006b) can be used to achieve DP by adding independent Laplace noise to each dimension of the query output. The scale $\lambda$ of the zero-mean Laplace noise $Lap(\lambda)$ is set as $\lambda=\Delta/\epsilon$, where $\Delta$ is the $\ell_{1}$-norm sensitivity of the query function $Q$, which measures the maximum change of the outputs over neighboring datasets (i.e. \Delta=\max_{\textrm{neighboring datasets D,D^{\prime}}}\|Q(D)-Q(D^{\prime})\|_{1}).

Problem Formulation. In our system model, each voter $i\in\{1,2,\cdots,N\}$ owns a dataset of $n$ pairwise comparisons (i.e., $n$ records), denoted by $D_{i}=\{\langle X_{1}^{(i)},Z_{1}^{(i)}\rangle,\langle X_{2}^{(i)},Z_{2}^{(i)}\rangle,\cdots,\langle X_{n}^{(i)},Z_{n}^{(i)}\rangle$ }. $X_{j}^{(i)}$ and $Z_{j}^{(i)}$ are pairwise alternatives capturing moral dilemmas in a scenario (e.g., $X_{j}^{(i)}$ means staying the course resulting in killing crossing pedestrians, while $Z_{j}^{(i)}$ means swerving leading to killing passengers). Any pair $\langle X_{j}^{(i)},Z_{j}^{(i)}\rangle$ for $j\in\{1,2,\cdots,n\}$ means that voter $i$ chose $X_{j}^{(i)}$ over $Z_{j}^{(i)}$. Each $X_{j}^{(i)}$ or $Z_{j}^{(i)}$ is a $d$-dimensional vector such that $X_{j}^{(i)}[k]$ and $Z_{j}^{(i)}[k]$ for $k\in\{1,2,\cdots,d\}$ denote the $k$-th dimensional value of $X_{j}^{(i)}$ and $Z_{j}^{(i)}$, respectively. The dimension of a scenario represents its features (e.g., the number of young passengers, the number of old pedestrians, the number of pets, etc).

Noothigattu* et al. (2018)* adopted the Thurstone–Mosteller process (Mosteller, 2006), which models the utility as a Gaussian random variable. Let $\bm{\beta}_{i}\in\mathbb{R}^{d}$ be the preference parameter of voter $i$, where $\mathbb{R}$ denotes the set of real numbers. It is assumed (Mosteller, 2006) that the utilities of alternatives $X_{j}^{(i)}$ and $Z_{j}^{(i)}$ follow Gaussian distributions $\mathcal{N}(\bm{\beta}_{i}^{\top}X_{j}^{(i)},\frac{1}{2})$ and $\mathcal{N}(\bm{\beta}_{i}^{\top}Z_{j}^{(i)},\frac{1}{2})$, respectively. Thus, the result of the utility by choosing $X_{j}^{(i)}$ minus the utility by choosing $Z_{j}^{(i)}$ follows a Gaussian distribution $\mathcal{N}(\bm{\beta}_{i}^{\top}(X_{j}^{(i)}-Z_{j}^{(i)}),1)$, so that

{\mathbb{P}}\left[{\text{a voter i$chooses$X_{j}^{(i)}$over$Z_{j}^{(i)}}}\right]=\Phi(\bm{\beta}_{i}^{\top}(X_{j}^{(i)}-Z_{j}^{(i)})), where $\Phi$ is the cumulative distribution function of the standard normal distribution (i.e. $\Phi(s):=\frac{1}{2}+\frac{1}{\sqrt{\pi}}\int_{0}^{\frac{s}{\sqrt{2}}}e^{-{t}^{2}}\hskip 1.0pt\emph{d}t$). Then, the maximum likelihood estimation (MLE) method is used to learn the parameter $\bm{\beta}_{i}$ for each voter $i\in\{1,2,\cdots,N\}$. In particular, the log-likelihood function is defined as follows:

[TABLE]

Based on Eq. (2), MLE is used to estimate the $\bm{\beta}_{i}$ of each voter (Noothigattu et al., 2018). However, the optimal parameter $\bm{\beta}_{i}$ to maximize Eq. (2) may not always exist. For example, given $i\in\{1,2,\cdots,N\}$, if $X_{j}^{(i)}-Z_{j}^{(i)}$ is positive for each $j\in\{1,2,\cdots,n\}$, suppose there exists $\bm{\beta}_{i}$ to maximize Eq. (2), then each dimension of $\bm{\beta}_{i}$ is positive. However, a contraction occurs with $\mathcal{L}(2\bm{\beta}_{i},D_{i})>\mathcal{L}(\bm{\beta}_{i},D_{i})$. Hence, in the case of positive $X_{j}^{(i)}-Z_{j}^{(i)}$ for all $j\in\{1,2,\cdots,n\}$, no $\bm{\beta}_{i}$ exists to maximize Eq. (2). In order to ensure that (i) the optimal parameter can always be found, and (ii) bounded sensitivity that will be used in the Laplace mechanism for achieving differential privacy, we introduce a constraint that each voter’s parameter has an $\ell_{1}$-norm at most $B$. Specifically, we define user $i$’s parameter $\overline{\bm{\beta}}_{i}(D_{i})$ by

[TABLE]

After learning the parameters $\overline{\bm{\beta}}_{i}(D_{i})$ for all voters, the preference parameter ${\overline{\bm{\beta}}}(D)$ for the whole society is computed by averaging all voters’ preference parameters (i.e. ${\overline{\bm{\beta}}}(D):=\frac{1}{N}\sum_{i=1}^{N}\overline{\bm{\beta}}_{i}(D_{i})$, for $D:=\{D_{1},D_{2},\cdots,D_{N}\}$ denoting the whole dataset of $N$ voters). Therefore, the purpose of this paper is to design privacy-preserving algorithms to guarantee voter privacy while learning a society’s preference parameter with high accuracy.

4. Our Solutions

This section first introduces four privacy protection paradigms and then presents our algorithms to achieve these paradigms.

4.1. Privacy Modeling

Our paper incorporates privacy protection into preference aggregation of Noothigattu et al. (2018). In this setting, each voter’s data consist of a number of records, with each record being the voter’s preference in a given scenario. In each scenario, no matter how the autonomous vehicle decides to act, someone will get hurt. A voter’s preference in each scenario means that the voter prefers a decision made by the autonomous vehicle (e.g., staying the course and killing crossing pedestrians) over the other alternative (e.g., swerving and killing passengers).

Based on the above observations, in our study of privacy protection for crowdsourced data collection in AI ethics, we will consider two variants for the meaning of neighboring datasets: 1) record-neighboring and 2) voter-neighboring datasets, which allow us to achieve voter-level privacy protection and record-level privacy protection, respectively:

• •

Voter-level privacy protection. Two datasets are voter-neighboring datasets if one can be obtained from the other by changing one voter’s records arbitrarily.

• •

Record-level privacy protection. Two datasets are record-neighboring if one can be obtained from the other by changing a single record of one voter.

For completeness, we also consider the case of a trusted aggregator and the case of an untrusted aggregator, where privacy is achieved with centralized perturbation and distributed perturbation, respectively:

• •

Centralized perturbation. When the aggregator is trusted by the voters, the aggregator perturbs the aggregated information (e.g., by adding noise) in a centralized manner to protect privacy. We refer to this as centralized perturbation.

• •

Distributed perturbation. When the aggregator is not trusted by the voters, each voter independently perturbs her/his local data (e.g., by adding noise) in a distributed manner for privacy protection. We refer to this as distributed perturbation.

Permutating centralized/distributed perturbation and voter-level/

record-level privacy protection yields four alternative privacy protection paradigms: (1) voter-level privacy protection with centralized perturbation (VLCP); (2) record-level privacy protection with centralized perturbation (RLCP); (3) voter-level privacy protection with distributed perturbation (VLDP); (4) record-level privacy protection with distributed perturbation (RLDP).

1. (1)

VLCP: For $\epsilon$-differential privacy with VLCP, the aggregator chooses a universal privacy parameter $\epsilon$ and enforces a randomization algorithm $Y$ such that for any two voter-neighboring datasets $D$ and $D^{\prime}$, and for any possible subset of outputs $\mathcal{Y}$ of $Y$, we have ${\mathbb{P}}\left[{Y(D)\in\mathcal{Y}}\right]\leq e^{\epsilon}\times{\mathbb{P}}\left[{Y(D^{\prime})\in\mathcal{Y}}\right]$. 2. (2)

RLCP: For $\epsilon$-differential privacy with RLCP, the aggregator chooses a universal privacy parameter $\epsilon$ and enforces a randomization algorithm $Y$ such that for any two record-neighboring datasets $D$ and $D^{\prime}$, and for any possible subset of outputs $\mathcal{Y}$ of $Y$, we have ${\mathbb{P}}\left[{Y(D)\in\mathcal{Y}}\right]\leq e^{\epsilon}\times{\mathbb{P}}\left[{Y(D^{\prime})\in\mathcal{Y}}\right]$. 3. (3)

VLDP: For $\epsilon_{i}$-differential privacy with VLDP, voter $i$ chooses a privacy parameter $\epsilon_{i}$ and enforces a randomization algorithm $Y_{i}$ such that for any two datasets $D_{i}$ and $D_{i}^{\prime}$ (which are naturally voter-neighboring), and for any possible subset of outputs $\mathcal{Y}_{i}$ of $Y_{i}$, we have ${\mathbb{P}}\left[{Y_{i}(D_{i})\in\mathcal{Y}_{i}}\right]\leq e^{\epsilon_{i}}\times{\mathbb{P}}\left[{Y_{i}(D_{i}^{\prime})\in\mathcal{Y}_{i}}\right]$. Note that VLDP is the same as the notion of $\epsilon$-local differential privacy (Duchi et al., 2013) which has recently received much interest (Wang et al., 2019; Tang et al., 2017; Erlingsson et al., 2014). 4. (4)

RLDP: For $\epsilon_{i}$-differential privacy with VLDP, voter $i$ chooses a privacy parameter $\epsilon_{i}$ and enforces a randomization algorithm $Y_{i}$ such that for any two record-neighboring datasets $D_{i}$ and $D_{i}^{\prime}$, and for any possible subset of outputs $\mathcal{Y}_{i}$ of $Y_{i}$, we have ${\mathbb{P}}\left[{Y_{i}(D_{i})\in\mathcal{Y}_{i}}\right]\leq e^{\epsilon_{i}}\times{\mathbb{P}}\left[{Y_{i}(D_{i}^{\prime})\in\mathcal{Y}_{i}}\right]$.

In what follows, we further disambiguate the above four privacy protection paradigms, and outline how each of them can be realized.

First, VLCP can be achieved with the data aggregator setting a universal privacy parameter $\epsilon$ and perturbing the averaged moral preference by adding Laplace noise to protect the privacy of each voter’s complete data. Second, RLCP can be achieved in the same way as VLCP when using Laplace mechanism. This is because the $\ell_{1}$ sensitivity of the averaged parameter under RLCP is the same as that under VLCP, which will be proved in Theorem 1. Third, VLDP is a strong privacy protection paradigm and can be achieved by perturbing each voter’s moral preference by adding Laplace noise to protect each record in the dataset. Each voter $i$ can choose a personalized privacy parameter $\epsilon_{i}$ to perturb her/his moral preference accordingly, and report the noisy moral preference to the aggregator. Finally, RLDP can be achieved by the same way as VLDP when using Laplace mechanism. The reason is that the $\ell_{1}$ sensitivity of each voter’s parameter under RLDP is the same as that under VLDP, which will be proved in Theorem 3.

However, achieving RLDP by adding Laplace noise directly leads to limited utility even for weak privacy protection, as illustrated in our experiments in Section 5. Therefore, to pursuing better utility, we propose to adopt the functional mechanism (Zhang et al., 2012), which enforces differential privacy by perturbing the object function of the optimization problem, rather than the optimized parameters (i.e., each voter’s moral preference). Note that we can’t adopt the functional mechanism to estimate the social preference parameter under voter-level privacy protection or centralized perturbation. This is because the social preference parameter is derived by averaging the preference parameters of all voters instead of the solution of the optimization problem. The functional mechanism itself is used for analyzing and solving the optimization problem.

Thus, we will consider the above four privacy protection paradigms to achieve differential privacy for ethical decision making of AI and will propose three algorithms which cover the above four paradigms. Specifically, the three algorithms are outlined as follows.

• •

VLCP/RLCP algorithm via perturbing the average moral preference parameters of all voters by Laplace mechanism.

• •

VLDP/RLDP algorithm via perturbing the moral preference parameters of each voter by Laplace mechanism.

• •

RLDP algorithm via perturbing the object function of each voter.

In the following subsections, we will introduce the proposed algorithms in detail.

4.2. VLCP/RLCP: Perturbing the Average Moral Preference Parameters

In this section, we propose an algorithm to achieve VLCP and RLCP by perturbing the average moral preference parameters estimated from maximum likelihood estimation. Each voter $i$ obtains its parameter $\overline{\bm{\beta}}_{i}(D_{i})$ according to Eq. (3), which enforces the $\ell_{1}$-norm of $\overline{\bm{\beta}}_{i}(D_{i})$ to be at most $B$. Then each voter $i$ sends it to the aggregator, and the aggregator computes the average estimate ${\overline{\bm{\beta}}}(D):=\frac{1}{N}\sum_{i=1}^{N}\overline{\bm{\beta}}_{i}(D_{i})$, for $D:=\{D_{1},D_{2},\cdots,D_{N}\}$. The $\ell_{1}$- sensitivity of $\overline{\bm{\beta}}$ with respect to neighboring datasets equals $2B/N$, since the maximal range that $\overline{\bm{\beta}}_{i}(D_{i})$ can change is no greater than $2B$ by the triangle inequality (see Theorem 1 for specific proofs). As shown in Algorithm 1, after computing the average parameters of all voters, a random noise vector $R$ will be drawn from the Laplace distribution $[Lap(\frac{2B}{N\epsilon})]^{d}$. Then, the perturbed preference parameter $\bm{\beta}^{*}(D)={\overline{\bm{\beta}}}(D)+\bm{R}$ is returned as the final social moral preference parameter. Note that we may suppress the argument $D_{i}$ in $\overline{\bm{\beta}}_{i}(D_{i})$ and the argument $D$ in ${\overline{\bm{\beta}}}(D),\bm{\beta}^{*}(D)$ for simplicity.

Theorem 1 (Privacy of Algorithm 1).

Algorithm 1 satisfies $\epsilon$-differential privacy for both VLCP and RLCP.

Proof.

First, we prove that the sensitivities $\Delta_{VLCP}$ and $\Delta_{RLCP}$ with respect to voter-neighboring datasets and record-neighboring datasets are both $2B/N$ under centralized perturbation, where

[TABLE]

Let $D=\{D_{1},\cdots,D_{i},\cdots,D_{N}\}$ and $D^{\prime}=\{D_{1},\cdots,D^{\prime}_{i},\cdots,D_{N}\}$ be record-neighboring datasets such that $D$ and $D^{\prime}$ differ in only a single voter $i$’s one record, i.e., $D_{i}$ and $D^{\prime}_{i}$. We assume $D_{i}:=\{\langle X_{1}^{(i)},Z_{1}^{(i)}\rangle,\langle X_{2}^{(i)},Z_{2}^{(i)}\rangle,\cdots,\langle X_{n}^{(i)},Z_{n}^{(i)}\rangle$ }, where each dimension of $X_{1}^{(i)}-Z_{1}^{(i)}$ is $1$, and each dimension of $X_{j}^{(i)}-Z_{j}^{(i)}$ for $j\in\{2,\ldots,n\}$ is close to [math], and $D_{i}^{\prime}:=\{\langle{X_{1}^{(i)}}^{\prime},{Z_{1}^{(i)}}^{\prime}\rangle,\langle X_{2}^{(i)},Z_{2}^{(i)}\rangle,\cdots,\langle X_{n}^{(i)},Z_{n}^{(i)}\rangle$ }, where each dimension of $X_{1}^{(i)}-Z_{1}^{(i)}$ is $-1$, and each dimension of $X_{j}^{(i)}-Z_{j}^{(i)}$ for $j\in\{2,\ldots,n\}$ is close to [math]. Therefore, based on Eq. (3), we know that each of the $d$ dimensions of $\overline{\bm{\beta}}_{i}(D_{i})$ and $\overline{\bm{\beta}}_{i}(D_{i}^{\prime})$ are close to $\frac{B}{d}$ and $-\frac{B}{d}$, respectively. Given ${\overline{\bm{\beta}}}(D):=\frac{1}{N}\sum_{i=1}^{N}\overline{\bm{\beta}}_{i}(D_{i})$ and ${\overline{\bm{\beta}}}(D^{\prime}):=\frac{1}{N}\overline{\bm{\beta}}_{i}(D_{i}^{\prime})+\frac{1}{N}\sum_{j\in\{1,2,\ldots,N\}\setminus\{i\}}\overline{\bm{\beta}}_{j}(D_{j})$, the $\ell_{1}$-norm difference between ${\overline{\bm{\beta}}}(D)$ and ${\overline{\bm{\beta}}}(D^{\prime})$ equals $\frac{1}{N}\overline{\bm{\beta}}_{i}(D_{i})-\frac{1}{N}\overline{\bm{\beta}}_{i}(D_{i}^{\prime})$, which can be made to be arbitrarily close to $\frac{2B}{N}$. Hence, the $\ell_{1}$-norm sensitivity of ${\overline{\bm{\beta}}}$ with respect to record-neighboring datasets is $\frac{2B}{N}$.

In a way similar to the above argument, the $\ell_{1}$-norm sensitivity of ${\overline{\bm{\beta}}}$ with respect to voter-neighboring datasets is $\frac{2B}{N}$. For voter-neighboring datasets $D=\{D_{1},\cdots,D_{i},\cdots,D_{N}\}$ and $D^{\prime}=\{D_{1},\cdots,D^{\prime}_{i},\cdots,D_{N}\}$, voter $i$’s dataset variants $D_{i}$ and $D_{i}^{\prime}$ can differ arbitrarily.

Thus, we obtain $\Delta_{RLCP}=\Delta_{VLCP}=2B/N$. Recall that Algorithm 1 considers the centralized perturbation paradigm and adds noises drawn from $[Lap(\frac{2B}{N\epsilon})]^{d}$ to the average parameters of all voters. From Laplace mechanism (Dwork et al., 2006b), it can be proved that Algorithm 1 satisfies $\epsilon$-differential privacy for both VLCP and RLCP. ∎

We now analyze the utility of Algorithm 1. At the end of Algorithm 1, the aggregator obtains the parameter vector of moral preference $\bm{\beta^{*}}$, which can be understood as a noisy version of the true result $\overline{\bm{\beta}}$. We consider the utility of Algorithm 1 by analyzing the probability that the $p$-norm of the estimation error $\bm{\beta^{*}}-\overline{\bm{\beta}}$ is no greater than a given quantity $\alpha$. To this end, we note that $\bm{\beta^{*}}-\overline{\bm{\beta}}$ follows a $d$-dimensional multivariate Laplace distribution, with each dimension being an independent zero-mean Laplace random variable with scale $\frac{2B}{N\epsilon}$. With $h(\bm{\mu})$ denoting the probability density function of random variable $\bm{\beta^{*}}-\overline{\bm{\beta}}$ being a given $\bm{\mu}$, the expression of $h(\bm{\mu})$ involves Bessel function. Hence, for general $p$-norm, it is difficult to compute ${\mathbb{P}}\left[{\left\|\bm{\beta^{*}}-\overline{\bm{\beta}}\right\|_{p}\leq\alpha}\right]=\int_{\bm{\mu}:\,\left\|\bm{\mu}\right\|_{p}\leq\alpha}h(\bm{\mu})\,\text{d}\bm{\mu}$. Below we consider the special case of $\infty$-norm following (Dwork and Roth, 2014), where we use the union bound to present the utility result of Algorithm 1 in the following Theorem 2.

Theorem 2 (Utility of Algorithm 1).

For any $\gamma\in(0,1)$, Algorithm 1 ensures $\left\|\bm{\beta^{*}}-\overline{\bm{\beta}}\right\|_{\infty}\leq\alpha$ with probability at least $1-\gamma\,$ for $\alpha:=\frac{2B}{N\epsilon}\cdot\ln(d/\gamma)$.

Proof.

With $\bm{R}:=\bm{\beta^{*}}-\overline{\bm{\beta}}$, we know that $\bm{R}$ follows the probability distribution $[Lap(\frac{2B}{N\epsilon})]^{d}$. Then denoting the $d$ dimensions of $\bm{R}$ by $R[1],R[2],\ldots,R[d]$, we have

[TABLE]

where the step of “$\leq$” uses the union bound. ∎

Theorem 2 shows that the utility of Algorithm 1 decreases as $d$ increases. This is confirmed by Figure 5 for experimental results.

From Theorem 1, both VLCP and RLCP assume a universal privacy parameter for all voters; i.e., the same privacy protection level for all voters. In the following, we focus on proposing algorithms with distributed perturbation so that voters can choose personalized privacy parameters to achieve different privacy protection levels.

4.3. VLDP/RLDP: Perturbing the Moral Preference Parameter of Each Voter

This section introduces an algorithm to achieve VLDP and RLDP by perturbing the moral preference parameter of each voter under Laplace mechanism. Algorithm 2 shows the pseudo-code of perturbing the preference parameter of each voter $i$ with a personalized privacy parameter $\epsilon_{i}$. Each voter $i$ obtains its parameter $\overline{\bm{\beta}}_{i}(D_{i})$ according to Eq. (3), which enforces the $\ell_{1}$-norm of $\overline{\bm{\beta}}_{i}(D_{i})$ to be at most $B$. Then, the sensitivity of each voter’s parameter $\bm{\beta}_{i}$ will $2B$ since the maximal changing range of $\bm{\beta}_{i}$ is no greater than $2B$ with respect to neighboring datasets (see Theorem 3 for the specific proofs). Then, the parameter of voter $i$ will be perturbed as $\overline{\bm{\beta}}_{i}^{*}(D_{i})=\overline{\bm{\beta}}_{i}(D_{i})+\bm{R}$, where $\bm{R}$ is a random Laplace noise vector drawn from $[Lap(2B/\epsilon_{i})]^{d}$.

After obtaining the noisy parameter of each voter $i$ in a distributed way, the final social moral preference parameter can be computed by averaging all voters’ parameters, that is $\bm{\beta}^{*}(D)=\frac{1}{N}\sum_{i=1}^{N}\overline{\bm{\beta}}_{i}^{*}(D_{i})$ for $D:=\{D_{1},\cdots,D_{i},\cdots,D_{N}\}$. Note that we may suppress the argument $D_{i}$ in $\overline{\bm{\beta}}_{i}(D_{i}),\overline{\bm{\beta}}_{i}^{*}(D_{i})$ and the argument $D$ in $\bm{\beta}^{*}(D)$ for simplicity.

Theorem 3 (Privacy of Algorithm 2).

For each voter $i$, Algorithm 2 satisfies $\epsilon_{i}$-differential privacy for both VLDP and RLDP.

Proof.

First, we prove that for voter $i$, the sensitivities $\Delta_{VLDP}^{(i)}$ and $\Delta_{RLDP}^{(i)}$ with respect to voter-neighboring and record-neighboring datasets are both $2B$ under distributed perturbation, where

[TABLE]

For voter $i$, let $D_{i}$ and $D_{i}^{\prime}$ be two record-neighboring datasets. Specifically, for $D_{i}:=\{\langle X_{1}^{(i)},Z_{1}^{(i)}\rangle,\langle X_{2}^{(i)},Z_{2}^{(i)}\rangle,\cdots,\langle X_{n}^{(i)},Z_{n}^{(i)}\rangle$ }, where each dimension of $X_{1}^{(i)}-Z_{1}^{(i)}$ is $1$, and each dimension of $X_{j}^{(i)}-Z_{j}^{(i)}$ for $j\in\{2,\ldots,n\}$ is close to [math], we know from Eq. (3) that each of the $d$ dimensions of $\overline{\bm{\beta}}_{i}(D_{i})$ is close to $\frac{B}{d}$. For $D_{i}^{\prime}:=\{\langle{X_{1}^{(i)}}^{\prime},{Z_{1}^{(i)}}^{\prime}\rangle,\langle X_{2}^{(i)},Z_{2}^{(i)}\rangle,\cdots,\langle X_{n}^{(i)},Z_{n}^{(i)}\rangle$ }, where each dimension of $X_{1}^{(i)}-Z_{1}^{(i)}$ is $-1$, and each dimension of $X_{j}^{(i)}-Z_{j}^{(i)}$ for $j\in\{2,\ldots,n\}$ is close to [math], we know from Eq. (3) that each of the $d$ dimensions of $\overline{\bm{\beta}}_{i}(D_{i}^{\prime})$ is close to $-\frac{B}{d}$. Hence, the $\ell_{1}$-norm difference between $\overline{\bm{\beta}}_{i}(D_{i})$ and $\overline{\bm{\beta}}_{i}(D_{i}^{\prime})$ can be made to be arbitrarily close to $2B$. Hence, for voter $i$, the $\ell_{1}$-norm sensitivity of $\overline{\bm{\beta}}_{i}$ with respect to record-neighboring datasets is $2B$.

In a way similar to the above argument, the $\ell_{1}$-norm sensitivity of $\overline{\bm{\beta}}_{i}$ with respect to voter-neighboring datasets is $2B$. Note that voter $i$’s voter-neighboring datasets $D_{i}$ and $D_{i}^{\prime}$ can differ arbitrarily.

Therefore, we obtain $\Delta_{VLDP}^{(i)}=\Delta_{RLDP}^{(i)}=2B$. Since Algorithm 2 adds noises drawn from $[Lap(2B/\epsilon_{i})]^{d}$ to the parameter of each voter $i$, thus it satisfies $\epsilon_{i}$-differential privacy for both VLDP and RLDP based on Laplace mechanism (Dwork et al., 2006b). ∎

Theorem 4 (Utility of Algorithm 2).

For any $\gamma\in(0,1)$, Algorithm 2 ensures $\left\|\overline{\bm{\beta}}_{i}^{*}-\overline{\bm{\beta}}_{i}\right\|_{\infty}\leq\alpha$ with probability at least $1-\gamma\,$ for $\alpha:=\frac{2B}{\epsilon_{i}}\cdot\ln(d/\gamma)$.

Proof.

With $\bm{R}:=\overline{\bm{\beta}}_{i}^{*}-\overline{\bm{\beta}}_{i}$, we know that $\bm{R}$ follows the probability distribution $[Lap(2B/\epsilon_{i})]^{d}$. Then, by referring to the proof of Theorem 2, we can easily obtain the conclusion of Theorem 4. We omit the specific proof due to the space limitation. ∎

Our experiments in Section 5 will show that achieving VLDP/RLDP by adding Laplace noise to each voter’s parameter will lead to a limited utility. Recall that VLDP (voter-level privacy protection with distributed perturbation) itself is the strongest notion among four paradigms focused in this paper. Thus, it accordingly has the worst data utility. Clearly, we can know that the definition of RLDP is relatively weaker than VLDP. However, RLDP holds the same low data utility as the VLDP when using Laplace mechanism. Therefore, in the next section, we focus on proposing a novel algorithm which can achieve RLDP while ensuring a higher data utility.

4.4. RLDP: Perturbing the Object Function of Each Voter

To further enhance the data utility, an alternative approach is to perturb the objective function of each voter when conducting MLE optimization. This is the functional mechanism of (Zhang et al., 2012). For comparison, we will show in Section 5 (Figures 4 and 8(d)) that the functional mechanism can provide better utility than adding Laplace noise directly for record-level differential privacy.

In view of Eq. (2), we define $V_{j}^{(i)}:=X_{j}^{(i)}-Z_{j}^{(i)}$. We split the log-likelihood function for voter $i\in\{1,2,\cdots,N\}$ as

[TABLE]

Later, we will focus on one arbitrary voter $i$, and omit the notation $i$ in the analysis for simplicity. Since the final parameter $\bm{\beta}$ is just the average of each $\bm{\beta}_{i}$, we can process each $\bm{\beta}_{i}$ one by one. We then split the objective function (8) for each record $j\in\{1,2,\cdots,n\}$ as

[TABLE]

Therefore, the optimal model parameter $\bm{\beta}^{o}$ for each voter is:

[TABLE]

We denote a $d$-dimensional vector $\bm{\beta}$ by $(\beta[1],\cdots,\beta[d])$. By the Stone-Weierstrass Theorem (Rudin, 1964), any continuous and differentiable $f(V_{j},\bm{\beta})$ can always be written as a (potentially infinite) polynomial of $(\beta[1],\cdots,\beta[d])$. For some $W\in[0,\infty]$, we have

[TABLE]

where $\lambda_{\phi_{,V_{j}}}\in\mathbb{R}$ is the coefficient of $\phi(\bm{\beta})$ in the polynomial. $\phi(\bm{\beta})$ denotes the product of $\beta[1],\cdots,\beta[d]$ for some $c_{1},\cdots,c_{d}\in\mathbb{N}$ (the set of non-negative integers). $\Psi_{w}(w\in\mathbb{N})$ contains the set of all products of $\beta[1],\cdots,\beta[d]$ with degree $w$, that is

$\Psi_{w}=\left\{(\beta[1])^{c_{1}}(\beta[2])^{c_{2}}\cdots(\beta[d])^{c_{d}}\mid\sum_{l=1}^{d}c_{l}=w\right\}$.

Thus, we will express the objective function (9) for each user $i$ and each record $j$ as a polynomial like Eq. (11). Let $f_{1}$ and $g_{1}$ be two functions defined as follows:

[TABLE]

Then we have $f(V_{j},\bm{\beta})=f_{1}(g_{1}(V_{j},\bm{\beta}))$.

Using the Taylor expansion at [math], we write $f(V_{j},\bm{\beta})$ as $\tilde{f}_{D}(\bm{\beta})$ below:

[TABLE]

Then we adopt an approximation approach to reduce the degree of the summation. In particular, we only use the value of $f_{1}^{(k)}(0)$ for $k=0,1,2$, where $f_{1}^{(k)}$ denotes the $k$-order derivative. Specifically, we can compute $f_{1}^{(0)}(0)=\ln\frac{1}{2}$. From $f_{1}^{(1)}(z)=\frac{{\Phi}^{{}^{\prime}}(z)}{\Phi(z)}$, we get $f_{1}^{(1)}(0)=\sqrt{2/\pi}$. From $f_{1}^{(2)}(z)=\frac{{\Phi}^{{}^{\prime\prime}}(z)\cdot\Phi(z)-{\Phi}^{{}^{\prime}}(z)\cdot{\Phi}^{{}^{\prime}}(z)}{\Phi^{2}(z)}$, we have $f_{1}^{(2)}(0)=-2/\pi$. Thus, we approximate $\tilde{f}_{D}(\bm{\beta})$ by

[TABLE]

For each $i\in\{1,2,\cdots,N\}$, $j\in\{1,2,\cdots,n\}$, we preprocess the data to make $\|X_{j}^{(i)}\|_{2}\leq 1/2$ and $\|Z_{j}^{(i)}\|_{2}\leq 1/2$. Then $\|V_{j}^{(i)}\|_{2}\leq 1$ by triangle inequality. For simplicity, we suppress both $i$ and $j$, and write $V\in D$ to mean $V\in\{V^{\prime}:X^{\prime}-Z^{\prime}|\langle X^{\prime},Z^{\prime}\rangle\in D\}$. Given $\|V\|_{2}=\sqrt{\sum_{k=1}^{d}(V[k])^{2}}\leq 1$, we have $\sum_{k=1}^{d}V[k]\leq\sqrt{d}$ by Cauchy-Schwarz inequality, where $V[k]$ denotes the $k$-th dimension of the vector $V$. Thus, from Lemma 1 in (Zhang et al., 2012) and Eq. (14), we can compute the $\ell_{1}$-sensitivity $\Delta$ of the coefficient vector for $\beta[1],\cdots,\beta[d]$ in Eq. (13) with respect to record-neighboring datasets as

[TABLE]

After computing the sensitivity’s upper bound $\Delta_{\text{upper}}$ as above, we inject Laplace noise with scale $\Delta_{\text{upper}}/\epsilon_{i}$ to the coefficients of the objective function and then solve the noisy objective function to estimate the moral parameters $\bm{\beta}_{i}^{*}$ of each voter $i$. Algorithm 3 shows the pseudo-code of our algorithm. From the above analysis, we have a Theorem 5 which follows the above analysis.

Theorem 5 (Privacy of Algorithm 3).

Algorithm 3 satisfies $\epsilon_{i}$-differential privacy for RLDP.

The utility of Algorithm 3 is bounded by a small quantity, which can be proved by following the utility analysis of the functional mechanism in (Zhang et al., 2012). We omit the details due to space limitation.

5. Experimental Evaluation

This section implements our proposed algorithms and evaluates their performance on synthetic datasets and a real-world dataset extracted from the Moral Machine, respectively.

5.1. Synthetic Data

We first present the experimental results on synthetic data. Denote the true parameter of each voter $i$ as $\bm{\beta}_{i}$ which is sampled from Gaussian distribution $\mathcal{N}(\bm{m},\bm{I}_{d})$, where each mean $m_{j}$ for $j\in\{1,2,\ldots,d\}$ is independently sampled from the uniform distribution $\mathcal{U}(-1,1)$, and $\bm{I}_{d}$ is the $d\times d$ identity matrix with $d$ being the dimension size (i.e., the number of features). By default, we set $d=10$ when not specified. For each voter $i\in\{1,2,\ldots,N\}$, we generate its $n$ pairwise comparison via the following two steps (by default $n=50$ when not specified). First, we sample two alternatives $\bm{x}_{1}$ and $\bm{x}_{2}$ independently from the Gaussian distribution $\mathcal{N}(0,\bm{I}_{d})$. Second, we sample their utilities $U_{\bm{x}_{1}}$ and $U_{\bm{x}_{2}}$ based on the Gaussian distribution $\mathcal{N}(\bm{\beta}_{i}^{\top}\bm{x}_{1},\frac{1}{2})$ and $\mathcal{N}(\bm{\beta}_{i}^{\top}\bm{x}_{2},\frac{1}{2})$, respectively. To evaluate the performance of our algorithms, we compute the accuracy which is defined as the fraction of test instances on which the true and noisy parameters give the same exact outcome. Besides, the parameter $B$ is set as $B=2$ in the following experiments.

The results in Figures 2, 2 and 4 show high accuracies of our Algorithms 1, Algorithms 2 and 3 on the synthetic dataset with the privacy parameter $\epsilon$ varying from 0.01 to 10; specifically, we consider $\epsilon\in$ $\{0.01,0.02,0.03,0.05,0.07,0.09,0.1,0.2,0.3,0.5,0.7,0.9,$ $1,2,3,5,10\}$. Note that we here use the same privacy parameter $\epsilon$ for all voters and will analyse the personalized privacy parameter specifically in the following. As shown in all figures, a smaller $\epsilon$ (i.e., higher privacy protection level) will lead to a lower accuracy.

Moreover, Figures 2, 2 and 4 (all with $d=10$) show that for three Algorithms 1, 2 and 3, a larger number of voters (i.e., $N$) or a larger number of records (i.e., $n$) leads to better accuracy. This is because our output is the average of each voter. Then with more voters, each one is relatively less sensitive. Therefore, it can achieve better accuracy (less noise) while keeping the same privacy.

We also compare the accuracies of three Algorithms 1, 2 and 3. As shown in Figure 4, Algorithms 1 outperforms the other two algorithms. Also, both Algorithms 1 and 3 have much better accuracy than Algorithm 2. This is because Algorithm 2 realizes distributed privacy protection by adding Laplace noise, which ensures strong privacy guarantees, thus leading a relatively low data utility. In contrast, Algorithm 3 also achieves distributed privacy protection by perturbing the objective function instead of parameters directly, which improves the data utility greatly. Therefore, Figure 4 illustrates the superiority of functional mechanism over Laplace for record-level privacy protection with distributed perturbation.

Impact of the dimension $d$. Figure 5 shows the comparisons of the impact of parameter $d$ on the accuracy of the Algorithms 1 and 3. Due to the space limitation, we no longer present the result of Algorithm 2 here since it has limited data utility as shown in Figure 4. It can be seen from Figure 5 that the accuracy of both Algorithms 1 and 3 will be decreased with the increase of dimension $d$. For Algorithm 1, the larger $d$ means more information to be protected so the accuracy will be reduced. For Algorithm 3, the sensitivity will increase with the increase of $d$, thus leading to the reduction of the accuracy when $d$ becomes larger.

Impact of the personalized privacy parameters. We also implement personalized privacy parameter choices for RLDP (i.e., Algorithm 3) and conduct extensive experiments to show the impact of the personalized privacy parameter on accuracies. Although Algorithm 2 can also achieve personalized privacy protection, we no longer present the result of Algorithm 2 here since it has limited data utility and due to the space limitation.

We set personalized privacy parameter specifications based on the findings from studies in (Jorgensen et al., 2015; Acquisti and Grossklags, 2005). The voters are randomly divided into three groups: (i) conservative, (ii) moderate, and (iii) liberal, representing voters with high/medium/low privacy concern, respectively. The fraction of voters in three groups are $f_{C}$, $f_{M}$, and $f_{L}$, where $f_{C}+f_{M}+f_{L}=1$. We set default values as $f_{C}=0.54$, $f_{M}=0.36$, and $f_{L}=0.1$, which are chosen based on the findings reported in (Acquisti and Grossklags, 2005). The privacy parameters $\epsilon$ of voters who belong to conservative or moderate groups are chosen uniformly at random from $[\epsilon_{C},\epsilon_{M}]$ or $[\epsilon_{M},\epsilon_{L}]$ correspondingly (and rounding to the nearest hundredth), where $\epsilon_{C}\in\{\underline{0.01},0.05,0.1,0.2,\cdots,0.5\}$, $\epsilon_{M}\in\{0.05,0.1,0.15,\underline{0.2},\cdots,0.5\}$, and $\epsilon_{L}=\underline{1}$, and the default values are underlined.

Then, we implement Algorithm 3 by setting personalized privacy parameters as specified above. Figure 7(a) shows the accuracy of Algorithm 3 by varying the fraction of conservative voters $f_{C}$ from 0.1 to 0.6 by setting $n=50$ and $d=10$. Note that for each set of $f_{C}$, the fraction of moderate voter is fixed as $f_{M}=0.36$ and the fraction of liberal voters is equal to $1-(f_{C}+f_{M})$. We can observe from Figure 7(a) the accuracy will be reduced when increasing $f_{C}$. This is because increasing $f_{C}$ will increase the number of conservative voters (prefer for choosing small $\epsilon$) but decrease the number of liberal voters (prefer for choosing large $\epsilon$), thus leading to more noise injections and accuracy reductions.

Furthermore, Figures 7(b) and 7(c) present experimental results on the impact on accuracy when varying the range of $\epsilon_{C}$ or $\epsilon_{M}$, respectively. Figure 7(b) shows the accuracy of Algorithm 3 when varying $\epsilon_{C}$ from 0.01 to 0.5 by fixing moderate privacy parameter as $\epsilon_{M}=0.5$ (rather than the default value of $\epsilon_{M}=0.2$ to ensure $\epsilon_{C}\leq\epsilon_{M}$ in all cases). It can be seen that the larger $\epsilon_{C}$ or $N$ will induce better accuracy. Besides, Figure 7(c) shows the accuracy of Algorithm 3 when changing $\epsilon_{M}$ from 0.05 to 0.5 by fixing $\epsilon_{C}=0.01$. We can observe again that larger $\epsilon_{M}$ or $N$ will lead to better accuracy. What’s more, the accuracy of Figure 7(b) is higher than that of Figure 7(c). This is because the conservative privacy parameter in Figure 7(c) always equals to 0.01 and the fraction of conservative voters is bigger than that of moderate voters.

Impact of the norm bound $B$. In addition, we also analyze the impact of norm bound $B$ on the accuracies of Algorithms 1 and 2, as shown in Figure 7. It can be observed that the accuracy will be relatively higher when norm bound around at $2$ or $3$.

5.2. Moral Machine Data

The Moral Machine data contain the moral decisions by voters about ethical dilemmas faced by autonomous vehicles. Each voter is asked to view 13 scenarios (i.e. $n=13$), each of which includes two possible moral dilemma alternatives. Each alternative is characterized by 23 features (i.e. $d=23$). We sample $200$ voters’ data ($N=200$) from the Moral Machine dataset to evaluate our algorithms.

Figures 8(a), 8(b) and 8(c) plot the accuracies of our Algorithms 1,2 and 3 on the real dataset for privacy parameters $\epsilon$ from 0.01 to 10 and for different numbers of voters (i.e., $N$), respectively. The privacy parameter specifications follow the same settings of the synthetic dataset in Section 5.1. As expected, larger $N$ or larger $\epsilon$ will induce better accuracy for all three Algorithms 1,2 and 3.

In addition, Figure 8(d) shows the comparisons of three Algorithms 1,2 and 3 with $N=100,n=13$. It can observe from Figure 8(d) that Algorithm 1 relatively outperforms Algorithms 2 and 3 and the Algorithm 2 has the lowest accuracy in all cases. This is because Algorithm 2 follows a strong privacy definition with distributed Laplace perturbation, which leads to larger noise addition. In contrast, Algorithm 3 also achieves distributed privacy protect while ensuring a good data utility since it perturbs objective functions instead of parameters. Thus, this demonstrates the superiority of functional mechanism over Laplace mechanism for record-level privacy protection with distributed perturbation.

Furthermore, extensive experiments for personalized parameter settings are also conducted on the Moral Machine data, which follows the same privacy parameter specifications as Section 5.1. Figure 10 shows the accuracies of the Algorithm 3 under different personalized privacy parameters. Two figures in Figure 10 present the impact of the conservative privacy parameters $\epsilon_{C}$ (resp., moderate privacy parameters $\epsilon_{M}$) on accuracies by varying $\epsilon_{C}$ from 0.01 to 0.5 with $\epsilon_{M}=0.5$ (reps., $\epsilon_{M}$ from 0.05 to 0.5 with $\epsilon_{C}=0.01$). Both figures show that larger $\epsilon_{C}$ or $\epsilon_{M}$ will lead to higher accuracy.

5.3. Comparisons with No Privacy

To further show the effectiveness of our privacy-preserving algorithms, we present the ratio of the accuracy of our algorithms to the baseline method that with no privacy protection (i.e., solution of Noothigattu et al. (Noothigattu et al., 2018)). As shown in Figure 10, Algorithms 1 and 3 hold better accuracy than Algorithm 2 on both synthetic dataset (with $N=50$ and $n=100$, Noothigattu et al.’s method has $92.4\%$ accuracy) and real dataset (with $N=200$ and $n=13$, Noothigattu et al.’s method has $98.2\%$ accuracy). In particular, we can observe that Algorithms 1 and 3 can ensure greater than $80\%$ accuracy when $\epsilon>0.3$ and greater than $90\%$ accuracy when $\epsilon>1$.

6. Discussions and Future Directions

Choice of $\epsilon$. Our algorithms adopt the formal notion of $\epsilon$-DP in which smaller $\epsilon$ means stronger privacy protection but less utility. In the literature, typical $\epsilon$ values chosen for experimental studies are mostly between $0.1$ and $10$ (Jorgensen et al., 2015; Abadi et al., 2016; Wang et al., 2019). In the industry, both Google Chrome and Apple iOS have implemented $\epsilon$-DP (in particular, a specific variant called $\epsilon$-LDP (Duchi et al., 2013)). In Google Chrome (Erlingsson et al., 2014), $\epsilon$ per each datum is at most $2$, and $\epsilon$ per each user is at most $9$. Apple claims that $\epsilon$ per each datum per day in iOS is between $1$ and $2$, but a recent study (Tang et al., 2017) shows that it can be as high as $16$.

Privacy Protection for Crowdsourced Data Collection for Fairness Studies of Machine Learning. Fairness of machine learning algorithms has received much interest recently (Srivastava et al., 2019). Among various fairness notions, demographic parity means equalizing the percentage of people who are predicted by the algorithm to be positive across different social groups (e.g., racial groups) (Dwork et al., 2012). Srivastava* et al. (2019)* have collected a large crowdsourced dataset to capture people’s perception of fairness of machine learning. As a user’s perception of fairness may contain sensitive information, a future direction in the same vein as our current paper is to preserve the privacy of users in the aggregation of fairness preferences.

7. Conclusion

This paper incorporates privacy protection into crowdsourced data collection used to guide ethical decision-making by AI. Based on the formal notion differential privacy, we proposed four different privacy protection paradigms with the consideration of voter-/record-level privacy protection and centralized/distributed perturbation. Then, we specifically propose three algorithms which achieve the above four granularities of privacy protection paradigms. We have proved the privacy guarantees and the data utilities of our algorithms. To show the effectiveness of our algorithms, extensive experiments have been conducted on both synthetic datasets and a real-world dataset from the Moral Machine project.

Acknowledgements.

The authors would like to thank the anonymous reviewers for their valuable comments. This work was partly supported by Natural Science Foundation of China (Nos. 61572398, 61772410, 61802298 and U1811461), the Fundamental Research Funds for the Central Universities (No. xjj2018237), and the China Postdoctoral Science Foundation (No. 2017M623177). Jun Zhao’s research was supported by Nanyang Technological University (NTU) Startup Grant M4082311.020, Alibaba-NTU Singapore Joint Research Institute (JRI), Singapore Ministry of Education Academic Research Fund (AcRF) Tier 1 RG128/18, and NTU-WASP Joint Project M4082443.020. Han Yu’s research was supported by the Nanyang Assistant Professorship (NAP) and the Joint NTU-WeBank Research Centre of Eco-Intelligent Applications (THEIA), Nanyang Technological University, Singapore.