Aggregating Probabilistic Judgments

TL;DR

This paper extends classical judgment aggregation methods to probabilistic opinions on related issues, proposing new aggregation functions and analyzing their properties and limitations.

Contribution

It introduces a modified Boolean framework for probabilistic judgments and generalizes classical aggregation functions to handle probabilistic data.

Findings

Proposes probabilistic aggregation functions based on classical methods.

Identifies desirable properties and discusses impossibility results.

Provides a theoretical foundation for aggregating probabilistic judgments.

Abstract

In this paper we explore the application of methods for classical judgment aggregation in pooling probabilistic opinions on logically related issues. For this reason, we first modify the Boolean judgment aggregation framework in the way that allows handling probabilistic judgments and then define probabilistic aggregation functions obtained by generalization of the classical ones. In addition, we discuss essential desirable properties for the aggregation functions and explore impossibility results.

Tables

Table 1. Table 1: An example of a judgment aggregation.

	$p$	$p\to q$	$q$
Minister 1	true	true	true
Minister 2	true	false	false
Minister 3	false	true	false
Majority	true	true	false

Table 2. Table 2 : An example of a source aggregation.

	$s\vee t$	$x$	$h$	$a$	$\neg e$
IS 1	0.6	1	1	-	1
IS 2	0.7	0.6	0.5	-	1
IS 3	0.1	0.4	0.2	-	1
IS 4	0.8	0.8	0.9	-	1
IS 5	0.7	0.7	0.4	-	1
IS 6	0.5	0.6	0.3	-	1

Table 3. Table 4: The set of all rational crisp judgment sets for the frame in Example  1.1 , their “scores” and Euclidean distance to the likelihood profile.

$J$
$s\vee t$	$x$	$e$	$h$	$a$	${\sum }_{\_}\phi \in JS(\phi ,\hat{P})$	$D_{\_}e(J,\hat{P})$
0	0	0	0	1	13,2	11.00837576
0	0	1	0	0	7.2	13.98315881
0	1	0	0	1	15.4	13.09902931
0	1	1	0	1	9.4	9.935132833
1	0	0	1	1	14,6	10.45417943
1	0	1	0	0	8	13.7033258
1	0	1	1	0	8.6	13.48039101
1	1	0	1	1	16.8	8.91436323
1	1	1	1	1	10.8	12.38906945

Table 4. Table 5: An example of a likelihood profile

	$p$	$\neg p$	$p\to q$	$\neg \left(p\to q\right)$	$q$	$\neg q$
${\hat{J}}_{\_}1$	0.0	0.3	0.8	0.1	0.6	0.2
${\hat{J}}_{\_}2$	0.1	0.4	0.5	0.2	0.3	0.6
${\hat{J}}_{\_}3$	0.8	0.0	0.1	0.8	0.3	0.7

Table 5. Table 6: The set of all rational crisp judgment sets for Φ = { p , p → q , q } \Phi=\{p,p\rightarrow q,q\} , Γ ^ = ⊤ ^ Γ top \hat{\Gamma}=\top and Γ = ⊤ Γ top \Gamma=\top , their “scores” and Euclidean distance to the likelihood profile from Table  5 .

$J$
$p$	$p\to q$	$q$	${\sum }_{\_}\phi \in JS(\phi ,\hat{P})$	$D_{\_}e(J,\hat{P})$
1	1	1	3.5	4.1818
1	0	0	3.5	3.8537
0	1	1	3.3	4.1064
0	1	0	3.6	4.0032

Equations

$$\Phi =\{{\phi }_{\_}1,\dots ,{\phi }_{\_}m\},$$

$$\ell (\phi ){\ge }^{\ast }a,$$

$$\hat{J}=$$

$$\hat{P}=({\hat{J}}_{\_}1,\dots ,{\hat{J}}_{\_}n),$$

$${\hat{J}}_{\_}k=\{\ell (\phi ){\ge }^{\ast }{a}_{\_}^k\phi \mid \phi \in {\Phi }^{\cup }\},$$

$$\textrm{crisp}(\ell(\varphi)\geq^{*}a,c)=\left\{\begin{array}[]{ll}\{\varphi\},&\textrm{ if }a\geq c\\ \emptyset,&\textrm{ otherwise }\end{array}\right.$$

$$\text{crisp}(\hat{J},\mathbf{c})=\bigcup \{\text{crisp}(\ell (\phi ){\ge }^{\ast }{a}_{\_}\phi ,c_{\_}\phi )\mid \ell (\phi ){\ge }^{\ast }{a}_{\_}\phi \in \hat{J}\}$$

$$J^{\ast }=\{\ell (\phi )=1,\ell (\neg \phi )=0\mid \phi \in J\}$$

$$P^{\ast }=(J_{\_}^{\ast }1,\dots ,J_{\_}^{\ast }n),$$

$$\begin{array}[]{l}\hat{f}_{\_}{q}(\hat{P})=\{\ell(\varphi)\geq^{*}a_{\_}\varphi\;:a_{\_}\varphi=\max_{\_}{0\leq a\leq 1}(|\{k:a^{k}_{\_}{\varphi}\geq a\}|\geq q)\}\;.\\ \hat{F}_{\_}{q}(\hat{P},\mathbf{c})=\textrm{crisp}(\hat{f}_{\_}{q}(\hat{P}),\mathbf{c}).\end{array}$$

$$m(P)=\{\phi :\phi \in {\Phi }^{\cup },|\{k:\phi \in J_{\_}k\}|>\frac{n}{2}\}$$

$$U_{\_}q(P)=\{\phi :\phi \in {\Phi }^{\cup },|\{k:\phi \in J_{\_}k\}|\ge q\}$$

$$N(P,\phi )=|\{k:\phi \in {\Phi }^{\cup },\phi \in J_{\_}k\}|.$$

$$\text{med}(P)=\underset{J\in \mathcal{J}}{\text{mbox}argmax}\sum _{\_}\phi \in JN(P,\phi ),$$

$$\hat{N}(\hat{P},\phi ,c)=|\{k:a_{\_}^k\phi \ge c\}|.$$

$$\widehat{\text{med}}(\hat{P})=\underset{J\in \mathcal{J}}{\text{mbox}argmax}\sum _{\_}\phi \in JS(\phi ,\hat{P}),$$

$$S(\phi ,\hat{P})=\sum _{\_}{J}_{\_}k\in \hat{P}{a}_{\_}^k\phi$$

$$E_{\hat{P}}(\phi )=\frac{1}{n}\sum _{\_}{k=1}^n{a}_{\_}{\phi }^k,$$

$${\mathbf{a}}_{\_}\hat{P}=(E_{\hat{P}}(\phi ){)}_{\_}\phi \in {\Phi }^{\cup }.$$

$${\mathbf{a}}_{\_}\hat{P}=(\underset{s\vee t}{\underbrace{0.56,0.44,}}\underset{h}{\underbrace{0.55,0.45,}}\underset{x}{\underbrace{0.75,0.25,}}\underset{a}{\underbrace{0.0,0.0,}}\underset{e}{\underbrace{1.0,0.0}}).$$

$${\mathbf{a}}_{\_}{\hat{P}}^{\prime }=(\underset{s\vee t}{\underbrace{0.56,0.44,}}\underset{h}{\underbrace{0.3,0.7,}}\underset{x}{\underbrace{0.75,0.25,}}\underset{a}{\underbrace{0.0,0.0,}}\underset{e}{\underbrace{1.0,0.0}})$$

$${\hat{F}}_S(P^{\ast })=\text{scleximax}(P),$$

$${\hat{F}}^{d,\Sigma }(\hat{P})=\underset{J\in \mathcal{J}}{\text{mbox}argmin}\sum _{\_}{\hat{J}}_{\_}k\in \hat{P}d(J^{\ast },{\hat{J}}_{\_}k),$$

$$d(\hat{J},{\hat{J}}_{\_}k)=d((a_{\_}\phi {)}_{\_}\phi \in {\Phi }^{\cup },(a_{\_}^k\phi {)}_{\_}\phi \in {\Phi }^{\cup }).$$

$${\hat{F}}^{d,\max }(\hat{P})=\underset{J\in \mathcal{J}}{\text{mbox}argmin}\underset{\_}{\max }{\hat{J}}_{\_}k\in \hat{P}d(J^{\ast },{\hat{J}}_{\_}k).$$

$${\hat{F}}^{e,\Sigma }(\hat{P})=\underset{J\in \mathcal{J}}{\text{mbox}argmin}{D}_{\_}e(J,\hat{P}),$$

$$D_{\_}e(J,\hat{P})=\sum _{\_}{\hat{J}}_{\_}k\in \hat{P}\sqrt[]{\sum _{\_}\phi \in {\Phi }^{\cup }(a_{\_}{\phi }^k-a_{\_}\phi {)}^2}.$$

$$F^{d_{\_}H,\Sigma }(P)=\underset{J\in \mathcal{J}}{\text{mbox}argmin}\sum _{\_}{J}_{\_}k\in P{d}_{\_}H(J,J_{\_}k).$$

$$Z(\hat{P})=\{\phi \in {\Phi }^{\cup }:a_{\_}^k\phi =1,\forall k\in N\}.$$

Full text

1 Introduction

Judgment aggregation (JA) is concerned with aggregating sets of binary truth valuations assigned to logically related issues [28, 20]. Various collective decision making problems in artificial intelligence can be modelled as JA problems, e.g., problems of constructing agreements, such as finding a collective goal in multi-agent systems [37, 3]. In agreement reaching problems each agent in a group is a source of judgments and also typically affected by the collective choice resulting from the aggregation of individual judgments. For example, I am a citizen voting on a referendum that decided not to impose global warming curbing methods, but I am also a citizen that has to live with the consequences of that collective decision. A typical JA example [28] is one concerning three issues: Current $CO_{\_}2$ emissions lead to global warming ($p$), If current $CO_{\_}2$ emissions lead to global warming, then we should reduce $CO_{\_}2$ emissions ($p\rightarrow q$), We should reduce $CO_{\_}2$ emissions ($q$). The individual sets of judgments are as in Table 1. As observed from the example, pooling the truth valuations on each issue does not always lead to a consistent set of collective judgments. JA designs and studies aggregators that produce a consistent outcome.

However, aggregation problems are not always Boolean, because the individual judgments on whether an issue is true or false are not always certain. We give an example.

Example 1.1**.**

You want a recommendation for a specific hotel, “The Grand Palace”, however, you want that recommendation to be compiled specifically for you. You are interested in:

• •

a hotel close to the centre or well connected with public transport ($s\vee t$);

• •

a hotel that is a unique experience ($x$),

• •

a hotel that is a good value for money ($a$).

The information that you can get from online information sources (IS), like booking.com, TripAdvisor, etc., can be processed automatically, by pooling reviews from recommendations regarding “The Grand Palace” hotel. An example of such collection of opinions is given in Table 2. What we obtain from each IS is the likelihood that an issue is true. You can find information online about $s\vee t$ (second column), about whether “The Grand Palace” hotel is a unique experience (third column) and also whether the hotel is recommended by the users, $h$, (fourth column). However, it is not enough that the hotel is recommended in the reviews. For you, a hotel should be recommended ($h$) iff both $s\vee t$ and $a$ are true, i.e. $((s\vee t)\wedge a)\leftrightarrow h$. Information about $a$ may not be available to extract. Assume that you define that a hotel is a good value for money ($a$) if it is not more than 80 Euro per night ($\neg e$) or if it is a unique experience, i.e., when $(\neg e\vee x)\leftrightarrow a$ holds. Then the information you need to extract is whether $\neg e$ is true. This is given in the sixth column of Table 2.

We want to be able to aggregate likelihood judgments like the ones represented in the rows of Table 2, but into a set of Boolean judgments: should the hotel be recommended and for which reasons. To achieve this purpose, we explore how methods from classical judgment aggregation can be adjusted to deal with probabilistic statements as judgments. Thus, we extend the propositional logic JA framework typically used [28, 20, 27, 15] using the logic of likelihood [21], and design probabilistic aggregation functions based on the classical ones. Thus, intuitively, what were desirable properties for aggregation in the classical case, remain desirable properties in the probabilistic framework.

Our framework allows sources to have uncertain probabilistic judgments that are rational and subject to inevitable probabilistic constraints, but also to aggregate into a collective judgment set that is Boolean and subject to a specific set of propositional constraints. Frameworks for representing non-binary judgments have been considered, see e.g., [20] for an overview, however no specific methods for aggregation have been designed for these frameworks. Rather, impossibility characterisations have been studied showing which sets of desirable properties cannot be mutually satisfied. Here we propose specific classes of aggregators for the framework we introduce.

There is a certain amount of literature on probabilistic opinion pooling (e.g., see [29] for a detailed survey) which is concerned with aggregating probability functions (representing opinions of agents) into a single one. The defined properties of the aggregating functions are similar to those in JA theory, and similarly as there, impossibility results are proved. However, opinion pooling presumes that every agent has its probabilistic judgments defined on a $\sigma$-algebra of events (or, equivalently on a set of possible worlds). Despite the inherent consistency, this is not always a realistic requirement. In our framework, we allow the agents to express their probabilistic opinions on any (logically related) propositional language statements (equivalently, on any subset of a $\sigma$-algebra of events), and, moreover, these opinions can be imprecise, i.e., expressed through likelihood inequalities (equivalently, a set of probability functions is provided by each source.) In this sense our work is more comparable to variants of opinion pooling that presume a general agenda [8] or deal with imprecise probabilities [38]. Aggregation functions in probabilistic opinion pooling are typically averaging functions like (weighted) linear or geometrical average. Here we take the approach of defining aggregation functions by generalizing judgment aggregation functions based on representative voting.

The paper is structured as follows. In Section 2 we introduce the judgment aggregation framework based on the logic of likelihood. In Section 3 we demonstrate how to generalise classical judgment aggregation functions into functions that handle likelihood judgments, and we also introduce some new classes of judgment aggregation functions. In Section 4 we discuss desirable properties of aggregation functions and revisit the classical impossibility results. In Section 5 we discuss related work and in Section 6 we make our conclusions and outline directions for future work.

2 Framework

We distinguish between an agenda setter and information sources. The agenda setter identifies the set of issues, i.e., the agenda for which Boolean judgments need to be made. The agenda setter can also set additional relations, which we call propositional constraints, that should hold among the agenda issues. The information sources are modelled as sets of likelihood formulas subject to different relations called probabilistic constraints. The probabilistic constraints model the natural and contextual properties of the issues.

2.1 Judgment aggregation model

To model the agenda and the propositional constraints we use a set $\mathcal{L}$ of propositional logic formulas. An agenda is a finite set \Phi~{}\subset~{}\mbox{\mathcal{L}},

[TABLE]

s.t. $\varphi_{\_}i$ is neither a tautology nor a contradiction. We call the elements of the agenda issues. The set of propositional constraints \Gamma\subset\mbox{\mathcal{L}} represents special relations that should hold among the agenda issues described by the agenda setter. $\Gamma$ should be satisfiable, and we allow $\Gamma=\{\top\}$. In Example 1.1, we have $\Phi=\{s\vee t,h,x,e,a\}$, and $\Gamma=\{(\neg e\vee x)\leftrightarrow a,((s\vee t)\wedge a)\leftrightarrow h\}$.

The agenda setter is interested in aggregating collections of judgments on the agenda issues from various information sources into a set of crisp (Boolean) judgments that is consistent with $\Gamma$. A crisp judgment on $\varphi\in\Phi$ is either $\varphi$ or $\neg\varphi$. A crisp judgment set $J$ is a set of crisp judgments. E.g., the judgments of Minister 2 in Table1 can be represented as a crisp judgment set $J_{\_}2=\{p,\neg(p\rightarrow q),\neg q\}$. We introduce the notation $\Phi^{\cup}=\Phi\cup\{\neg\varphi\mid\varphi\in\Phi\}$. Then a crisp judgment set $J$ is a subset of $\Phi^{\cup}$. The set $J$ is consistent if $J\cup\Gamma$ is a consistent set of formulas in classical propositional logic. $J$ is complete if it contains one crisp judgment for each of the issues in the agenda. If the crisp judgment set $J$ is consistent and complete, we say that it is rational. Given an agenda $\Phi$ and propositional constraints $\Gamma$, the set of all consistent and complete, i.e., rational crisp judgment sets is $\mathcal{J}$.

We model the information sources as sets of likelihood judgments on $\Phi^{\cup}$. A likelihood judgment on the issue $\varphi\in\Phi^{\cup}$ is a simple likelihood formula of the type:

[TABLE]

where $\geq^{*}\in\{\geq,=\}$ and $a\in[0,1]$.111The formula (2) is an instance of the logic of likelihood in [19], [21] that consists of Boolean combinations of linear likelihood formulas of the type $a_{\_}1\ell(\varphi_{\_}1)+\ldots+a_{\_}n\ell(\varphi_{\_}n)\geq b$, where $a_{\_}i,b$ are real numbers, and $\varphi_{\_}i$ are pure propositional formulas. Likelihood formulas are interpreted in probability spaces where the term $\ell(\varphi)$ is interpreted as the probability of the set of worlds (outcomes) at which $\varphi$ is true.

The likelihood judgment $\ell(\varphi)\geq a$ expresses that the likelihood (probability)222 In this paper we interpret likelihood as probability and we use the two terms interchangeably. Note that, however, likelihood can also be interpreted as other measure of belief, see [21]. of the statement $\varphi$ being true is at least $a$. This intuition immediately implies that $\ell(\neg\varphi)\leq 1-a$. This and other entailments we mention later can formally be proved in the axiomatic system for the logic of likelihood that consists of axioms for propositional reasoning, reasoning about inequalities, and the following axioms for probabilistic reasoning given in [21]:

(L1) $\ell(\varphi)\geq 0$,

(L2) $\ell(\top)=1$,

(L3) $\ell(\varphi)=\ell(\varphi\wedge\psi)+\ell(\varphi\wedge\neg\psi)$,

(L4) From $\varphi\leftrightarrow\psi$ infer $\ell(\varphi)=\ell(\psi)$.

Having $\ell(\neg\varphi)\leq 1-a$ gives us an upper, but not a lower bound for the likelihood of $\neg\varphi$. Therefore, we ask that an explicit judgment for the likelihood of $\neg\varphi$ is given.

$\ell(\varphi)=a$ is a stronger statement than $\ell(\varphi)\geq a$ expressing that the likelihood of $\varphi$ being true is exactly $a$. In this case, a judgment for $\neg\varphi$ is implied, namely, $\ell(\neg\varphi)=1-a$.

Each of the information sources is represented as a set of likelihood judgments $\hat{J}$. The set $\hat{J}$ has one likelihood judgment on each of the issues in $\Phi^{\cup}$:

[TABLE]

where $\geq^{*}\in\{=,\geq\}$, $a_{\_}{\varphi}\in[0,1]$.

Note that providing likelihood formulas for both $\varphi$ and $\neg\varphi$ in Eq.(3) is equivalent with providing intervals for the likelihood of either $\varphi$ or $\neg\varphi$ (hence the information sources are free to do that) but for the discussion in this paper the formulation in Eq.(3) is a more suitable one.

A set of likelihood judgments is always complete in the sense that it contains a likelihood judgment for each of the issues. This assumption does not limit the freedom of not having a specific likelihood estimate for a given issue $\varphi$. To represent the absence of a specific likelihood, or an “abstention” on an issue $\varphi$ we use the tautology $\ell(\varphi)\geq 0$. We usually omit explicitly writing these type of formulas in the examples of judgment sets. Also, if we have $\ell(\varphi)=a\in\hat{J}$, we can omit including $\ell(\neg\varphi)=1-a$ as an element of $\hat{J}$.

Given a finite set of $n$ information sources $N=\{1,\ldots,n\}$, a likelihood profile:

[TABLE]

is a collection of sets of likelihood judgments for an agenda $\Phi$, each representing one information source $k\in N$. We slightly abuse notation and write $\hat{J}_{\_}k\in\hat{P}$ to denote that $\hat{J}_{\_}k$ is the $k$-th likelihood judgment set in $\hat{P}$:

[TABLE]

where $a^{k}_{\_}{\varphi}\in[0,1]$, for $k=1,\ldots,n$. The profile set of likelihood judgments that will be obtained from the information in Example 1.1 is given in Table 3.

We require that the sets of likelihood judgments in the profile are rational. We now define what are rational likelihood judgments.

2.2 Rationality of probabilistic judgment sets

A probabilistic judgment set is consistent if it is a consistent set of formulas in the logic of likelihood (according to the canonical definition of consistency). Note that a probabilistic judgment set is not always consistent. Consider, for example, the agenda $\Phi=\{p_{\_}1,p_{\_}1\wedge p_{\_}2,p_{\_}1\wedge\neg p_{\_}2\}$ and $\hat{J}=\{\ell(p_{\_}1)=0.5,\ell(p_{\_}1\wedge p_{\_}2)\geq 0.4,\ell(p_{\_}1\wedge\neg p_{\_}2)\geq 0.7\}$. The set $\hat{J}$ is an inconsistent set of formulas, because it implies $\ell(p_{\_}1)\geq 1.1$ by the axiom (L3) of likelihood logic. Furthermore, note that a judgment set $\hat{J}$ defined as in (3) has to satisfy $a_{\_}{\varphi}+a_{\_}{\neg\varphi}\leq 1$, for every $\varphi\in\Phi$, in order to be consistent.

In the probabilistic case, consistency and completeness are not enough of conditions for rationality. For example, $\hat{J}=\{\ell(p_{\_}1)\geq 0.3,\ell(p_{\_}1\wedge p_{\_}2)\geq 0.4,\ell(p_{\_}1\wedge\neg p_{\_}2)\geq 0.1\}$ is a consistent set. However, the second formula in it implies that $\ell(p_{\_}1)\geq 0.4$, which is stronger than the existing $\ell(p_{\_}1)\geq 0.3$ and, as such, is a more valuable judgment. We can formalize the notion of a stronger judgment as follows: if $\ell(\varphi)\geq^{*}a$ implies $\ell(\varphi)\geq^{*}b$ we will say that $\ell(\varphi)\geq^{*}a$ is a stronger judgment than $\ell(\varphi)\geq^{*}b$. For example, $\ell(\varphi)=a$ implies $\ell(\varphi)\geq a$. To ensure that we always have the strongest possible judgments in the consistent judgment sets, we introduce the notion of a final judgment. A consistent probabilistic judgment set is final if it does not imply stronger judgments than the ones it contains.333We recognize that it some cases it can be hard to check if a judgment set is final or not. In that sense, we note that this property of the judgment sets would not inflict the application of the judgment aggregation methods defined below, but it would influence the relevance of the produced results and the quality of the decision.

Probabilistic judgments can be subject to probabilistic constraints $\hat{\Gamma}$, where $\hat{\Gamma}$ is a set of likelihood formulas to denote that certain combinations of issues must have a certain likelihood. For example for agenda $\Phi=\{p_{\_}1,p_{\_}2,p_{\_}3\}$, where $p_{\_}1$, $p_{\_}2$, and $p_{\_}3$ represent the three possible states of a random variable, we can have the integrity constraint $\ell(p_{\_}1)+\ell(p_{\_}2)+\ell(p_{\_}3)=1$. Unlike the constraints $\Gamma$ which are given by the agenda setter, the probabilistic constraints $\hat{\Gamma}$ describe facts of the world and we assume that all information sources produce probabilistic judgment sets that are consistent with the probabilistic constraints.

A probabilistic judgment set $\hat{J}$ is rational if it is complete and final, and $\hat{J}\cup\hat{\Gamma}$ is a consistent set of likelihood formulas. Given an agenda $\Phi$ and probabilistic constraints $\hat{\Gamma}$, the set of all rational likelihood judgment sets is denoted by $\hat{\mathcal{J}}$. A profile is rational if all the judgment sets in it are rational.

We call a probabilistic aggregation frame the tuple $(\Phi,N,\Gamma,\hat{\Gamma})$, where $\Phi$ is an agenda, $N$ is a set of information sources, $\hat{\Gamma}$ is probabilistic constraints to be satisfied by the individual judgments of the sources, and $\Gamma$ are propositional constraints to be satisfied by the collective judgment. We call a crisp aggregation frame the tuple $(\Phi,N,\Gamma)$, but now $\Gamma$ are constraints to be satisfied by the individual judgments as well.

3 Aggregating likelihood judgments

We distinguish between crispifying aggregators and direct aggregators. Crispifying aggregators first aggregate the likelihood profile into likelihood judgment set(s) and then use given threshold values to “crispify” these sets. Direct aggregators assign a crisp judgment set (or sets) to a likelihood profile directly.

The rest of this section is organized as follows: We first consider details of the “crispification step” and introduce the formal definition of an aggregator; in Section 3.2 we propose a way to compare the likelihood aggregators with the classical ones; and finally in Section 3.3 and Section 3.4 we introduce several likelihood aggregators and analyse connections with the corresponding classical ones.

3.1 Crispifying

Given a probabilistic judgment, we can obtain a crisp judgment by choosing a threshold coefficient $c\in[0,1]$. This coefficient can be by default set to 0.5 for each issue, but otherwise we assume that it is specified by the agenda setter, in response to the question: How likely should an issue be in the least in order to be accepted as true? We define the judgment crispifying function $\textrm{crisp}()$ as follows:

[TABLE]

According to the above definition, if a likelihood judgment on a statement $\varphi\in\Phi^{\cup}$ has a (minimal) likelihood strictly greater than or equal to $c$, we assign it a Boolean judgment $\varphi$. Otherwise, no Boolean judgment is assigned for this issue. If we decide to be strict on an issue $\varphi$ and accept it only if true, we set $c_{\_}{\varphi}=1.$

We can crispify a probabilistic judgment set $\hat{J}$ by crispifying each of its judgments. We distinguish between issue-wise crispifying when a different coefficient is assigned for every agenda issue and uniform crispifying when the same coefficient is used for every agenda issue.

Let $\mathbf{c}=(c_{\_}{\varphi})_{\_}{\varphi\in\Phi^{\cup}}$ be a vector of coefficients, where each $c_{\_}{\varphi}\in[0,1]$, and $c_{\_}{\varphi}+c_{\_}{\neg\varphi}>1$. We call this a vector of crispifying coefficients. A judgment set crispifying is defined as follows:

[TABLE]

The condition $c_{\_}{\varphi}+c_{\_}{\neg\varphi}>1$, along with the consistency requirements $a_{\_}{\varphi}+a_{\_}{\neg\varphi}\leq 1$, assures that only one element of the set $\{\varphi,\neg\varphi\}$ is in $\textrm{crisp}(\hat{J},\mathbf{c})$ for each $\varphi\in\Phi$. If $c_{\_}{\varphi}=c$, for every $\varphi\in\Phi^{\cup}$, and some $c\in(1/2,1]$, the crispifying defined by Eq.(7) is uniform, and we denote it by $\textrm{crisp}(\hat{J},c)$. Note that the constraint $c>1/2$ follows from the consistency requirement on the crispifying vector.

Observe that the obtained crisp set of judgments may be incomplete. Further, we allow the agenda setter to freely choose whichever coefficients she wants for any of the issues, depending on the given context. This freedom of choice is done here for simplicity. We can, however, argue that a freely chosen vector of crispifying coefficients may be seen as imposing a certain level of independence on the issues. We can argue that if for two issues $\varphi_{\_}1,\varphi_{\_}2\in\Phi$ when it holds that $\{\varphi_{\_}1\rightarrow\varphi_{\_}2\}\in\Gamma$, then it should not be allowed that $c_{\_}1<c_{\_}2$, i.e., in this case we would need the additional constraint $c_{\_}1\geq c_{\_}2$. Also, if $\varphi_{\_}1\leftrightarrow\varphi_{\_}2\in\Gamma$, we would need to have $c_{\_}1=c_{\_}2$, i.e., logically equivalent issues should have the same likelihood threshold requirement. Restricting the values in $\mathbf{c}$ with respect to $\Gamma$ is an interesting aspect of our framework and is a line of future work we intend to pursue.

We now give a formal definition for an aggregator.

Definition 3.1**.**

Let $(\Phi,N,\Gamma,\hat{\Gamma})$ be a probabilistic aggregation frame and let $\hat{\mathbb{P}}$ be the set of all rational likelihood profiles for it, while $[0,1]^{2m}$, for $m=|\Phi|$, is the set of all crispifying vectors $\mathbf{c}$. Let $\hat{f}$ be a mapping from $\hat{\mathbb{P}}$ to $\hat{\mathcal{J}}$. A crispifying judgment aggregation function $\hat{F}$ is a mapping from $\hat{\mathbb{P}}\times[0,1]^{2m}$ to $\mathcal{P}(\mathcal{J})$, i.e. $\hat{F}(\hat{P},\mathbf{c})\subseteq\mathcal{J}$, where $\hat{F}(\hat{P},\mathbf{c})=\textrm{crisp}(\hat{f}(\hat{P}),\mathbf{c})$ is the classical judgment set obtained by crispifying the likelihood judgment set $\hat{f}(\hat{P})$. A direct judgment aggregation function $F$ is a mapping from $\hat{\mathbb{P}}$ to $\mathcal{P}(\mathcal{J})$, i.e. $F(\hat{P})\subseteq\mathcal{J}$.

According to the above definition an aggregator is defined for every rational profile and always produces rational judgment sets as a result, properties that are later introduced as universal domain and rationality, correspondingly. We embed these properties in the definition since they are the most basic desirable properties of the aggregation process, usually satisfied by design. However, while the universal domain is satisfied by all the aggregators defined below, we sometimes deviate from Definition 3.1 by defining some aggregators that are not rational.

Notice also that, even thought we insist on the collective judgment being crisp, in every crispifying aggregator (and, implicitly, in many direct aggregators) an intermediate probabilistic aggregate is available if needed in the decision process.

3.2 Classical vs probabilistic aggregators

Same as they do in [27], we define a classical irresolute aggregation function $\bar{F}(P)$ as one that maps each crisp rational profile $P$ of judgments to a nonempty set of crisp rational judgment sets.

Consider a crisp judgment set $J\in\mathcal{J}$. We define its corresponding probabilistic judgment set $J^{*}$ in the following way:

[TABLE]

Note that $\textrm{crisp}(J^{*},\mathbf{c})=J$ for every vector of crispifying coefficients $\mathbf{c}=(c_{\_}{\varphi})_{\_}{\varphi\in\Phi^{\cup}}$ such that $c_{\_}{\varphi}=1$, for $\varphi\in J$.

Given a crisp profile $P=(J_{\_}1,\ldots,J_{\_}n)$ we define

[TABLE]

to be its correspondent probabilistic profile. We can now define what it means for a likelihood aggregator to generalize a crisp aggregator.

Definition 3.2**.**

Let $(\Phi,N,\Gamma,\hat{\Gamma})$ be a probabilistic aggregation frame. Consider the corresponding crisp frame $(\Phi,N,\Gamma)$ and let $\mathbb{P}$ be the set of all rational likelihood profiles for it. Let $P^{*}$ be a corresponding profile for a $P\in\mathbb{P}$. A direct likelihood aggregator $F$ generalizes a crisp aggregator $\bar{F}$ if $\bar{F}(P)=F(P^{*})$ for each $P\in\mathbb{P}$. A crispifying likelihood aggregator $\hat{F}$ generalizes a crisp aggregator $\bar{F}$ if there exists $\mathbf{c}\in[0,1]^{2m}$ such that $\bar{F}(P)=\hat{F}(P^{*},\mathbf{c})$ for each $P\in\mathbb{P}$.

3.3 Crispifying aggregators

We now consider two classes of crispifying aggregators.

Uniform quota aggregators

Quota aggregators assign a crisp judgment to elements in $\Phi^{\cup}$ in two steps. First, the collective likelihood of $\varphi$ is assigned. The collective likelihood for $\varphi$ is the maximal $a\in[0,1]$ such that the number of agents in the profile who assign a likelihood of at least $a$ reaches a given quota $q$. Second, the collective likelihood judgments are crispified using a crispifying coefficient. The formal definition follows.

Definition 3.3**.**

Given a profile $\hat{P}$, a crispifying vector $\mathbf{c}$ and a quota $q\in\mathbb{N}$, $1\leq q\leq n$, we define the uniform quota function $\hat{f}_{\_}{q}$:

[TABLE]

As an illustration, consider the example in Table 3. For a uniform $\mathbf{c}=(0.6,\ldots,0.6)$ and a quota $q=3$ we obtain $\hat{F}_{\_}{3}(\hat{P},0.6)=\{s\vee t,x,\neg h,\neg e\}$, which is inconsistent with $\Gamma$.

If $q=n$, we obtain the unanimous function that selects as collective only those judgments $\varphi$ who are assigned a likelihood $a_{\_}{\varphi}^{k}\geq c_{\_}{\varphi}$ by all the agents $k$. For $q=\lfloor\frac{n}{2}\rfloor+1$ we obtain the issue-by-issue majority function, which we denote with $M$. Under issue-by-issue majority function the profile is aggregated by selecting the judgments that are in the most (more than a half) of the judgment sets in the profile. The set $M(\hat{P},\mathbf{c})=\hat{F}_{\_}{\lfloor\frac{n}{2}\rfloor+1}(\hat{P},\mathbf{c})$ is called a majoritarian set for $\hat{P}$ and $\mathbf{c}$.

The majoritarian set of a crisp profile $P$ is denoted $m(P)$ and contains all the elements of $\Phi^{\cup}$ that are supported by a strict majority of the individual judgment sets:

[TABLE]

The following theorem can be easily proved:

Theorem 3.1**.**

Let $(\Phi,N,\Gamma,\hat{\Gamma})$ be a probabilistic likelihood frame, and $\mathbf{c}=(c_{\_}{\varphi})_{\_}{\varphi\in\Phi^{\cup}}$ be a vector of coefficients such that $c_{\_}{\varphi}>0$, for every $\varphi\in\Phi^{\cup}$. Then $m(P)=M(P^{*},\mathbf{c})$, where $P^{*}$ is the corresponding probabilistic profile to the crisp profile $P$.

Aggregators based on the majoritarian set

One way to aggregate probabilistic judgments into a rational crisp judgment set is to minimally modify the set $M(\hat{P},\mathbf{c})$ so that it becomes consistent with $\Gamma$. This approach is used in crisp judgment aggregation to define several aggregators based on the majoritarian set [27]. We can extend the definition of aggregators based on the majoritarian set to likelihood aggregators as follows.

Definition 3.4**.**

A crispifying likelihood aggregator $\hat{F}$ is based on the majoritarian set if for every $\hat{P},\hat{P}^{\prime}\in\hat{\mathbb{P}}$ it holds that $\hat{F}(\hat{P},\mathbf{c})=\hat{F}(\hat{P}^{\prime},\mathbf{c})$ if $\hat{f}_{\_}{q}(\hat{P})=\hat{f}_{\_}{q}(\hat{P}^{\prime})$, where $q=\lfloor\frac{n}{2}\rfloor+1$ and $n$ is the number of agents in $\hat{P}$ and $\hat{P}^{\prime}$.

Since classical aggregators based on the majoritarian set use as an input not the entire profile but just the set of majority judgments their definitions can be easily extended to handle profiles of probabilistic judgments as well. Proposition 3.1 proves that the latter is not necessary.

First, recall the uniform quota rule for “classical” JA [20]. For profiles $P=(J_{\_}1,\ldots,J_{\_}n)$ of crisp judgment sets, the crisp uniform quota function $U_{\_}{q}$ is defined to give as output the set of those judgments that are in at least $q$ judgment sets in $P$:

[TABLE]

Let $P_{\_}\mathbf{c}=(\textrm{crisp}(\hat{J}_{\_}1,\mathbf{c}),\ldots,\textrm{crisp}(\hat{J}_{\_}n,\mathbf{c}))$ be the profile obtained by crispifying each probabilistic judgment set in a $\hat{\mathbb{P}}$-profile $\hat{P}=(\hat{J}_{\_}1,\ldots,\hat{J}_{\_}n)$ by a vector $\mathbf{c}$. We show that first calculating $\hat{f}_{\_}{q}(\hat{P})$ and then crispifying is the same as first crispifying each judgment sets in the profile into $P_{\_}\mathbf{c}$ and then applying $U_{\_}{q}$ to this $P_{\_}\mathbf{c}$. Namely, we show that the quota function commutes with the crispifying function.

Proposition 3.1**.**

For every $\hat{P}\in\hat{\mathbb{P}}$, crispifying coefficients $\mathbf{c}$, and quota $q\leq n$ it holds that $\hat{F}_{\_}{q}(\hat{P},\mathbf{c})=U_{\_}{q}(P_{\_}\mathbf{c}).$

Proof.

We prove that $\varphi\in\hat{F}_{\_}{q}(\hat{P},\mathbf{c})$ iff $\varphi\in U_{\_}{q}(P_{\_}\mathbf{c})$. Consider $\varphi\in\Phi$. The proof is similar for $\neg\varphi\in\Phi^{\cup}$. We have that $\varphi\in\hat{F}_{\_}{q}(\hat{P},\mathbf{c})$ iff there exists at least $q$ agents $k$ s.t. $a^{k}_{\_}{\varphi}\geq c_{\_}{\varphi}$. This is the case iff there are at least $q$ agents $k$ in $P_{\_}\mathbf{c}$ s.t. $\varphi\in J_{\_}k$. Thus necessarily $\varphi\in\hat{f}_{\_}{q}(P_{\_}\mathbf{c})$ and we get $\varphi\in\hat{F}_{\_}{q}(\hat{P},\mathbf{c})$ iff $\varphi\in U_{\_}{q}(P_{\_}\mathbf{c})$. ∎

Proposition 3.1 shows that we can use classical aggregators based on the majoritarian set to aggregate likelihood judgments. We simply crispify the profile first and then apply the classical aggregator. As a consequence, however, we can conclude that finding the collective judgments for probabilistic profiles is as computationally hard as for crisp profiles when these aggregators are used. Complexity results for these aggregators are given in [25, 16].

Let us consider the weighted majoritarian aggregation rules defined in [27]. These rules, in addition to using a (crisp) majoritarian set as input, also use the number $N(P,\varphi)$ of agents that support each judgment in that majoritarian set:

[TABLE]

In general, according to [27], a classical irresolute aggregation function $\bar{F}$ is based on the weighted majoritarian set if for every two JA-profiles $P$ and $P^{\prime}$, $N(P,\varphi)=N(P^{\prime},\varphi)$ implies $\bar{F}(P)=\bar{F}(P^{\prime})$, for every $\varphi\in\Phi^{\cup}$.

An example of such an aggregator is the median rule of [27]. We give the definition of this aggregator using our notation:

[TABLE]

where $N(P,\varphi)$ is defined as in Eq.(12).

Proposition 3.1 shows that the weighted majoritarian set can also be directly used to aggregate likelihood judgments. This can be done by generalizing the definition of $N(P,\varphi)$.

Let us define $\hat{N}(\hat{P},\varphi,c)$ to be the number of agents that assign to $\varphi$ a likelihood greater than or equal to some $c\in[0,1]$ in the profile $\hat{P}$:

[TABLE]

When each of the judgment sets in the profile $\hat{P}$ is crispified by a vector of coefficients $\mathbf{c}=(c_{\_}{\varphi})_{\_}{\varphi\in\Phi^{\cup}}$, such that $c_{\_}{\varphi}=c$, then $\hat{N}(\hat{P},\varphi,c)$ is exactly $N(P,\varphi)$ for the resulting crisp profile $P$.

However, we do not have to constrain ourselves with just using $\hat{N}(\hat{P},\varphi,c)$, we can further generalize the weighted majoritarian rules of [27] to consider not only how many agents assigned a likelihood over the threshold but also the likelihoods they do assign. This is one of the ways in which we can obtain direct aggregators.

3.4 Direct aggregators

Let us consider again the median rule of [27]. We can define the median likelihood aggregator to generalize the median rule.

Definition 3.5**.**

Given a profile $\hat{P}$, the median likelihood aggregator is defined as

[TABLE]

where

[TABLE]

The median likelihood aggregator assigns to a given profile the classical judgment set that gives the maximum sum of likelihoods assigned by all the agents to all the issues in it. E.g., the outcome using $\widehat{\textrm{{med}}}$ for the Example 1.1 profile is the crisp judgment set $\{s\vee t,h,x,\neg e\}$ with a “score” $\sum_{\_}{\varphi\in J}S(\varphi,\hat{P})$ of 16.8.

Recall that, for a crisp profile $P$, the median rule is defined as in Eq.(13). Proposition 3.2 is straightforward.

Proposition 3.2**.**

$\textrm{{med}}(P)=\widehat{\textrm{{med}}}(P^{*})$.

We now define three classes of direct aggregators.

Sequential direct aggregators.

An intuitive way to define direct aggregators is to aggregate the judgments issue-by-issue in a sequence by first “settling” the judgment on the issue for which the agents have assigned the highest likelihood. To do this, we need to define what it means for a judgment to have “the highest likelihood” in a profile. Several options exist and each of them leads to a different aggregator. We consider only one here, in order to illustrate the process.

We define “the highest likelihood” to be the highest average likelihood assigned to a judgment in a profile.

Definition 3.6** (Average likelihood).**

Given $\varphi\in\Phi^{\cup}$ and a profile $\hat{P}$, the average likelihood for $\varphi$ in $\hat{P}$ is defined as

[TABLE]

Note that since, in general, we have likelihood judgments with inequalities, these average likelihoods are actually average minimal likelihoods. Equivalently, we could have a vector of average maximal likelihoods taking $1-a_{\_}{\neg\varphi}$ instead of $a_{\_}{\varphi}$ for every $\varphi$. We could possibly consider linear weighted average or any other opinion pooling function to define ${E_{\hat{P}}(\varphi)}$ in Eq.(17).

Let $\mathbf{a}_{\_}{\hat{P}}$ be the vector of average likelihoods assigned to each $\varphi\in\Phi^{\cup}$ given a profile $\hat{P}$. Namely

[TABLE]

The sequential average aggregator $\hat{F}_{S}$ builds a crisp collective judgment set sequentially, adding first as many as possible of the judgments with highest average likelihood then moving on to judgments with the next highest average likelihood and adding them only if they are consistent with the already added judgments (skipping them otherwise).

E.g., for the profile in Table 3 we have the following:

[TABLE]

We obtain $\hat{F}_{S}(\hat{P})=\{\neg e,x,s\vee t,h,a\}$, with the judgments written in the order in which they were added. If instead we had

[TABLE]

for some profile $\hat{P}^{\prime}$, after adding $\{\neg e,x,\neg h\}$ next we should have had to add $s\vee t$ because its average likelihood is $0.56$. But since $\{\neg e,x,\neg h,s\vee t,a\}$ is not consistent with $\Gamma$, we would obtain $\hat{F}_{S}(\hat{P})=\{\neg e,x,\neg h,\neg(s\vee t),a\}$.

For likelihood profiles $P^{*}$ corresponding to a crisp profile $P$, we have that

[TABLE]

where leximax is the non-probabilistic judgment aggregation rule defined in [31, 17]. We omit the definition of leximax here due to space issues and the triviality of the proof of Eq.(19).

Many different functions can be defined using the average likelihood. The immediate approach would be to build aggregators inspired by the class of scoring rules [7]. Furthermore, we can work with not only the mean but also with max, min or otherwise polled individually assigned likelihoods.

Next we focus on the class of distance-based aggregation functions.

Distance-based aggregation

Distance-based aggregators aggregate profiles by considering all possible collective outcomes and choosing the one that is “most similar” to the profile at hand. Similarity is defined by a distance measure - the greater the distance between two judgment sets, the less similar they are. Distance from a profile to an outcome (judgment set) is defined as the sum or the maximum of the distances between the outcome and each of the judgment sets to the profile. Thus, to define a direct aggregator using the distance-based approach, we need to define a distance from a crisp judgment set to a likelihood judgment set. To do this, recall that a crisp judgment can be seen as special case of a likelihood judgment.

Given a distance function $d$ (defined over vectors of reals) we can define a distance-based aggregation function $\hat{F}_{\_}d$ as

[TABLE]

where the distance between two judgment sets is defined as:

[TABLE]

Alternatively instead of sum we can use max:

[TABLE]

When we take $d$ to be the Euclidean distance, (20) becomes

[TABLE]

where

[TABLE]

E.g., the outcome of applying the rule in Eq.(23) to the profile in the Example 1.1 is the crisp judgment set $\{s\vee t,h,x,\neg e,a\}$ at a distance 9.17 from the profile, see Table 4.

Numerous statistical distance measures can be used, [5] offers variety of examples. Further research is needed to establish what distance measure is a good choice.

In “classical” judgment aggregation, the distance-based aggregator, also known as the Kemeny rule [27] is defined as follows. Let $d_{\_}H$, the Hamming distance, between two crisp judgment sets $J_{\_}1$ and $J_{\_}2$ on the same crisp frame $(\Phi,N,\Gamma)$ be defined as the number of judgments on which $J_{\_}1$ and $J_{\_}2$ differ. For example, for $J_{\_}1=\{p,p\rightarrow q,q\}$ and $J_{\_}2=\{\neg p,p\rightarrow q,\neg q\}$ we have $d_{\_}H(J_{\_}1,J_{\_}2)=2$. The Kemeny rule, for a given $\mathcal{J}$ is defined as

[TABLE]

We can however observe that the Euclidean distance is the same as the Hamming distance when the likelihood judgment values are in $\{0,1\}$ and thus we obtain that Proposition 3.3 holds.

Proposition 3.3**.**

$F^{d_{\_}H,\Sigma}(P)=\hat{F}^{e,\Sigma}(P^{*}).$**

If we use the $L1$ distance (namely, the sum of differences between absolute values issue-by-issue), we obtain another generalization of the Hamming distance.

It is well known that for every crisp frame $(\Phi,N,\Gamma)$ it holds that $\textrm{med}(P)=F^{d_{\_}H,\Sigma}(P)$, see for example [27] for a formal proof. This relationship does not extend to $\hat{F}^{e,\Sigma}$ and $\widehat{\textrm{{med}}}$. We give a counter example. Consider the profile on Table 5. For this profile, $\hat{\Gamma}=\top$ and $\Gamma=\top$. All the rational judgment sets for $\Phi=\{p,p\rightarrow q,q\}$, $\hat{\Gamma}=\top$ and $\Gamma=\top$ are given in Table 6. We have that $\widehat{\textrm{{med}}}(\hat{P})=\{p,\neg(p\rightarrow q),\neg q\}$, while $\hat{F}^{e,\Sigma}(\hat{P})=\{p,p\rightarrow q,q\}.$

The relationship between the distance-based aggregator and the median rule is broken when the $L1$ distance is used as well and a counter example is not difficult to be found. This is because the relationship between the judgment on $\varphi$ and the judgment on $\neg\varphi$ is broken - it is not always the case that the likelihood of $\varphi$ is a function of the likelihood of $\neg\varphi$.

Lastly we consider a new class of direct aggregators that are not reducible to “classical” JA aggregators.

Most likely prime implicant

One of the oldest and most studied aggregators in “classical” JA is the so called premise-based procedure (PBP) [11]. Some aggregation problems are such that the agenda can be naturally split into two sets: conclusions (or decisions) and premises (or reasons why a decision is taken). For example, the agenda in the example in Table 1 can naturally be split into an agenda of premises $\Phi_{\_}p=\{p,p\rightarrow q\}$ and an agenda of conclusions $\Phi_{\_}c=\{q\}$. The PBP aggregator works in two steps: first the majority is calculated for each issue in the premise agenda subset. In the example in Table 1 this would yield the set of premises $\{p,p\rightarrow q\}$. Then the constraint is used to entail the judgments on the issues in the conclusion agenda subset. In the example in Table 1 this would yield the collective judgment set $\{p,p\rightarrow q,q\}$. PBP is an aggregator that has many good properties, but it is only applicable to agendas that are split into premises and conclusions. Here we propose a likelihood judgment aggregator that “operates” in the same way as PBP but it is applicable to any agenda.

When the agenda is split into premises and conclusions, the problem is such that the judgments on the premises entail each of the judgments on the conclusions. From a logical perspective, the set of premises is an implicant of the agenda. Let us formally define this concept generalizing it to any agenda.

Definition 3.7**.**

Given an agenda $\Phi$ (not explicitly partitioned into premises and conclusions) and constraints $\Gamma$ we say that the set $I\subseteq\Phi^{\cup}$ is an implicant of $\Phi$ if $I$ is a consistent (with respect to $\Gamma$) set and either $I\cup\Gamma\models\varphi$ or $I\cup\Gamma\models\neg\varphi$, for every $\varphi\in\Phi^{\cup}\setminus I$. $I$ is a prime implicant of $\Phi$ if $I$ is an implicant and there exists no smaller set $I^{\prime}$ ($I^{\prime}\subset I$) that is also an implicant of $\Phi$.

Consider the agenda and constraints of Example 1.1. This agenda has eight prime implicants, i.e., all the consistent three-element subsets of $\{x,\neg x,e,\neg e,s\vee t,\neg(s\vee t),a,\neg a\}$.

We can define a class of irresolute likelihood aggregation functions based on agenda prime implicants and a definition of most likely prime implicant. There are several ways a most likely prime implicant can be defined. We give a few examples. Let $\mathcal{I}(\Phi)$ be the set of all prime implicants of $\Phi$. Then the most likely prime implicant of $\Phi$ is the one with:

• •

the highest sum of average likelihoods

$\underset{I\in\mathcal{I}(\Phi)}{\textrm{argmax}}\sum_{\_}{\varphi\in I}{E_{\hat{P}}(\varphi)}$

• •

the highest minimum average likelihood,

$\underset{I\in\mathcal{I}(\Phi)}{\textrm{argmax}}\min_{\_}{\varphi\in I}{E_{\hat{P}}(\varphi)}$

• •

the highest number of majority supported judgments,

$\underset{I\in\mathcal{I}(\Phi)}{\textrm{argmax}}\sum_{\_}{\varphi\in I}\hat{N}(\hat{P},\varphi,c)$, etc.

Note that the three definitions given above determine three (possibly) different most likely prime implicants for which we could use different names, but for simplicity we omit that.

Once a most likely prime implicant $I^{\ast}$ is determined in one of the above described ways, the collective judgment is a union of $I^{\ast}$ and the elements of $\Phi^{\cup}$ implied by $I^{\ast}$.

Example 3.1**.**

For the profile, agenda and constraints of Example 1.1, the prime implicant that has the highest sum of average likelihoods is $\{s\vee t,x,\neg e\}$ yielding the collective outcome of $\{s\vee t,x,\neg e,h,a\}$.

If the agenda and constraints are given in DNF (disjunctive normal form), the prime implicants can be found in polynomial time [39]. To the best of our knowledge, prime implicants have not been used to define aggregation functions in judgment aggregation, with the possible exception of [36] where a distance based function for measuring dissimilarity between two classical judgment sets based on prime implicants has been defined.

4 Properties of aggregators

Having generalized the classical judgment aggregation framework, the immediate question to consider is whether the typical impossibility properties results also hold for aggregators of probabilistic judgments. To establish this, we need to generalize the definitions of aggregation properties. We also need to see whether there are new interesting desirable properties that need to be considered in the new framework. We begin some of this work here.

We begin by exploring the “classical” impossibility theorem [6]. For this we need to define a resolute likelihood aggregator. We then define the properties of universal domain, unanimity, rationality and systematicity.

Definition 4.1**.**

Let $(\Phi,N,\Gamma,\hat{\Gamma})$ be a probabilistic aggregation frame and let $\hat{\mathbb{P}}$ be the set of all rational likelihood profiles for it. A likelihood (resolute) aggregator $R$ is a mapping $R:\hat{\mathbb{P}}\rightarrow\mathcal{J}$ from the set of rational likelihood profiles to the set of consistent and complete crisp judgment sets.

In other words, a (crispifying) aggregator is resolute if it assigns only one collective judgment set to each profile.

Universal domain is the requirement that an aggregator $R$ has to be defined for all the probabilistic rational profiles (and all allowable crispifying vectors where applicable). Rationality is the property that $R$ produces only consistent and complete crisp judgment sets. These properties are embedded in the Definition 4.1.

An aggregator $R$ is dictatorial if there is an information source $k\in N$ (a dictator) such that for each likelihood profile $\hat{P}=(\hat{J}_{\_}1,\ldots,\hat{J}_{\_}n)$, the collective judgment is equal to the collective judgment on the profile $\hat{P}_{\_}k=(\hat{J}_{\_}k,\ldots,\hat{J}_{\_}k,\ldots,\hat{J}_{\_}k)$, i.e., only the judgment set of the dictator is considered in the aggregation process. Non-dictatorship is the requirement that no information source is a dictator.

In “classical” judgment aggregation, unanimity is the property requiring that if a judgment is in every judgment set in the profile it has to be in the collective judgment set as well. When aggregating likelihood profiles, unanimity has to be defined with respect to some crispifying coefficient $c$, regardless of whether $R$ is a direct aggregator or not.

Definition 4.2** (Unanimity).**

Let $c\in[0,1]$. The aggregator $R$ satisfies $\mathbf{c}$-unanimity* if for every profile of rational probabilistic judgments $\hat{P}\in\hat{\mathbb{P}}$, $\hat{P}=(\hat{J}_{\_}1,\ldots,\hat{J}_{\_}n)$, and every $\varphi\in\Phi^{\cup}$ it holds that: if $\forall k\in N$: $a^{k}_{\_}{\varphi}\geq c$, then $\varphi\in R(\hat{P})$.*

Lastly we define systematicity. Intuitively, systematicity is satisfied if every two issues that are judged as equally probable in two different profiles are treated equivalently by the aggregation rule $R$.

Definition 4.3** (Systematicity).**

Given a profile $\hat{P}\in\hat{\mathbb{P}}$, let us define $\hat{P}_{\_}{\varphi}$ to be the projection of $\hat{P}$ on the issue $\varphi$: $\hat{P}_{\_}{\varphi}=(\ell(\varphi)\geq^{*}a^{k}_{\_}{\varphi}\mid k\in N)$. The aggregator $R$ satisfies systematicity, if for every two profiles $\hat{P},\hat{P}^{\prime}\in\hat{\mathbb{P}}$ and issues $\varphi,\psi\in\Phi^{\cup}$, the following holds: $\hat{P}_{\_}{\varphi}=\hat{P}^{\prime}_{\_}{\psi}$ implies [$\varphi\in R(\hat{P})$ iff $\psi\in R(\hat{P}^{\prime})$].

The following theorem can easily be proved following the proof method of Theorem 3.7. in [20].

Theorem 4.1**.**

Consider a frame $(\Phi,N,\Gamma,\hat{\Gamma})$. Let $\hat{\mathbb{P}}$ be the set of all rational likelihood profiles that can be defined for the given frame. The aggregation function $R$ satisfies unanimity, rationality and systematicity if and only if $R$ is a dictatorial aggregation function.

For other desirable properties that could be applied in our framework we can look to the literature of probabilistic opinion pooling [29, 9] for inspiration.

The systematicity requirement corresponds to a property called Strong setwise function property (SSFP) in opinion pooling [29]. This property requires that the group probability of an event $A$ depends only on the individually assigned probabilities of $A$. It was shown that SSFP gives rise to impossibility results in opinion pooling [29]. A weaker property, that is implied by SSFP, is the Zero Preservation Property (ZPP). ZPP is satisfied when for profiles where all of the individually assigned probabilities of an issue are zero, the collective probability of this issue is also zero. The ZPP property is related to the unanimity property in “classical” JA. Here we can define ZPP as $1$-unanimity, namely $\mathbf{c}$-unanimity where $c=1$.

Intuitively, 1-unanimity is desirable: whenever every source is sure that an issue is true (or false), a 1-unanimity satisfying aggregator will capture that certainty. However, unanimity on $\ell(\varphi)=1$ for $\varphi$ does not mean that a rational judgment set $J$ such that $\varphi\in J$ exists! Recall that we have constraints $\Gamma$ imposed by the agenda setter on the outcome but not on the judgment sets of the profile. This means that 1-unanimity can only be satisfied in a specific aggregation frame if the set of constraints $\Gamma$ of the frame allows it. Let us give an example.

Example 4.1**.**

Let $(\Phi,N,\Gamma,\hat{\Gamma})$ be a probabilistic aggregation frame such that $\varphi_{\_}1,\varphi_{\_}2\in\Phi$ and $\{\varphi_{\_}1\rightarrow\varphi_{\_}2\}\in\Gamma$. Let $\hat{P}$ be a consistent probabilistic profile in this frame such that $\ell(\varphi_{\_}1)=1$ and $\ell(\varphi_{\_}2)=0$ are in every individual judgment set in $\hat{P}$. Then 1-unanimity requires that $\{\varphi_{\_}1,\neg\varphi_{\_}2\}\subseteq J$ for every $J\in R(\hat{P})$ but none such $J$ is in $\mathcal{J}$.

As seen in the above example, whether an aggregation function can satisfy 1-unanimity is a property of the aggregation frame. We can define the ZPP property for likelihood aggregators $R$ (the definition also extends to irresolute direct and indirect aggregators as well) as 1-unanimity when the constraints allow it.

Definition 4.4** (Zero preservation property).**

Let $(\Phi,N,\Gamma,\hat{\Gamma})$ be a probabilistic aggregation frame. Given a profile $\hat{P}\in\hat{\mathbb{P}}$, we define the set $Z(\hat{P})$ as

[TABLE]

Namely, the set $Z(\hat{P})$ contains all the agenda issues that have been unanimously awarded likelihood 1 in $\hat{P}$. We say that an aggregator $R$ satisfies the zero preservation property if for all $\hat{P}\in\hat{\mathbb{P}}$ it holds that if $Z(\hat{P})$ is consistent with $\Gamma$, then $Z(\hat{P})\subseteq R(\hat{P})$.

We consider one more intuitive property of probabilistic aggregators, that of convexity [29]. Convexity states that the collectively assigned (minimal) probability on an issue should be a value no smaller than the smallest and no higher than the highest individually assigned probability on that issue. For the direct aggregators this property is not applicable, but it is so for the crispifying ones.

Definition 4.5** (Convexity).**

Let $\hat{J}$ be one of the probabilistic judgment sets assigned to profile $\hat{P}$ by a function $\hat{F}$ (before crispification). For a given $\varphi\in\Phi^{\cup}$, let $a^{\max}_{\_}{\varphi}=\max_{\_}{k}a^{k}_{\_}{\varphi}$ and $a^{\min}_{\_}{\varphi}=\min_{\_}{k}a^{k}_{\_}{\varphi}$. We say that $\hat{F}$ satisfies convexity when for all collective $\hat{J}$ and every $\varphi\in\Phi^{\cup}$, if $\ell(\varphi)\geq^{*}a_{\_}{\varphi}\in\hat{J}$ then $a^{\min}_{\_}{\varphi}\leq a_{\_}{\varphi}\leq a^{\max}_{\_}{\varphi}$.

It can directly be observed that $\hat{f}_{\_}{q}(\hat{P})$ satisfies convexity and that $\hat{F}_{\_}{q}(\hat{P},\mathbf{c})$ satisfies ZPP and universal domain. However $\hat{F}_{\_}{q}(\hat{P},\mathbf{c})$ does not satisfy rationality because the sets in $\hat{F}_{\_}{q}(\hat{P},\mathbf{c})$ are not always rational sets of judgments.

The $\hat{F}_{S}$ aggregator satisfies universal domain and non-dictatorship by design. It is clear that $\hat{F}_{\_}S$ also satisfies ZPP.

The distance-based direct aggregators satisfy universal domain and non-dictatorship by design, however it is safe to conjecture that they will not satisfy ZPP for the same reason that distance-based classical aggregators do not satisfy unanimity [32] – for an agenda with sufficiently many issues, a judgment set that does not contain the unanimously likely judgment might end up being “closer” to the profile.

With the most likely prime implicant class of aggregators, universal domain and rationality will be satisfied by design. However, ZPP will not be satisfied - when the unanimously supported issue is not in the prime implicant its inclusion in the collective judgment set will not be guaranteed.

5 Related work

As mentioned in the introduction, the area of probabilistic opinion pooling is concerned with aggregating probability functions into a single one. As opposed to standard probabilistic opinion pooling, our logic-based approach:

allows for an arbitrary agenda, namely instead of taking the entire $\sigma$-algebra, the agenda can be limited to the important issues of consideration in the actual context; 2. 2.

We do not limit ourselves to expressing point probabilities over the issues (but we do include that option as well); 3. 3.

The result of the opinion aggregation is a set of propositional statements, hence a final decision, and not a probabilistic consensus.

Dietrich and List [9] generalize opinion pooling to general agendas and examine properties and impossibility results. However, their work does not define any particular aggregators and also 2) and 3) are not the case there.

The problem of transforming degrees of belief into binary beliefs is known as belief-binarization. Dietrich and List [10] study how a profile of Boolean judgments, that has been transformed into a vector of beliefs (for example a profile from Table 1 becomes the vector $(\frac{2}{3},\frac{2}{3},\frac{1}{3})$) can be “binarized” into a consistent set of Boolean judgments. In [10], however, only binary profiles are aggregated.

There are several approaches towards aggregating imprecise probabilities (IP), like aggregation of probability intervals in [30], subjective opinion fusion in [23], etc. More recently [38] extended pooling properties to IPs using convex functions. Moreover, they go further and aggregate precise probabilities into imprecise (the convex hull of the input probabilities as a proof of concept) arguing that IP models are better suited as models of rational consensus. Allowing for inequalities in the likelihood judgments, we allow for modeling IP in the individual judgments. However, unlike [38] we require the collective judgment to be crisp since our goal is to define specific aggregators that support the decision making in various contexts.

Dietrich and List [6] generalize classical JA assuming formulas from a general logic and prove impossibility theorems. They show that the model is applicable to, for example, propositional, modal, and many-valued logic. The model is not directly applicable to the likelihood logic we use here, since it assumes that the agenda issues are formulas in the particular logic. Since it does not make sense to choose a finite agenda of likelihood formulas, we express the issues in propositional logic and use likelihood formulas for the judgments. The latter makes our framework fit better in the general theory of aggregation of propositional attitudes in [8], that integrates probabilistic opinion pooling and judgment aggregation. In this theory, profiles consist of attitude functions (which can be probability functions, truth-value functions, etc.) defined over finite subset of a $\sigma$-algebra (an agenda). We believe that defining notions on the level of a syntax has certain advantages, explicitly defining the concept of rationality being one of them.

It is not always possible to have complete information, sometimes some sources will not be able to provide information on all of the issues. Although impossibility results involving abstentions have been shown [12], designing functions to aggregate the so called incomplete judgments is not given a lot of attention in the JA literature [40, 35]. By showing how crisp profiles can be represented as likelihood ones and designing likelihood judgment aggregators, we enable probabilistic judgment aggregation to also be used for aggregating crisp incomplete judgment sets in a straightforward way.

Interpreting the likelihood operator with a possibility measure leads to the formula $\ell(\varphi)\geq a$ being equivalent to the formula $(\varphi,a)$ as well as the (uniform) crispifying being equivalent to $\alpha$-cut in possibility theory. There are various methods of information fusion considered in this theory [13]. However, they focus on merging information about the true state of a variable or a proposition and take a set theoretic approach to defining the merging functions while we, on the other hand, follow the tradition of judgment aggregation and social-choice theory and take an agenda of (logically related) issues as a starting point. This means that both the (choice of) definitions of the aggregators, and the choice of crispifying coefficients depend on the agenda. Moreover, our goal is not just to merge the imprecise information coming from the different sources, but to make a decision about the true state of the agenda issues.

Probabilistic belief merging is considered in [33, 34]. In belief merging sets of formulas, possibly likelihood formulas, are called knowledge bases. Knowledge bases from several sources, that can be mutually inconsistent, are merged to obtain a consistent knowledge base. The difference between belief merging and judgment aggregation has been analyzed in [18]. Essentially, in belief merging the knowledge bases do not share the same agenda, which entails different properties to be desired for the merging operators as compared to the desired properties for judgment aggregation functions.

6 Conclusions and future work

We consider as the main contribution of our paper the definition of various functions for aggregating likelihood judgments on logically related issues. Furthermore, we show how these aggregators relate to classical judgment aggregation function, and in turn, through the results shown in [24] and [14], how likelihood judgment aggregation relates to voting methods. We also define desirable properties for the aggregation functions and show that the classical impossibility results hold here as well.

Some more consideration needs to be given to further distinguish the likelihood profile aggregators. From the examples we can observe that very different outcomes are produced for the same profile by different aggregators. The minimal set of properties we discuss is not sufficient to allow a user to choose which is the best aggregator for a given probabilistic frame. In light of new properties, particularly the direct aggregators need to be carefully studied.

More properties from opinion pooling can be considered and “translated” into our extended JA framework. An interesting candidate is the so called Independence Preservation (IP) property which intuitively requires that if two issues are probabilistically independent according to all information sources, then this independence should be preserved in the collectively assigned probabilities for the two issues. This has been explored by Wagner in [41] for the case of aggregating point probabilities over an agenda of mutually exclusive events; [38] explores the imprecise probabilities case. Note that to represent probabilistic independencies in the judgments, we need to either extend the logic of likelihood with polynomial likelihood formulas or include likelihood independence formulas as defined in [22] directly in the language. Then we could define IP properties alike [41] and [38] and see what are the consequencies of their impossibility results in our platform. We notice also that the IP property is reminiscent of the agenda separability property studied in [26]. One direction of future work is to establish this intuitive connection and explore other such connections between probability aggregation properties and JA properties.

We believe that with this work we have made several contributions to the classical JA theory as well: We have significantly extended the classical binary judgment aggregation framework, opening up this social choice method for applications in new AI domains, particularly involving the aggregation of uncertain judgments; our framework allows for not only uncertainties but also abstentions to be modelled using $\ell(\varphi)\geq 0$, which is a neglected feature in judgment aggregation frameworks overall; furthermore, we generalize the assumption that the same relations between issues should hold for both the information sources and the aggregated result. This “double constraint” framework is actually very intuitive [14].

Having a probabilistic framework also opens possibilities to study the *truth-tracking * properties of judgment aggregators, namely how good is a function in aggregating profiles into the most likely judgments. This area of judgment aggregation is still relatively little explored [4]. We intend to explore truth-tracking in future work.

We believe that there is a possibility for applying our work to prediction markets [42], specifically in extending the agenda of predictions (which is typically consisting of states of a random variable) to a set of logically related statements. Prediction markets [2] are forums for trading contracts for outcomes of future events. Each market participant possesses certain information about the event in question, and conveys this information to the market by the way she trades contracts. The contract price is a result of aggregation of the information possessed by all the participants, hence is an estimator of the probability of the event in question. We believe we could seek for inspiration in defining new aggregators under our platform by studying the methods of information fusion that various prediction markets apply.