On the Semantics and Complexity of Probabilistic Logic Programs

TL;DR

This paper explores the meaning and computational complexity of probabilistic logic programs, analyzing different semantics and their implications for inference tasks in various program classes.

Contribution

It characterizes the credal semantics as sets of probability models dominating monotone capacities and analyzes inference complexity across program types.

Findings

Credal semantics produce sets of models dominating monotone capacities.

Inference complexity varies from counting hierarchy to exponential levels.

Results apply to acyclic, stratified, and cyclic programs.

Abstract

We examine the meaning and the complexity of probabilistic logic programs that consist of a set of rules and a set of independent probabilistic facts (that is, programs based on Sato's distribution semantics). We focus on two semantics, respectively based on stable and on well-founded models. We show that the semantics based on stable models (referred to as the "credal semantics") produces sets of probability models that dominate infinitely monotone Choquet capacities, we describe several useful consequences of this result. We then examine the complexity of inference with probabilistic logic programs. We distinguish between the complexity of inference when a probabilistic program and a query are given (the inferential complexity), and the complexity of inference when the probabilistic program is fixed and the query is given (the query complexity, akin to data complexity as used in database theory). We obtain results on the inferential and query complexity for acyclic, stratified, and cyclic propositional and relational programs, complexity reaches various levels of the counting hierarchy and even exponential levels.

Tables

Table 1. Table 1: Total choices, the induced assignments, and their probabilities, for Example 10 .

$𝖺$	$𝖻$	$\mathrm{𝖼𝗈𝗅𝖽}$	$\mathrm{𝗁𝖾𝖺𝖽𝖺𝖼𝗁𝖾}$	Probability
$\mathrm{𝗍𝗋𝗎𝖾}$	$\mathrm{𝗍𝗋𝗎𝖾}$	$\mathrm{𝖿𝖺𝗅𝗌𝖾}$	$\mathrm{𝖿𝖺𝗅𝗌𝖾}$	$0.34\times 0.25=0.085$
$\mathrm{𝗍𝗋𝗎𝖾}$	$\mathrm{𝖿𝖺𝗅𝗌𝖾}$	$\mathrm{𝗍𝗋𝗎𝖾}$	$\mathrm{𝗍𝗋𝗎𝖾}$	$0.34\times 0.75=0.255$
$\mathrm{𝖿𝖺𝗅𝗌𝖾}$	$\mathrm{𝗍𝗋𝗎𝖾}$	$\mathrm{𝗎𝗇𝖽𝖾𝖿𝗂𝗇𝖾𝖽}$	$\mathrm{𝗎𝗇𝖽𝖾𝖿𝗂𝗇𝖾𝖽}$	$0.66\times 0.25=0.165$
$\mathrm{𝖿𝖺𝗅𝗌𝖾}$	$\mathrm{𝖿𝖺𝗅𝗌𝖾}$	$\mathrm{𝖿𝖺𝗅𝗌𝖾}$	$\mathrm{𝗍𝗋𝗎𝖾}$	$0.66\times 0.75=0.495$

Table 2. Table 2: Complexity results. All entries refer to completeness with respect to many-one reductions. Columns “Propositional”, “ 𝖯𝖫𝖯 𝖻 subscript 𝖯𝖫𝖯 𝖻 \mathsf{PLP_{b}} ”, and “ 𝖯𝖫𝖯 𝖯𝖫𝖯 \mathsf{PLP} respectively refer to the inferential complexity of propositional plp s, the inferential complexity of plp s with a bound on predicate arity, and plp s with no bound on predicate arity. Column “Query” refers to the query complexity of relational plp s.

	Propositional	${\mathrm{𝖯𝖫𝖯}}_{𝖻}$	$\mathrm{𝖯𝖫𝖯}$	Query
Acyclic	$\mathrm{𝖯𝖯}$	${\mathrm{𝖯𝖯}}^{\mathrm{𝖭𝖯}}$	$\mathrm{𝖯𝖤𝖷𝖯}$	$\mathrm{𝖯𝖯}$
Stratified	$\mathrm{𝖯𝖯}$	${\mathrm{𝖯𝖯}}^{\mathrm{𝖭𝖯}}$	$\mathrm{𝖯𝖤𝖷𝖯}$	$\mathrm{𝖯𝖯}$
General, credal	${\mathrm{𝖯𝖯}}^{\mathrm{𝖭𝖯}}$	${\mathrm{𝖯𝖯}}^{{\mathrm{𝖭𝖯}}^{\mathrm{𝖭𝖯}}}$	?	${\mathrm{𝖯𝖯}}^{\mathrm{𝖭𝖯}}$
General, well-founded	$\mathrm{𝖯𝖯}$	${\mathrm{𝖯𝖯}}^{\mathrm{𝖭𝖯}}$	$\mathrm{𝖯𝖤𝖷𝖯}$	$\mathrm{𝖯𝖯}$

Equations

$$\mathsf{up}:-\mathsf{actionUp},\mathbf{not}\mathsf{disturbanceUp}\text{mbox}.$$

$$\mathbb{P}\left(\mathsf{disturbanceUp}=\mathsf{true}\right)=\mathrm{0.1.}$$

$$\mathsf{sleep}:-\mathbf{not}\mathsf{work},\mathbf{not}\mathsf{insomnia}\text{mbox}.\mathsf{work}:-\mathbf{not}\mathsf{sleep}\text{mbox}.$$

$$\mathbb{P}\left(\mathsf{insomnia}=\mathsf{true}\right)=\mathrm{0.3.}$$

$$\mathbb{P}\left(X_1=x_1,\dots ,X_n=x_n\right)=\prod _{i=1}^n\mathbb{P}\left(X_i=x_i|\mathrm{pa}(X_i)={\pi }_i\right),$$

$$A_0:-A_1,\dots ,A_m,\mathbf{not}{A}_{m+1},\dots ,\mathbf{not}{A}_n\text{mbox}.$$

$$\begin{array}[]{l}\mathsf{smokes}(X)\colonminus\mathsf{stress}(X)\mbox{\large.}\\ \mathsf{smokes}(X)\colonminus\mathsf{influences}(Y,X),\mathsf{smokes}(Y)\mbox{\large.}\\ \mathsf{influences}(a,b)\mbox{\large.}\;\;\mathsf{influences}(b,a)\mbox{\large.}\;\;\mathsf{stress}(b)\mbox{\large.}\end{array}$$

$$\mathsf{wins}(X):-\mathsf{move}(X,Y),\mathbf{not}\mathsf{wins}(Y)\text{mbox}.$$

$$\mathsf{move}(a,b)\text{mbox}.\mathsf{move}(b,a)\text{mbox}.\mathsf{move}(b,c)\text{mbox}.\mathsf{move}(c,d)\text{mbox}.$$

$$\begin{array}[]{c}\mathsf{shaves}(X,Y)\colonminus\mathsf{barber}(X),\mathsf{villager}(Y),\mathbf{not}\;\mathsf{shaves}(Y,Y)\mbox{\large.}\\ \mathsf{villager}(a)\mbox{\large.}\quad\mathsf{barber}(b)\mbox{\large.}\quad\mathsf{villager}(b)\mbox{\large.}\end{array}$$

$$Q_1{\mathbf{X}}_1:Q_2{\mathbf{X}}_2:\dots \exists {\mathbf{X}}_k:\varphi ({\mathbf{X}}_0,{\mathbf{X}}_1,\dots ,{\mathbf{X}}_k),$$

$$Q_1{\mathbf{X}}_1:Q_2{\mathbf{X}}_2:\dots \forall {\mathbf{X}}_k:\varphi ({\mathbf{X}}_0,{\mathbf{X}}_1,\dots ,{\mathbf{X}}_k),$$

$$\begin{array}[]{l}0.7::\mathsf{burglary}.\qquad 0.2::\mathsf{earthquake}\mbox{\large.}\\ \mathsf{alarm}\colonminus\mathsf{burglary},\mathsf{earthquake},\mathsf{a1}\mbox{\large.}\\ \mathsf{alarm}\colonminus\mathsf{burglary},\mathbf{not}\ \mathsf{earthquake},\mathsf{a2}\mbox{\large.}\\ \mathsf{alarm}\colonminus\mathbf{not}\ \mathsf{burglary},\mathsf{earthquake},\mathsf{a3}\mbox{\large.}\\ 0.9::\mathsf{a1}\mbox{\large.}\quad 0.8::\mathsf{a2}\mbox{\large.}\quad 0.1::\mathsf{a3}\mbox{\large.}\\ \mathsf{calls}(X)\colonminus\mathsf{alarm},\mathsf{neighbor}(X)\mbox{\large.}\\ \mathsf{neighbor}(a)\mbox{\large.}\qquad\mathsf{neighbor}(b)\mbox{\large.}\end{array}$$

$$0.5::\mathsf{r}\text{mbox}.0.5::\mathsf{s}\text{mbox}.\mathsf{v}:-\mathsf{r},\mathsf{s}\text{mbox}.$$

$$0.5::\mathsf{r}\text{mbox}.0.6::\mathsf{r}\text{mbox}.0.2::\mathsf{s}(a)\text{mbox}.0.3::\mathsf{s}(X)\text{mbox}.\mathsf{v}:-\mathsf{r},\mathsf{s}(a),\mathsf{s}(b)\text{mbox}.$$

$$\mathsf{r}\text{mbox}.\mathsf{v}:-\mathsf{r},\mathsf{s}(a),\mathsf{s}(b)\text{mbox}.,$$

$$\begin{array}[]{l}\mathsf{smokes}(X)\colonminus\mathsf{stress}(X)\mbox{\large.}\\ \mathsf{smokes}(X)\colonminus\mathsf{influences}(Y,X),\mathsf{smokes}(Y)\mbox{\large.}\\ 0.3::\mathsf{influences}(a,b)\mbox{\large.}\;\;0.3::\mathsf{influences}(b,a).\;\;0.8::\mathsf{stress}(b)\mbox{\large.}\end{array}$$

$$\mathsf{smokes}(X):-\mathbf{not}\mathsf{stress}(X)\text{mbox}.$$

$$\begin{array}[]{c}\mathsf{path}(X,Y)\colonminus\mathsf{edge}(X,Y)\mbox{\large.}\\ \mathsf{path}(X,Y)\colonminus\mathsf{edge}(X,Y),\mathsf{path}(X,Y)\mbox{\large.}\\ 0.6::\mathsf{edge}(\mathsf{1},\mathsf{2})\mbox{\large.}\quad 0.1::\mathsf{edge}(\mathsf{1},\mathsf{3})\mbox{\large.}\quad 0.4::\mathsf{edge}(\mathsf{2},\mathsf{5})\mbox{\large.}\quad 0.3::\mathsf{edge}(\mathsf{2},\mathsf{6})\mbox{\large.}\\ 0.3::\mathsf{edge}(\mathsf{3},\mathsf{4})\mbox{\large.}\quad 0.8::\mathsf{edge}(\mathsf{4},\mathsf{5})\mbox{\large.}\quad 0.2::\mathsf{edge}(\mathsf{5},\mathsf{6})\mbox{\large.}\end{array}$$

$$\mathsf{p}:-\mathbf{not}\mathsf{q},\mathbf{not}\mathsf{r}\text{mbox}.\mathsf{q}:-\mathbf{not}\mathsf{p}\text{mbox}.\alpha ::\mathsf{r}\text{mbox}.$$

$$\mathsf{p}:-\mathbf{not}\mathsf{q},\mathbf{not}\mathsf{r}\text{mbox}.\mathsf{q}:-\mathbf{not}\mathsf{p}\text{mbox}.\mathsf{r}\text{mbox}.$$

$$\mathsf{p}:-\mathbf{not}\mathsf{q},\mathbf{not}\mathsf{r}\text{mbox}.\mathsf{q}:-\mathbf{not}\mathsf{p}\text{mbox}.$$

$$\begin{array}[]{c}0.9::\mathsf{man}(\mathsf{dilbert})\mbox{\large.}\\ \mathsf{single}(X)\colonminus\mathsf{man}(X),\mathbf{not}\;\mathsf{husband}(X)\mbox{\large.}\\ \mathsf{husband}(X)\colonminus\mathsf{man}(X),\mathbf{not}\;\mathsf{single}(X)\mbox{\large.}\end{array}$$

$$s_2$$

$$s_3$$

$$\begin{array}[]{c}\mathsf{cold}\colonminus\mathsf{headache},\mathsf{a}\mbox{\large.}\qquad\mathsf{cold}\colonminus\mathbf{not}\;\mathsf{headache},\mathbf{not}\;\mathsf{a}.\qquad 0.34::\mathsf{a}\mbox{\large.}\\ \mathsf{headache}\colonminus\mathsf{cold},\mathsf{b}\mbox{\large.}\qquad\mathsf{headache}\colonminus\mathbf{not}\;\mathsf{b}\mbox{\large.}\qquad 0.25::\mathsf{b}\mbox{\large.}\end{array}$$

$$\begin{array}[]{ccc}\mathbb{P}\!\left(\mathsf{cold}=\mathsf{true}\right)=0.255,&&\mathbb{P}\!\left(\mathsf{headache}=\mathsf{true}\right)=0.750,\\ \mathbb{P}\!\left(\mathsf{cold}=\mathsf{undefined}\right)=0.165,&&\mathbb{P}\!\left(\mathsf{headache}=\mathsf{undefined}\right)=0.165,\\ \mathbb{P}\!\left(\mathsf{cold}=\mathsf{false}\right)=0.580,&&\mathbb{P}\!\left(\mathsf{headache}=\mathsf{false}\right)=0.085.\end{array}$$

$$\begin{array}[]{c}\mathsf{color}(V,\mathsf{red})\colonminus\-\mathbf{not}\;\mathsf{color}(V,\mathsf{yellow}),\mathbf{not}\;\mathsf{color}(V,\mathsf{green}),\mathsf{vertex}(V)\mbox{\large.}\\ \mathsf{color}(V,\mathsf{yellow})\colonminus\-\mathbf{not}\;\mathsf{color}(V,\mathsf{red}),\mathbf{not}\;\mathsf{color}(V,\mathsf{green}),\mathsf{vertex}(V)\mbox{\large.}\\ \mathsf{color}(V,\mathsf{green})\colonminus\-\mathbf{not}\;\mathsf{color}(V,\mathsf{red}),\mathbf{not}\;\mathsf{color}(V,\mathsf{yellow}),\mathsf{vertex}(V)\mbox{\large.}\\ \mathsf{clash}\colonminus\mathbf{not}\;\mathsf{clash},\mathsf{edge}(V,U),\mathsf{color}(V,C),\mathsf{color}(U,C)\mbox{\large.}\end{array}$$

$$\begin{array}[]{c}\mathsf{color}(\mathsf{2},\mathsf{red})\mbox{\large.}\quad\mathsf{color}(\mathsf{5},\mathsf{green})\mbox{\large.}\\ 0.5::\mathsf{edge}(\mathsf{4},\mathsf{5})\mbox{\large.}\\ \mathsf{edge}(\mathsf{1},\mathsf{3})\mbox{\large.}\quad\mathsf{edge}(\mathsf{1},\mathsf{4})\mbox{\large.}\quad\mathsf{edge}(\mathsf{2},\mathsf{1})\mbox{\large.}\quad\mathsf{edge}(\mathsf{2},\mathsf{4})\mbox{\large.}\quad\mathsf{edge}(\mathsf{3},\mathsf{5})\mbox{\large.}\quad\mathsf{edge}(\mathsf{4},\mathsf{3})\mbox{\large.}\end{array}$$

$$\begin{array}[]{c}\mathsf{wins}(X)\colonminus\mathsf{move}(X,Y),\mathbf{not}\;\mathsf{wins}(Y)\mbox{\large.}\\ \mathsf{move}(\mathsf{a},\mathsf{b})\mbox{\large.}\quad\mathsf{move}(\mathsf{b},\mathsf{a})\mbox{\large.}\quad\mathsf{move}(\mathsf{b},\mathsf{c})\mbox{\large.}\quad 0.3::\mathsf{move}(\mathsf{c},\mathsf{d})\mbox{\large.}\end{array}$$

$$\begin{array}[]{c}\mathsf{shaves}(X,Y)\colonminus\mathsf{barber}(X),\mathsf{villager}(Y),\mathbf{not}\;\mathsf{shaves}(Y,Y).\\ \mathsf{villager}(a).\quad\mathsf{barber}(b).\quad 0.5::\mathsf{villager}(b).\end{array}$$

Full text

1 Introduction

The combination of deterministic and uncertain reasoning has led to many mixtures of logic and probability (?, ?, ?). In particular, combinations of logic programming constructs and probabilistic assessments have been pursued in several guises (?, ?, ?, ?, ?), and the topic has generated significant literature (?, ?).

Among probabilistic logic programming languages, the approach started by Poole’s probabilistic Horn abduction (?) and Sato’s distribution semantics (?) has been very popular. Basically, there a logic program is enlarged with independent probabilistic facts. For instance, consider a rule

[TABLE]

and probabilistic fact

[TABLE]

Depending on $\mathsf{disturbanceUp}$, $\mathsf{actionUp}$ may succeed or not in leading to $\mathsf{up}$.

Sato’s distribution semantics at first focused on definite programs, and was announced “roughly, as distributions over least models” (?). Poole and Sato originally emphasized acyclic logic programs (?, ?, ?, ?), even though Sato did handle cyclic ones. Since then, there has been significant work on non-definite and on cyclic probabilistic logic programs under variants of the distribution semantics (?, ?, ?, ?).

In this paper we examine the meaning and the computational complexity of probabilistic logic programs that extend Sato’s distribution semantics. We look at standard function-free normal programs containing negation as failure and probabilistic facts. The goal is to compute an inference; that is, to compute the probability $\mathbb{P}\!\left(\mathbf{Q}|\mathbf{E}\right)$, where both $\mathbf{Q}$ and $\mathbf{E}$ are sets of facts. The pair $(\mathbf{Q},\mathbf{E})$ is referred to as the query. We distinguish between the complexity of inference when a probabilistic program and a query are given (the inferential complexity), and the complexity of inference when the probabilistic program is fixed and the query is given (the query complexity). Query complexity is similar to data complexity as used in database theory, as we discuss later.

We first examine acyclic programs; for those programs all existing semantics coincide. Given the well-known relationship between acyclic probabilistic logic programs and Bayesian networks, it is not surprising that inference for propositional acyclic programs is $\mathsf{PP}$-complete. However, it is surprising that, as we show, inference with bounded arity acyclic programs without negation is $\mathsf{PP}^{\mathsf{NP}}$-equivalent, thus going up the counting hierarchy. And we show that acyclic programs without a bound on predicate arity take us to $\mathsf{PEXP}$-completeness.

Many useful logic programs are cyclic; indeed, the use of recursion is at the heart of logic programs and its various semantics (?, ?). Many applications, such as non-recursive structural equation models (?, ?) and models with “feedback” (?, ?), defy the acyclic character of Bayesian networks.

We study cyclic normal logic programs in a few steps. First we look at the inferential and query complexity of locally stratified programs. For these programs, again we see that most existing semantics coincide; in particular semantics based on stable and well-founded models are identical. To summarize, we show that the complexity of stratified programs is the same as the complexity of acyclic programs.

We then move to general, possibly cyclic, programs. There are various semantics for such programs, and relatively little discussion about them in the literature. For instance, take a program consisting of two rules,

[TABLE]

and a fact associated with a probabilistic assessment:

[TABLE]

With probability $0.3$, we have that $\mathsf{insomnia}$ is $\mathsf{true}$, and then $\mathsf{sleep}$ is $\mathsf{false}$ and $\mathsf{work}$ is $\mathsf{true}$. This is simple enough. But with probability $0.7$, we have that $\mathsf{insomnia}$ is $\mathsf{false}$, and then the remaining two rules create a cycle: $\mathsf{sleep}$ depends on $\mathsf{work}$ and vice-versa. The question is how to define a semantics when a cycle appears.

We focus on two semantics for such programs, even though we mention a few others. First, we look at a semantics for probabilistic logic programs that can be extracted from the work of Lukasiewicz on probabilistic description logics (?, ?). His proposal is that a probabilistic logic program defines a set of probability measures, induced by the various stable models of the underlying normal logic program. The second semantics we examine is based on the well-founded semantics of normal logic programs: in this case there is always a single distribution induced by a probabilistic logic program (?).

We first study Lukasiewicz’s semantics, referred to as the “credal semantics”. We show that credal semantics produces sets of probability models that dominate infinitely monotone Choquet capacities; the latter objects are relatively simple extensions of probability distributions and have been often used in the literature, from random set theory to Dempster-Shafer theory. We then derive results concerning inferential and query complexity. We show that the complexity of general probabilistic logic programs goes up the counting hierarchy, up to $\mathsf{PP}^{\mathsf{NP}^{\mathsf{NP}}}$ levels; overall the complexity of the well-founded semantics is in lower classes than the complexity of the stable model semantics.

The paper begins in Section 2 with a review of logic programming and complexity theory. Section 3 presents basic notions concerning probabilistic logic programs and their semantics. In Section 4 we contribute with a comparison between the credal and the well-founded semantics. Our main results appear in Sections 5, 6, 7 and 8. In Section 5 we show that the credal semantics of a probabilistic logic program is a set of probability measures induced by a 2-monotone Choquet capacities. Sections 6, 7 and 8 analyze the complexity of inferences under the credal and the well-founded semantics. The paper concludes, in Section 9, with a summary of our contributions and a discussion of future work.

2 Background

We briefly collect here some well known terminology and notation regarding logic programming and complexity theory. Before we plunge into those topics, we briefly fix notation on Bayesian networks as we will need them later. A Bayesian network is a pair consisting of a directed acyclic graph $\mathbb{G}$ whose nodes are random variables, and a joint probability distribution $\mathbb{P}$ over all variables in the graph, such that $\mathbb{G}$ and $\mathbb{P}$ satisfy the “Markov condition” (that is, a random variable is independent of its parents given its nondescendants) (?, ?, ?). If all random variables are discrete, then one can specify “local” conditional probabilities $\mathbb{P}\!\left(X_{i}=x_{i}|\mathrm{pa}(X_{i})=\pi_{i}\right)$, and the joint probability distribution is necessarily the product of these local probabilities:

[TABLE]

where $\pi_{i}$ is the projection of $\{x_{1},\dots,x_{n}\}$ on the set of random variables $\mathrm{pa}(X_{i})$; whenever $X_{i}$ has no parents, $\mathbb{P}\!\left(X_{i}=x_{i}|\mathrm{pa}(X_{i})=\pi_{i}\right)$ stands for $\mathbb{P}\!\left(X_{i}=x_{i}\right)$.

2.1 Normal logic programs: syntax and semantics

Take a vocabulary consisting of set of logical variable symbols $X,Y,\dots$, a set of predicate symbols $\mathsf{r},\mathsf{s},\dots$, and a set of constants $a,b,\dots$. A term is a constant or a logical variable; an atom is written as $\mathsf{r}(t_{1},\dots,t_{n})$, where $\mathsf{r}$ is a predicate of arity $n$ and each $t_{i}$ is a term. A zero-arity atom is written simply as $\mathsf{r}$. An atom is ground if it does not contain logical variables.

A normal logic program consists of rules written as (?)

[TABLE]

where the $A_{i}$ are atoms and $\mathbf{not}$ is interpreted according to some selected semantics, as discussed later. The head of this rule is $A_{0}$; the remainder of the rule is its body. A rule without a body, written simply as $A_{0}\mbox{\large.}$, is a fact. A subgoal in the body is either an atom $A$ (a positive subgoal) or $\mathbf{not}\;A$ (a negative subgoal). A program without negation is definite, and a program without variables is propositional.

Example 1**.**

Here is a program describing the relation between smoking, stress, and social influence (?):

[TABLE]

This program is definite, but not propositional. $\hfill\Box$

The Herbrand base of a program is the set of all ground atoms built from constants and predicates in the program. We do not consider functions in this paper, to stay with finite Herbrand bases.

A substitution is a (partial) function that maps logical variables into terms. A grounding is a substitution mapping into constants. The grounding of a rule is a ground rule obtained by applying the same grounding to each atom. The grounding of a program is the propositional program obtained by applying every possible grounding all rules, using only the constants in the program (i.e., using only ground atoms in the Herbrand base). An atom $A$ unifies with an atom $B$ if there is a substitution that makes both (syntactically) equal.

A literal $L$ is either an atom $A$ or a negated atom $\neg A$. A set of literals is inconsistent if $A$ and $\neg A$ belong to it. Given a normal logic program $\mathbf{P}$, a partial interpretation is a consistent set of literals whose atoms belong to the Herbrand base of $\mathbf{P}$. An interpretation is a consistent set of literals such that every atom in the Herbrand base appears in a literal. An atom is $\mathsf{true}$ (resp., $\mathsf{false}$) in a (partial) interpretation if it appears in a non-negated (resp., negated) literal. A subgoal is $\mathsf{true}$ in an interpretation if it is an atom $A$ and $A$ belongs to the interpretation, or the subgoal is $\mathbf{not}\;A$ and $\neg A$ belongs to the interpretation. A grounded rule is satisfied in a partial interpretation if its head is $\mathsf{true}$ in the interpretation, or any of its subgoals is $\mathsf{false}$ in the interpretation. A model of $\mathbf{P}$ is an interpretation such that every grounding of a rule in $\mathbf{P}$ is satisfied. A minimal model of $\mathbf{P}$ is a model with minimum number of non-negated literals.

The dependency graph of a program is a directed graph where each predicate is a node, and where there is an edge from a node $B$ to a node $A$ if there is a rule where $A$ appears in the head and $B$ appears in the body; if $B$ appears right after $\mathbf{not}$, the edge is negative; otherwise, it is positive. The grounded dependency graph is the dependency graph of the propositional program obtained by grounding. For instance, the grounded dependency graph of the program in Example 1 is depicted in Figure 1.

A program is acyclic when its grounded dependency graph is acyclic.

Concerning the semantics of normal logic programs, there are, broadly speaking, two strategies to follow. One strategy is to translate programs into a first-order theory that is called a completion of the program. Then the semantics of the program is the set of first-order models of its completion. The most famous completion is Clark’s (?), roughly defined as follows. First, rewrite each body by replacing commas by $\wedge$ and $\mathbf{not}$ by $\neg$. Second, remove constants from heads: to do so, consider a rule $A_{0}(a)\colonminus B_{i}\mbox{\large.}$, where $a$ is a constant and $B_{i}$ is the body; then this rule is replaced by $A_{0}(X)\colonminus(X=a)\wedge B_{i}\mbox{\large.}$. Then, for each set of rules that share the same head $A_{0}$, write $A_{0}\Leftrightarrow B_{1}\vee B_{2}\vee\dots\vee B_{k}$, where each $B_{i}$ is the body of one of the rules.

The second strategy that is often used to define the semantics of normal logic programs is to select some models of the program to be its semantics. There are many proposals in the literature as to which models should be selected; however, currently there are two selections that have received most attention: the stable model (?) and the well-founded (?) semantics. We now describe these semantics; alas, their definitions are not simple.

Consider first the stable model semantics. Suppose we have a normal logic program $\mathbf{P}$ and an interpretation $\mathcal{I}$. Define the reduct $\mathbf{P}^{\mathcal{I}}$ to be a definite program that contains rule $A_{0}\colonminus A_{1},\dots,A_{m}\mbox{\large.}$ iff one of the grounded rules from $\mathbf{P}$ is $A_{0}\colonminus A_{1},\dots,A_{m},\mathbf{not}\;A_{m+1},\dots,\mathbf{not}\;A_{n}\mbox{\large.}$ where each $A_{m+1},\dots,A_{n}$ is $\mathsf{false}$ in $\mathcal{I}$. That is, the reduct is obtained by (i) grounding $\mathbf{P}$, (ii) removing all rules that contain a subgoal $\mathbf{not}\;A$ in their body such that $A$ is an atom that is $\mathsf{true}$ in $\mathcal{I}$, (iii) removing all remaining literals of the form $\mathbf{not}\;A$ from the remaining rules. An interpretation $\mathcal{I}$ is a stable model if $\mathcal{I}$ is a minimal model of $\mathbf{P}^{\mathcal{I}}$. Note that a normal program may fail to have a stable model, or may have several stable models.

There are two types of logical reasoning under the stable mode semantics (?). Brave reasoning asks whether there is a stable model containing a specific atom (and possibly returns it if it exists). Cautious reasoning asks whether a specific atom appears in all stable models (and possibly lists all such models).

Consider now the well-founded semantics. Given a subset $\mathcal{U}$ of the Herbrand base of a program, and a partial interpretation $\mathcal{I}$, say that an atom $A$ is unfounded with respect to $\mathcal{U}$ and $\mathcal{I}$ iff for each grounded rule whose head is $A$, we have that (i) some subgoal $A_{i}$ or $\mathbf{not}\;A_{i}$ is $\mathsf{false}$ in $\mathcal{I}$, or (ii) some subgoal that is an atom $A_{i}$ is in $\mathcal{U}$. Now say that a subset $\mathcal{U}$ of the Herbrand base is an unfounded set with respect to interpretation $\mathcal{I}$ if each atom in $\mathcal{U}$ is unfounded with respect to $\mathcal{U}$ and $\mathcal{I}$. This is a complex definition: roughly, it means that, for each possible rule that we might apply to obtain $A$, either the rule cannot be used (given $\mathcal{I}$), or there is an atom in $\mathcal{U}$ that must be first shown to be $\mathsf{true}$. Now, given normal logic program $\mathbf{P}$, define $\mathbb{T}_{\mathbf{P}}(\mathcal{I})$ to be a transformation that takes interpretation $\mathcal{I}$ and returns another interpretation: $A\in\mathbb{T}_{\mathbf{P}}(\mathcal{I})$ iff there is some grounded rule with head $A$ such that every subgoal in the body is $\mathsf{true}$ in $\mathcal{I}$. Also define $\mathbb{U}_{\mathbf{P}}(\mathcal{I})$ to be the greatest unfounded set with respect to $\mathcal{I}$ (there is always such a greatest set). Define $\mathbb{W}_{\mathbf{P}}(\mathcal{I})=\mathbb{T}_{\mathbf{P}}(\mathcal{I})\cup\neg\mathbb{U}_{\mathbf{P}}(\mathcal{I})$, where the notation $\neg\mathbb{U}_{\mathbf{P}}(\mathcal{I})$ means that we take each literal in $\mathbb{U}_{\mathbf{P}}(\mathcal{I})$ and negate it (that is, $A$ becomes $\neg A$; $\neg A$ becomes $A$). Intuitively, $\mathbb{T}_{\mathbf{P}}$ is what we can “easily prove to be positive” and $\mathbb{U}_{\mathbf{P}}$ is what we can “easily prove to be negative”.

Finally: the well-founded semantics of $\mathbf{P}$ is the least fixed point of $\mathbb{W}_{\mathbf{P}}(\mathcal{I})$; this fixed point always exists. That is, apply $\mathcal{I}_{i+1}=\mathbb{W}_{\mathbf{P}}(\mathcal{I}_{i})$, starting from $\mathcal{I}_{0}=\emptyset$, until it stabilizes; the resulting interpretation is the well-founded model. The iteration stops in finitely many steps given that we have finite Herbrand bases.

The well-founded semantics determines the truth assignment for a subset of the atoms in the Herbrand base; for the remaining atoms, their “truth values are not determined by the program” (?, Section 1.3). A very common interpretation of this situation is that the well-founded semantics uses three-valued logic with values $\mathsf{true}$, $\mathsf{false}$, and $\mathsf{undefined}$. It so happens that any well-founded model is a subset of every stable model of a normal logic program (?, Corollary 5.7); hence, if a program has a well-founded model that is an interpretation for all atoms, then this well-founded model is the unique stable model (the converse is not true).

There are other ways to define the well-founded semantics that are explicitly constructive (?, ?, ?). One is this, where the connection with the stable model semantics is emphasized (?): write $\mathbb{LFT}_{\mathbf{P}}(\mathcal{I})$ to mean the least fixpoint of $\mathbb{T}_{\mathbf{P}^{\mathcal{I}}}$; then the well-founded semantics of $\mathbf{P}$ consists of those atoms $A$ that are in the least fixpoint of $\mathbb{LFT}_{\mathbf{P}}(\mathbb{LFT}_{\mathbf{P}}(\cdot))$ plus the literals $\neg A$ for those atoms $A$ that are not in the greatest fixpoint of $\mathbb{LFT}_{\mathbf{P}}(\mathbb{LFT}_{\mathbf{P}}(\cdot))$. Note that $\mathbb{LFT}_{\mathbf{P}}(\mathbb{LFT}_{\mathbf{P}}(\cdot))$ is a monotone operator.

It is instructive to look at some examples.

Example 2**.**

First, take a program $\mathbf{P}$ with two rules: $\mathsf{p}\colonminus\mathbf{not}\;\mathsf{q},\mathbf{not}\;\mathsf{r}\mbox{\large.}$ and $\mathsf{q}\colonminus\mathbf{not}\;\mathsf{p}\mbox{\large.}$ (identical to rules in Expression (1)). This program has two stable models: both assign $\mathsf{false}$ to $\mathsf{r}$; one assigns $\mathsf{true}$ to $\mathsf{p}$ and $\mathsf{false}$ to $\mathsf{q}$, while the other assigns $\mathsf{true}$ to $\mathsf{q}$ and $\mathsf{false}$ to $\mathsf{p}$ (note $\mathbf{P}^{\{\mathsf{p},\neg\mathsf{q},\neg\mathsf{r}\}}=\{\mathsf{p}.\}$ and $\mathbf{P}^{\{\neg\mathsf{p},\mathsf{q},\neg\mathsf{r}\}}=\{\mathsf{q}.\}$). The well-founded semantics assigns $\mathsf{false}$ to $\mathsf{r}$ and leaves $\mathsf{p}$ and $\mathsf{q}$ as $\mathsf{undefined}$. $\Box$

Example 3**.**

Consider a game where a player wins if there is another player with no more moves (?, ?), as expressed by the cyclic rule:

[TABLE]

Suppose the available moves are given as the following facts:

[TABLE]

There are two stable models: both assign $\mathsf{true}$ to $\mathsf{wins}(c)$ and $\mathsf{false}$ to $\mathsf{wins}(d)$; one assigns $\mathsf{true}$ to $\mathsf{wins}(a)$ and $\mathsf{false}$ to $\mathsf{wins}(b)$, while the other assigns $\mathsf{true}$ to $\mathsf{wins}(b)$ and $\mathsf{false}$ to $\mathsf{wins}(a)$. The well-founded semantics leads to partial interpretation $\{\mathsf{wins}(c),\neg\mathsf{wins}(d)\}$, leaving $\mathsf{wins}(a)$ and $\mathsf{wins}(b)$ as $\mathsf{undefined}$. If $\mathsf{move}(a,b)$ is not given as a fact, it is assigned $\mathsf{false}$, and the well-founded semantics leads to $\{\neg\mathsf{wins}(a),\mathsf{wins}(b),\mathsf{wins}(c),\neg\mathsf{wins}(d)\}$. $\Box$

Example 4**.**

The Barber Paradox: If the barber shaves all, and only, those villagers who do not shave themselves, does the barber shave himself? Consider:

[TABLE]

There is no stable model for this normal logic program: the facts and the rule lead to the pattern $\mathsf{shaves}(b,b)\colonminus\mathbf{not}\;\mathsf{shaves}(b,b)\mbox{\large.}$, thus eliminating any possible stable model. The well-founded semantics assigns $\mathsf{false}$ to $\mathsf{barber}(a)$, to $\mathsf{shaves}(a,a)$ and to $\mathsf{shaves}(a,b)$. Also, $\mathsf{shaves}(b,a)$ is assigned $\mathsf{true}$, and $\mathsf{shaves}(b,b)$ is left $\mathsf{undefined}$. That is, even though the semantics leaves the status of the barber as $\mathsf{undefined}$, it does produce meaningful answers for other villagers. $\Box$

2.2 Complexity theory: the counting hierarchy

We adopt basic terminology and notation from computational complexity (?). A language is a set of strings. A language defines a decision problem; that is, the problem of deciding whether an input string is in the language. A complexity class is a set of languages; we use well-known complexity classes such as $\mathsf{P}$, $\mathsf{NP}$, $\mathsf{EXP}$, $\mathsf{NEXP}$. The complexity class $\mathsf{PP}$ consists of those languages $\mathcal{L}$ that satisfy the following property: there is a polynomial time nondeterministic Turing machine $M$ such that $\ell\in\mathcal{L}$ iff more than half of the computations of $M$ on input $\ell$ end up accepting. Analogously, we have $\mathsf{PEXP}$, consisting of those languages $\mathcal{L}$ with the following property: there is an exponential time nondeterministic Turing machine $M$ such that $\ell\in\mathcal{L}$ iff half of the computations of $M$ on input $\ell$ end up accepting (?).

An oracle Turing machine $M^{\mathcal{L}}$, where $\mathcal{L}$ is a language, is a Turing machine that can write a string $\ell$ to an “oracle” tape and obtain from the oracle, in unit time, the decision as to whether $\ell\in\mathcal{L}$ or not. Similarly, for a function $f$, an oracle Turing machine $M^{f}$ can be defined. If a class of languages/functions $\mathsf{A}$ is defined by a set of Turing machines $\mathcal{M}$ (that is, the languages/functions are decided/computed by these machines), then $\mathsf{A}^{\mathcal{L}}$ is the set of languages/functions that are decided/computed by $\{M^{\mathcal{L}}:M\in\mathcal{M}\}$. Similarly, for any class $\mathsf{A}$ we have $\mathsf{A}^{f}$. If $\mathsf{A}$ and $\mathsf{B}$ are classes of languages/functions, $\mathsf{A}^{\mathsf{B}}=\cup_{x\in\mathsf{B}}\mathsf{A}^{x}$. The polynomial hierarchy consists of classes $\Pi^{\mathsf{P}}_{i}=\mathsf{co}\Sigma^{\mathsf{P}}_{i}$ and $\Sigma^{\mathsf{P}}_{i}=\mathsf{NP}^{\Sigma^{\mathsf{P}}_{i-1}}$, with $\Sigma^{\mathsf{P}}_{0}=\mathsf{P}$. Later we also use classes $\Delta^{\mathsf{P}}_{i}=\mathsf{P}^{\Sigma^{\mathsf{P}}_{i-1}}$ and $\mathsf{PH}=\cup_{i}\Pi^{\mathsf{P}}_{i}=\cup_{i}\Sigma^{\mathsf{P}}_{i}$.

Wagner’s polynomial counting hierarchy is the smallest set of classes containing $\mathsf{P}$ and, recursively, for any class $\mathsf{C}$ in the polynomial counting hierarchy, the classes $\mathsf{PP}^{\mathsf{C}}$, $\mathsf{NP}^{\mathsf{C}}$, and $\mathsf{coNP}^{\mathsf{C}}$ (?, Theorem 4) (?, Theorem 4.1). The polynomial hierarchy is included in Wagner’s counting polynomial hierarchy.

A many-one reduction from $\mathcal{L}$ to $\mathcal{L}^{\prime}$ is a polynomial time algorithm that takes the input to decision problem $\mathcal{L}$ and transforms it into the input to decision problem $\mathcal{L}^{\prime}$ such that $\mathcal{L}^{\prime}$ has the same output as $\mathcal{L}$. For a complexity class $\mathsf{C}$, a decision problem $\mathcal{L}$ is $\mathsf{C}$-hard with respect to many-one reductions if each decision problem in $\mathsf{C}$ can be reduced to $\mathcal{L}$ with many-one reductions. A decision problem is then $\mathsf{C}$-complete with respect to many-one reductions if it is in $\mathsf{C}$ and it is $\mathsf{C}$-hard with respect to many-one reductions.

In proofs we will often use propositional formulas; such a formula is in Conjunctive Normal Form (CNF) when it is a conjunction of clauses (where a clause is a disjunction of literals). A $k$CNF is a CNF in which each clause has $k$ literals. We use the following $\mathsf{PP}^{\Sigma_{k}^{\mathsf{P}}}$-complete problem (?, Theorem 7), that we refer to as $\#_{k}\mathsf{3CNF}(>)$:

Input:

A pair $(\phi,M)$, where $\phi(\mathbf{X}_{0},\mathbf{X}_{1},\dots,\mathbf{X}_{k})$ is a propositional formula in 3CNF and each $\mathbf{X}_{i}$ is a tuple of logical variables, and $M$ is an integer.

Output:

Whether or not the number of truth assignments for $\mathbf{X}_{0}$ in the formula

[TABLE]

is strictly larger than $M$, where the quantifiers alternate and each logical variable not in $\mathbf{X}_{0}$ is bound to a quantifier.

Another $\mathsf{PP}^{\Sigma_{k}^{\mathsf{P}}}$-complete problem, referred to as $\#_{k}\mathsf{DNF}(>)$ is:

Input:

A pair $(\phi,M)$, where $\phi(\mathbf{X}_{0},\mathbf{X}_{1},\dots,\mathbf{X}_{k})$ is a propositional formula in DNF and each $\mathbf{X}_{i}$ is a tuple of logical variables, and $M$ is an integer.

Output:

Whether or not the number of truth assignments for $\mathbf{X}_{0}$ in the formula

[TABLE]

is strictly larger than $M$, where the quantifiers alternate and each logical variable not in $\mathbf{X}_{0}$ is bound to a quantifier.

A detail is that Wagner defines a $\mathsf{PP}^{\Sigma_{k}^{\mathsf{P}}}$-complete problem using “$\geq k$” instead of “$>M$”, but the former is equivalent to “$>M-1$”, so both inequalities can be used.

3 Probabilistic normal logic programs

In this paper we focus on a particularly simple combination of logic programming and probabilities (?, ?). A probabilistic logic program, abbreviated plp, is a pair $\left<\mathbf{P},\mathbf{PF}\right>$ consisting of a normal logic program $\mathbf{P}$ and a set of probabilistic facts $\mathbf{PF}$. A probabilistic fact is a pair consisting of an atom $A$ and a probability value $\alpha$; we use the notation $\alpha::A\mbox{\large.}$ borrowed from the ProbLog package111At https://dtai.cs.kuleuven.be/problog/index.html. (?).

We assume that every probability value is a rational number.

Example 5**.**

Here is a syntactically correct ProbLog program:

[TABLE]

There are four rules, two facts, and five probabilistic facts. $\Box$

A probabilistic fact may contain logical variables; for instance, we may write $\alpha::\mathsf{r}(X_{1},\dots,X_{n})\mbox{\large.}$. We interpret such a parameterized probabilistic fact as the set of all grounded probabilistic facts obtained by substituting variables with constants in the Herbrand base.222ProbLog additionally has “probabilistic rules” but those are simply syntactic sugar that we do not need here.

Given a plp $\left<\mathbf{P},\mathbf{PF}\right>$ where $\mathbf{P}$ is acyclic, we say the plp is acyclic. Likewise, if $\mathbf{P}$ is definite, stratified, cyclic, etc, we use the same adjective for the whole plp.

3.1 The semantics of probabilistic facts

The interpretation of probabilistic facts requires some pause. Suppose we have a plp $\left<\mathbf{P},\mathbf{PF}\right>$ with $n$ probabilistic facts (which may be groundings of probabilistic facts containing logical variables). From $\left<\mathbf{P},\mathbf{PF}\right>$ we can generate $2^{n}$ normal logic programs: for each probabilistic fact $\alpha::A\mbox{\large.}$, we can either choose to keep fact $A\mbox{\large.}$, or choose to erase fact $A\mbox{\large.}$ altogether. These choices are assumed independent: this is Sato’s independence assumption.

For instance, consider the plp:

[TABLE]

We have four ways to write a normal logic program out of this plp; that is, $\mathsf{r}$ can be kept or removed, and likewise for $\mathsf{s}$. All these normal logic programs are obtained with the same probability $0.25$, and in one of them $\mathsf{v}$ is $\mathsf{true}$; consequently, the probability $\mathbb{P}\!\left(\mathsf{v}=\mathsf{true}\right)=0.25$.

A total choice $\theta$ for the plp is a subset of the set of grounded probabilistic facts. We interpret $\theta$ as a set of ground facts that are probabilistic selected to be included in $\mathbf{P}$; all other ground facts obtained from probabilistic facts are to be discarded. The probability of a total choice is easily computed: it is a product over the grounded probabilistic facts, where probabilistic fact $\alpha::A\mbox{\large.}$ contributes with factor $\alpha$ if $A\mbox{\large.}$ is kept, or factor $(1-\alpha)$ if $A\mbox{\large.}$ is removed. Now for each total choice $\theta$ we obtain a normal logic program, that we denote by $\mathbf{P}\cup\mathbf{PF}^{\downarrow\theta}$.

For instance, the plp in Expression (4) has two probabilistic facts, leading to four total choices, each with probability $0.25$. Now consider a more complicated plp:

[TABLE]

There are five ground probabilistic facts (after grounding $\mathsf{s}(X)$ appropriately); hence there are $32$ total choices. Suppose we choose to keep the fact in the first probabilistic fact, and discard all the others (with probability $0.5\times 0.4\times 0.8\times 0.7\times 0.7$); then we obtain

[TABLE]

a program with a single stable model where $\mathsf{r}$ is the only $\mathsf{true}$ atom. By going through all possible total choices, we have that $\mathbb{P}\!\left(\mathsf{r}=\mathsf{true}\right)=0.8$ (as $\mathsf{r}\mbox{\large.}$ is kept in the program by a first choice with probability $0.5$ or by a second choice with probability $0.6$, hence $0.5+0.6-0.5\times 0.6=0.8$). Similarly, $\mathbb{P}\!\left(\mathsf{s}(a)=\mathsf{true}\right)=0.2+0.3-0.2\times 0.3=0.44$; note however that $\mathbb{P}\!\left(\mathsf{s}(b)=\mathsf{true}\right)=0.3$. And finally, $\mathbb{P}\!\left(\mathsf{v}=\mathsf{true}\right)=0.8\times 0.44\times 0.3=0.1056$.

Sato assumes that no probabilistic fact unifies with the head of a non-fact rule (that is, a rule with a nonempty body); this is called the disjointness condition (?). From a modeling perspective this is a convenient assumption even though we do not need it in our complexity results. In fact from a modeling perspective an even stronger disjointness condition makes sense: no probabilistic fact should unify with the head of any rule (with a body or not), nor with any other probabilistic fact. Under this assumption, the probabilistic fact $\alpha::A\mbox{\large.}$ can be directly interpreted as a probabilistic assessment $\mathbb{P}\!\left(A=\mathsf{true}\right)=\alpha$. Again, we do not need such an assumption for our results, but our examples will always satisfy it, and it makes sense to assume that it will always be adopted in practice.

3.2 The semantics of definite/acyclic/stratified probabilistic logic programs

We can now discuss the semantics of a plp $\left<\mathbf{P},\mathbf{PF}\right>$. First, take the grounding of this plp. Now for each total choice $\theta$ we obtain the normal logic program $\mathbf{P}\cup\mathbf{PF}^{\downarrow\theta}$. Hence the distribution over total choices induces a distribution over normal logic programs.

A common assumption is that, for each total choice $\theta$, the resulting normal logic program $\mathbf{P}\cup\mathbf{PF}^{\downarrow\theta}$ yields a single model (?). For instance, if $\mathbf{P}$ is definite, then $\mathbf{P}\cup\mathbf{PF}^{\downarrow\theta}$ is definite for any $\theta$, and $\mathbf{P}\cup\mathbf{PF}^{\downarrow\theta}$ has a unique stable model that is also its unique well-founded model. Thus the unique distribution over total choices becomes a unique distribution over stable/well-founded models. This distribution is exactly Sato’s distribution semantics (?). Similarly, suppose that $\mathbf{P}$ is acyclic; then $\mathbf{P}\cup\mathbf{PF}^{\downarrow\theta}$ is acyclic for any $\theta$, and $\mathbf{P}\cup\mathbf{PF}^{\downarrow\theta}$ has a unique stable model that is also its unique well-founded model (?).

Poole’s and Sato’s original work focused respectively on acyclic and definite programs; in both cases the semantics of resulting normal logic programs is uncontroversial. The same can be said of the larger class of stratified programs; a normal logic program is stratified when cycles in the grounded dependency graph contain no negative edge (this is often referred to as locally stratified in the literature) (?). Both the stable and the well-founded semantics are identical for stratified programs, and both generate a unique interpretation for all atoms. As a consequence, a plp $\left<\mathbf{P},\mathbf{PF}\right>$ has a unique distribution semantics whenever $\mathbf{P}$ is stratified. Note that both acyclic and definite programs are stratified.

Example 6**.**

The plp in Example 5 is acyclic, and thus stratified, but not definite. The grounded dependency graph of this program is depicted in Figure 2. This graph can be interpreted as a Bayesian network, as we discuss later (?). There are $2^{5}$ total choices, and the probability of $\mathsf{calls}(\mathsf{a})$ is $0.58$.

Example 7**.**

Consider a probabilistic version of the “smokers” program in Example 1 (?):

[TABLE]

The grounded dependency graph of this program is identical to the one shown in Figure 1. It is tempting to interpret this graph as a Bayesian network, but of course this is not quite right as the graph is cyclic. Indeed the program is not acyclic, but it is definite and therefore stratified, hence a unique distribution is defined over ground atoms. For instance, we have $\mathbb{P}\!\left(\mathsf{smokes}(a)\right)=0.06$ and $\mathbb{P}\!\left(\mathsf{smokes}(b)\right)=0.2$. The program would still be stratified if the first rule were replaced by

[TABLE]

In this case there would still be a cycle, but the negative edge in the dependency graph would not belong to the cycle. $\Box$

Often a stratified program is used to implement recursion, as illustrated by the next example:

Example 8**.**

Consider the following plp, based on an example in the ProbLog distribution:

[TABLE]

That is, we have a random graph with nodes $\mathsf{1},\dots,\mathsf{6}$, and probabilities attached to edges. The query $\mathbb{P}\!\left(\mathsf{path}(\mathsf{1},\mathsf{6})=\mathsf{true}\right)$ yields the probability that there is a path between nodes $\mathsf{1}$ and $\mathsf{6}$. Using ProbLog one obtains $\mathbb{P}\!\left(\mathsf{path}(\mathsf{1},\mathsf{6})=\mathsf{true}\right)=0.217$. $\Box$

3.3 The semantics of general probabilistic logic programs

If a normal logic program is non-stratified, then its well-founded semantics may be a partial interpretation, and some atoms may be left as $\mathsf{undefined}$; it may have several stable models, or no stable model at all. Thus we must accommodate these cases when we contemplate non-stratified plps.

3.3.1 The credal semantics

A first possible semantics for general probabilistic logic programs can be extracted from work by ? (?, ?) on probabilistic description logic programs. To describe that proposal, a few definitions are needed. A plp $\left<\mathbf{P},\mathbf{PF}\right>$ is consistent if there is at least one stable model for each total choice of $\mathbf{PF}$. A probability model for a consistent plp $\left<\mathbf{P},\mathbf{PF}\right>$ is a probability measure $\mathbb{P}$ over interpretations of $\mathbf{P}$, such that:

(i) every interpretation $\mathcal{I}$ with $\mathbb{P}\!\left(\mathcal{I}\right)>0$ is a stable model of $\mathbf{P}\cup\mathbf{PF}^{\downarrow\theta}$ for the total choice $\theta$ that agrees with $\mathcal{I}$ on the probabilistic facts (that is, $\theta$ induces the same truth values as $\mathcal{I}$ for the grounded probabilistic facts); and

(ii) the probability of each total choice $\theta$ is the product of the probabilities for all individual choices in $\theta$.

The set of all probability models for a plp is the semantics of the program. Later examples will clarify this construction.

Lukasiewicz calls his proposed semantics the answer set semantics for probabilistic description logic programs; however, note that this name is both too restrictive (the semantics can be used for programs with functions, for instance) and a bit opaque (it does not emphasize the fact that it deals with uncertainty). We prefer the term credal semantics, which we adopt from now on. The reason for this latter name is that a set of probability measures is often called a credal set (?).

Now given a consistent plp, we may be interested in the smallest possible value of $\mathbb{P}\!\left(\mathbf{Q}\right)$ for a set $\mathbf{Q}$ of truth assignments, with respect to the set $\mathbb{K}$ of all probability models of the plp. This is conveyed by the lower probability of $\mathbf{Q}$, $\underline{\mathbb{P}}\!\left(\mathbf{Q}\right)=\inf_{\mathbb{P}\in\mathbb{K}}\mathbb{P}\!\left(\mathbf{Q}\right)$. Similarly, we have the upper probability of $\mathbf{Q}$, $\overline{\mathbb{P}}\!\left(\mathbf{Q}\right)=\sup_{\mathbb{P}\in\mathbb{K}}\mathbb{P}\!\left(\mathbf{Q}\right)$. Suppose that we also have a set of $\mathbf{E}$ of truth assignments for ground atoms; then we may be interested in the conditional lower and upper probabilities, respectively $\underline{\mathbb{P}}\!\left(\mathbf{Q}|\mathbf{E}\right)=\inf_{\mathbb{P}\in\mathbb{K}:\mathbb{P}\!\left(\mathbf{E}\right)>0}\mathbb{P}\!\left(\mathbf{Q}|\mathbf{E}\right)$ and $\overline{\mathbb{P}}\!\left(\mathbf{Q}|\mathbf{E}\right)=\sup_{\mathbb{P}\in\mathbb{K}:\mathbb{P}\!\left(\mathbf{E}\right)>0}\mathbb{P}\!\left(\mathbf{Q}|\mathbf{E}\right)$. We leave conditional lower/upper probabilities undefined when $\overline{\mathbb{P}}\!\left(\mathbf{E}\right)=0$ (that is, when $\mathbb{P}\!\left(\mathbf{E}\right)=0$ for every probability model). This is not the only possible convention: ? (?, Section 3) adopts $\underline{\mathbb{P}}\!\left(\mathbf{Q}|\mathbf{E}\right)=1$ and $\overline{\mathbb{P}}\!\left(\mathbf{Q}|\mathbf{E}\right)=0$ in this case, while Walley’s style of conditioning prescribes $\underline{\mathbb{P}}\!\left(\mathbf{Q}|\mathbf{E}\right)=0$ and $\overline{\mathbb{P}}\!\left(\mathbf{Q}|\mathbf{E}\right)=1$ whenever $\overline{\mathbb{P}}\!\left(\mathbf{E}\right)=0$ (?).

3.3.2 The well-founded semantics

The approach by ? (?) is to allow probabilities directly over well-founded models, thus allowing probabilities over atoms that are $\mathsf{undefined}$. That is, given a plp $\left<\mathbf{P},\mathbf{PF}\right>$, associate to each total choice $\theta$ the unique well-founded model of $\mathbf{P}\cup\mathbf{PF}^{\downarrow\theta}$ to $\theta$; the unique distribution over total choices induces a unique distribution over well-founded models. Note that probabilities may be assigned to $\mathsf{undefined}$ values in this sort of semantics. As we discuss in Section 4, this is a bold proposal whose interpretation is far from simple.

Regardless of its meaning, the approach deserves attention as it is the only one in the literature that genuinely combines well-founded semantics with probabilities. Accordingly, we refer to it as the well-founded semantics of probabilistic logic programs (the combination of language and semantics is named WF-PRISM by Hadjichristodoulou and Warren).

3.3.3 Other semantics

Sato et al.  propose a semantics where distributions are defined over models produced by Fitting’s three-valued semantics (?). We note that Fitting’s semantics is weaker than the well-founded semantics, and the literature on logic programming has consistently preferred the latter, as we do in this paper.

Another three-valued approach, proposed by ? (?, ?), leaves the probability of any formula as $\mathsf{undefined}$ whenever the formula is $\mathsf{undefined}$ for any total choice (to determine whether a formula is $\mathsf{undefined}$ or not in a particular partial interpretation, three-valued logic is used). Hence, when a formula gets a (non-$\mathsf{undefined}$) numeric probability value, its truth value is the same for all stable models; thus any numeric probability calculations that are produced with this semantics agree with the semantics based on stable models ? (?, Theorem 4.5). That is, Lukasiewicz’ proposal is more akin to the credal semantics than to the well-founded semantics.

A different semantics for non-stratified plps is adopted by the P-log language (?). P-log allows for disjunction in heads and other features, but when restricted to normal logic programs it is syntactically similar to ProbLog. The semantics of a P-log program is given by a single probability distribution over possibly many stable models; whenever necessary default assumptions are called to distribute probability evenly, or to avoid inconsistent realizations (by re-normalization). We leave an analysis of this sort of semantics to the future; here we prefer to focus on semantics that do not make default assumptions concerning probabilities.

It is also important to mention the constraint logic programming language of ? (?), a significant contribution that is also based on credal sets. However, they use a syntax and semantics that is markedly different from Lukasiewicz’s approach, as they allow continuous variables but do not let a program have multiple stable models per total choice. They also present expressions for (conditional) lower and upper probabilities, by direct optimization; in Section 5 we show that such expressions can be derived from properties of infinitely monotone Choquet capacities.

Finally, ? (?) have introduced a semantics that allows for inconsistent plps to have meaning without getting into three-valued logic. They adopt a much more sophisticated family of logic programs (within the Datalog± language), and they provide a thorough analysis of complexity that we discuss later. This is also a proposal that deserves future study.

In this paper we focus on the credal and the well-founded semantics in the remainder of this paper, whenever non-stratified plps are discussed, but certainly there are other avenues to explore.

4 The semantics of the credal and the well-founded semantics

It does not seem that any comparison is available in the literature between the credal and the well-founded semantics for non-stratified plps. Indeed, the credal semantics has not been adopted since its appearance, a turn of events we find unfortunate as it is quite a sensible semantics for general plps. In this section we present some examples that emphasize differences between these semantics, and we examine their interpretation.

Example 9**.**

Consider a probabilistic version of Example 2:

[TABLE]

This is in essence identical to the plp in Expression (1). To interpret the plp, note that with probability $\alpha$ we obtain the normal logic program

[TABLE]

The unique stable/well-founded model of this program assigns $\mathsf{true}$ to $\mathsf{r}$ and $\mathsf{q}$, and $\mathsf{false}$ to $\mathsf{p}$. That is, we have the stable model $s_{1}=\{\neg\mathsf{p},\mathsf{q},\mathsf{r}\}$. On the other hand, with probability $1-\alpha$ we obtain a program with different behavior, namely:

[TABLE]

This program has two stable models: $s_{2}=\{\mathsf{p},\neg\mathsf{q},\neg\mathsf{r}\}$ and $s_{3}=\{\neg\mathsf{p},\mathsf{q},\neg\mathsf{r}\}$. But this program has a single well-founded model, where $\mathsf{r}$ is $\mathsf{false}$ and both $\mathsf{p}$ and $\mathsf{q}$ are $\mathsf{undefined}$.

Consider the credal semantics. There is a probabilty model such that $\mathbb{P}\!\left(s_{2}\right)=1-\alpha$ and $\mathbb{P}\!\left(s_{3}\right)=0$, and another probability model such that $\mathbb{P}\!\left(s_{2}\right)=0$ and $\mathbb{P}\!\left(s_{3}\right)=1-\alpha$. Indeed any probability measure such that $\mathbb{P}\!\left(s_{1}\right)=\alpha$ and $\mathbb{P}\!\left(s_{2}\right)=\gamma(1-\alpha)$, $\mathbb{P}\!\left(s_{3}\right)=(1-\gamma)(1-\alpha)$, for $\gamma\in[0,1]$, is also a probability model for this plp.

The well-founded semantics is instead a single distribution that assigns $\mathbb{P}\!\left(s_{1}\right)=1-\alpha$, and assigns probability mass $\alpha$ to the partial interpretation $\{\neg\mathsf{r}\}$.

Now consider an inference; say for instance one wants $\mathbb{P}\!\left(\mathsf{r}=\mathsf{true}\right)$. Clearly $\mathbb{P}\!\left(\mathsf{r}=\mathsf{true}\right)=1-\alpha$, regardless of the semantics. But consider $\mathsf{p}$. With respect to the credal semantics, the relevant quantities are $\underline{\mathbb{P}}\!\left(\mathsf{p}=\mathsf{true}\right)=0$ and $\overline{\mathbb{P}}\!\left(\mathsf{p}=\mathsf{true}\right)=1-\alpha$. And with respect to the well-founded semantics we have instead $\mathbb{P}\!\left(\mathsf{p}=\mathsf{true}\right)=0$ and $\mathbb{P}\!\left(\mathsf{p}=\mathsf{false}\right)=\alpha$, while $\mathbb{P}\!\left(\mathsf{p}=\mathsf{undefined}\right)=1-\alpha$.

To elaborate on this sort of programming pattern, consider the following non-propositional example, adapted from ? (?):

[TABLE]

When $\mathsf{man}(\mathsf{dilbert})$ is discarded, the resulting normal logic program has a single stable model $s_{1}=\{\neg\mathsf{man}(\mathsf{dilbert}),\neg\mathsf{husband}(\mathsf{dilbert}),\neg\mathsf{single}(\mathsf{dilbert})\}$. When $\mathsf{man}(\mathsf{dilbert})$ is a fact, the resulting program two stable models:

[TABLE]

the well-founded semantics instead leads to $\mathsf{undefined}$ values both for $\mathsf{husband}(\mathsf{dilbert})$ and $\mathsf{single}(\mathsf{dilbert})$.

Note that any probability measure such that $\mathbb{P}\!\left(s_{1}\right)=0.1$, $\mathbb{P}\!\left(s_{2}\right)=0.9\gamma$, and $\mathbb{P}\!\left(s_{3}\right)=0.9(1-\gamma)$, for $\gamma\in[0,1]$, is a probability model. Hence we have $\underline{\mathbb{P}}\!\left(\mathsf{husband}(\mathsf{dilbert})=\mathsf{true}\right)=0$ and $\overline{\mathbb{P}}\!\left(\mathsf{husband}(\mathsf{dilbert})=\mathsf{true}\right)=0.9$ with respect to the credal semantics, while we have $\mathbb{P}\!\left(\mathsf{husband}(\mathsf{dilbert})=\mathsf{true}\right)=0$, $\mathbb{P}\!\left(\mathsf{husband}(\mathsf{dilbert})=\mathsf{false}\right)=0.1$, and finally we have $\mathbb{P}\!\left(\mathsf{husband}(\mathsf{dilbert})=\mathsf{undefined}\right)=0.9$ with respect to the well-founded semantics. $\hfill\Box$

Example 10**.**

Now take a plp adapted from an example by ? (?, Example IV.1), where the same pattern of cyclic negation observed in the previous example seems to appear:

[TABLE]

There are four total choices, each inducing a normal logic program. In one case, namely $\{\neg\mathsf{a},\mathsf{b}\}$, the resulting normal logic program has no stable model. Hence, this plp has no credal semantics. However, it does have a well-founded semantics. Table 1 shows the assignments for $\mathsf{cold}$ and $\mathsf{headache}$ induced by the various total choices; we obtain

[TABLE]

by collecting probabilities from Table 1. $\hfill\Box$

Example 11**.**

Consider a graph coloring problem consisting of the rules:

[TABLE]

and the facts: for $i\in\{\mathsf{1},\dots,\mathsf{5}\}$, $\mathsf{vertex}(i)\mbox{\large.}$, and

[TABLE]

The facts mentioning $\mathsf{vertex}$ and $\mathsf{edge}$ encode the graph in Figure 3 (left); the probabilistic fact is indicated as a dashed edge. A total choice determines a particular graph. For a fixed total choice, the stable models of the program are the 3-colorings of the resulting graph (this is indeed a popular example of answer set programming (?)).

Now, if probabilistic fact $\mathsf{edge}(\mathsf{4},\mathsf{5})$ is $\mathsf{true}$, there is a single stable model; otherwise, there are two stable models. Using the credal semantics we obtain: $\underline{\mathbb{P}}\!\left(\mathsf{color}(\mathsf{1},\mathsf{yellow})=\mathsf{true}\right)=0$ and $\overline{\mathbb{P}}\!\left(\mathsf{color}(\mathsf{1},\mathsf{yellow})=\mathsf{true}\right)=1/2$; also, we have $\underline{\mathbb{P}}\!\left(\mathsf{color}(\mathsf{4},\mathsf{yellow})=\mathsf{true}\right)=1/2$ and $\overline{\mathbb{P}}\!\left(\mathsf{color}(\mathsf{4},\mathsf{yellow})=\mathsf{true}\right)=1$; and $\underline{\mathbb{P}}\!\left(\mathsf{color}(\mathsf{3},\mathsf{red})=\mathsf{true}\right)=\overline{\mathbb{P}}\!\left(\mathsf{color}(\mathsf{3},\mathsf{red})=\mathsf{true}\right)=1$.

On the other hand, the well-founded semantics leaves undefined the colors of vertices $\mathsf{1}$, $\mathsf{3}$, and $\mathsf{4}$, both when $\mathsf{edge}(\mathsf{4},\mathsf{5})$ is $\mathsf{true}$ and when it is $\mathsf{false}$. Thus we have, for $V\in\{\mathsf{1},\mathsf{3},\mathsf{4}\}$ and $C\in\{\mathsf{red},\mathsf{yellow},\mathsf{green}\}$, that $\mathbb{P}\!\left(\mathsf{color}(V,C)=\mathsf{undefined}\right)=1$. $\hfill\Box$

Example 12**.**

Take the normal logic program discussed in Example 3, and consider the following probabilistic version (there is one probabilistic move in the game):

[TABLE]

If $\mathsf{move}(\mathsf{c},\mathsf{d})$ is discarded, there is a single stable model (where $\mathsf{b}$ is the only winning position); otherwise, there are two stable models ($\mathsf{wins}(\mathsf{c})$ is $\mathsf{true}$ and $\mathsf{wins}(\mathsf{d})$ is $\mathsf{false}$ in both of them; $\mathsf{wins}(\mathsf{a})$ is $\mathsf{true}$ in one, while $\mathsf{wins}(\mathsf{b})$ is $\mathsf{true}$ in the other). Thus the credal semantics yields $\underline{\mathbb{P}}\!\left(\mathsf{wins}(\mathsf{b})=\mathsf{true}\right)=0.7$ and $\overline{\mathbb{P}}\!\left(\mathsf{wins}(\mathsf{b})=\mathsf{true}\right)=1.0$; $\underline{\mathbb{P}}\!\left(\mathsf{wins}(\mathsf{c})=\mathsf{true}\right)=0.3$ and $\overline{\mathbb{P}}\!\left(\mathsf{wins}(\mathsf{c})=\mathsf{true}\right)=0.3$.

Now if $\mathsf{move}(\mathsf{c},\mathsf{d})$ is discarded, the well-founded model is the unique stable model where $\mathsf{b}$ is the only winning position. But if $\mathsf{move}(\mathsf{c},\mathsf{d})$ is $\mathsf{true}$, then the well-founded model assigns $\mathsf{true}$ to $\mathsf{wins}(\mathsf{c})$ and $\mathsf{false}$ to $\mathsf{wins}(\mathsf{d})$, leaving both $\mathsf{wins}(\mathsf{a})$ and $\mathsf{wins}(\mathsf{b})$ as $\mathsf{undefined}$. Hence the well-founded semantics yields $\mathbb{P}\!\left(\mathsf{wins}(\mathsf{c})=\mathsf{true}\right)=0.3$ and $\mathbb{P}\!\left(\mathsf{wins}(\mathsf{c})=\mathsf{false}\right)=0.7$, while $\mathbb{P}\!\left(\mathsf{wins}(\mathsf{b})=\mathsf{true}\right)=0.7$ and $\mathbb{P}\!\left(\mathsf{wins}(\mathsf{b})=\mathsf{undefined}\right)=0.3$. $\Box$

Example 13**.**

Return to the Barber Paradox discussed in Example 4, now with a probabilistic twist:

[TABLE]

This program does not have a stable model when $\mathsf{villager}(b)$ is a fact. Thus the plp fails to have a credal semantics.

However, the well-founded semantics is clear even when $\mathsf{villager}(b)$ is $\mathsf{true}$: in this case, $\mathsf{barber}(a)$, $\mathsf{shaves}(a,a)$ and $\mathsf{shaves}(a,b)$ are $\mathsf{false}$, while $\mathsf{shaves}(b,a)$ is $\mathsf{true}$, and $\mathsf{shaves}(b,b)$ is $\mathsf{undefined}$. And the well-founded semantics is also clear when $\mathsf{villager}(b)$ is discarded (that is, when $\mathsf{villager}(b)$ is $\mathsf{false}$): only $\mathsf{shaves}(b,a)$ is $\mathsf{true}$. Hence we obtain $\mathbb{P}\!\left(\mathsf{shaves}(b,a)=\mathsf{true}\right)=1$, while $\mathbb{P}\!\left(\mathsf{shaves}(b,b)=\mathsf{false}\right)=0.5$ and $\mathbb{P}\!\left(\mathsf{shaves}(b,b)=\mathsf{undefined}\right)=0.5$. $\Box$

These examples should suffice to show that there are substantial differences between the credal and the well-founded semantics. What to choose?

We start our analysis with the well-founded semantics. At first it may seem that this semantics is very attractive because if attaches a unique probability distribution to every well-formed plp (even in cases where the credal semantics is not defined). Besides, the well-founded semantics for plps is conceptually simple for anyone who has mastered the well-founded semantics for normal logic programs.

On the other hand, some of the weaknesses of the well-founded semantics already appear in non-probabilistic programs. Certainly the point here is not to emphasize non-probabilistic programs, but consider the difficulty of the well-founded semantics in “reasoning by cases”. For instance, consider the program (?):

[TABLE]

The well-founded semantics leaves every atom $\mathsf{undefined}$. However, it is apparent that $\mathsf{p}$ should be assigned $\mathsf{true}$, for we can find two ways to understand the relation between $\mathsf{a}$ and $\mathsf{b}$, and both ways take $\mathsf{p}$ to $\mathsf{true}$ (these two interpretations are exactly the stable models: one contains $\mathsf{a}$ and $\neg\mathsf{b}$, the other contains $\neg\mathsf{a}$ and $\mathsf{b}$). The reader should note that this is similar to the situation in Example 11: there the well-founded semantics cannot even fix the color of vertex $\mathsf{3}$, even though this vertex must clearly be colored $\mathsf{red}$.

The well-founded semantics of non-probabilistic normal logic programs has also drawn criticism in its reliance on three-valued logic, and the status of the $\mathsf{undefined}$ truth value has received attention not only in philosophical inquiry (?, ?), but in the practical development of databases (?, ?). In short, it is difficult to determine whether $\mathsf{undefined}$ should be taken as simply an expression of subjective ignorance, or the indication that something really is neither $\mathsf{true}$ nor $\mathsf{false}$ (?, Section 1.2.1.2).

In any case, we do not want to repeat the old and unresolved debate on three-valued logic here; we want to focus on the even bigger problems that appear when three-valued logic is mixed with probabilities. The problem is that $\mathsf{undefined}$ values reflect a type of uncertainty, and probability is supposed to deal with uncertainty; by putting those together we may wish to invite collaboration but we may end up with plain confusion. Consider for instance Example 10. What does it mean to say that $\mathbb{P}\!\left(\mathsf{headache}=\mathsf{undefined}\right)=0.165$? Supposedly probability is here to tell us the odds of $\mathsf{true}$ and $\mathsf{false}$; by learning the probability of $\mathsf{undefined}$, the next question should be about the probability of $\mathsf{headache}$ to be $\mathsf{true}$ when one is saying that it is $\mathsf{undefined}$. In fact, one might ask for the value of $\mathbb{P}\!\left(\mathsf{headache}=\mathsf{true}|\mathsf{headache}=\mathsf{undefined}\right)$, not realizing that in the well-founded semantics this value is simply zero. To emphasize the difficulty in interpretation, suppose we add to Example 10 the simple rule

[TABLE]

and one asks for $\mathbb{P}\!\left(\mathsf{c}=\mathsf{false}|\mathsf{cold}=\mathsf{undefined}\right)$. Should this number really be $1$, as obtained through the well-founded semantics, or should we expect this to be a question about $\mathbb{P}\!\left(\mathsf{c}=\mathsf{false}\right)$, given that nothing of substance is observed about $\mathsf{cold}$?

The probabilistic Barber Paradox discussed in Example 13 describes a situation where the well-founded semantics can answer questions for some individuals, even as it fails to find definite answers for other questions. This is rather attractive, but one must ask: What exactly is the meaning of $\mathbb{P}\!\left(\mathsf{shaves}(b,b)=\mathsf{undefined}\right)=0.5$? Note that, for the logical program described in Example 4, it makes sense to return an $\mathsf{undefined}$ value: we are at a logical corner. However, for the probabilistic program it is less sensible to obtain a non-zero probability that some particular fact is $\mathsf{undefined}$.

A difficulty here is that $\mathsf{undefined}$ values appear due to a variety of situations that should apparently be treated in distinct ways: (i) programs may be contradictory (as it happens in Example 10); (ii) programs may fail to have a clear meaning (as in the Barber Paradox); or (iii) programs may simply have several possible meanings (for instance, various stable models as in Example 11). In case (i), it is even surprising that one would try to assign probabilities to contradictory cases. In cases (ii) and (iii), probabilities may be contemplated, but then there is a confusing mix of probabilities and $\mathsf{undefined}$ values. The interpretation of the various possible meanings of $\mathsf{undefined}$, already difficult in three-valued logic, is magnified by the challenges in interpreting probabilities.

Now consider the credal semantics. There are two possible criticisms one may raise against it. First, a program may fail to have a credal semantics: consider the probabilistic Barber Paradox. Second, the credal semantics relies on sets of probability measures (credal sets), not on unique measures. We examine these two points in turn.

The fact that some programs may fail to have a credal semantics is an annoyance in that programs must be checked for consistency. However, as we have noted already, some programs can seem contradictory, and in those cases one could argue that it is appropriate not to have semantics. So, one may be perfectly satisfied with failure in Example 10, for the total choice $\{\neg\mathsf{a},\mathsf{b}\}$ in essence leads to the following clearly unsatisfiable pair of rules:

[TABLE]

What seems to be needed here is a verifier that checks consistency of plps; we look into this later in this paper.

Now consider the fact that the credal semantics relies on credal sets. Anyone expecting any inference to produce a single probability value may be puzzled, but reliance on sets of probabilities does not seem to be a flaw when examined in detail. One argument in favor of sets of probabilities is that they are legitimate representations for incomplete, imprecise or indeterminate beliefs, that can be justified in a variety of ways (?, ?, ?). But even if one is not willing to take credal sets as a final representation of beliefs, the credal semantics is wholly reasonable from a least commitment perspective. That is, the main question should always be: What are the best bounds on probabilities that one can safely assume, taking into account only the given rules, facts, and assessments? From this point of view, Examples 9, 11, and 12 are entirely justified: the options given to the program are not decided by the given information, so one must leave them open. In particular Example 11 seems to be an excellent argument for the credal semantics: basically, the program generates all 3-colorings of a given graph; why should we insist on singling out a distribution over colorings when no preference over them is expressed?

All in all, we find that the credal semantics is conceptually stronger than the well-founded semantics, even though the latter is uniquely defined for every plp. We now examine the structural and computation properties of these two semantics; a final comparison is left to Section 9.

5 The structure of credal semantics

Given the generality of plps, one might think that credal sets generated by the credal semantics could have an arbitrarily complex structure. Surprisingly, the structure of the credal semantics of a plp is a relatively simple object:

Theorem 14**.**

Given a consistent plp, its credal semantics is a set of probability measures that dominate an infinitely monotone Choquet capacity.

Before we present a proof of this theorem, let us pause and define a few terms. An infinitely monotone Choquet capacity is a set function $\underline{\mathbb{P}}$ from an algebra $\mathcal{A}$ on a set $\Omega$ to the real interval $[0,1]$ such that (?, Definition 4.2): $\underline{\mathbb{P}}\!\left(\Omega\right)=1-\underline{\mathbb{P}}\!\left(\emptyset\right)=1$ and, for any $A_{1},\dots,A_{n}$ in the algebra, $\underline{\mathbb{P}}\!\left(\cup_{i}A_{i}\right)\geq\sum_{J\subseteq\{1,\dots,n\}}(-1)^{|J|+1}\underline{\mathbb{P}}\!\left(\cap_{j\in J}A_{j}\right)$. Infinitely monotone Choquet capacities appear in several formalisms; for instance, they are the belief functions of Dempster-Shafer theory (?), summaries of random sets (?), and inner measures (?).

Given an infinitely monotone Choquet capacity $\underline{\mathbb{P}}$, we can construct a set of measures that dominate $\underline{\mathbb{P}}$; this is the set $\{\mathbb{P}:\forall A\in\mathcal{A}:\mathbb{P}\!\left(A\right)\geq\underline{\mathbb{P}}\!\left(A\right)\}$. We abuse language and say that a set consisting of all measures that dominate an infinitely monotone Choquet capacity is an infinitely monotone credal set. If a credal set $\mathbb{K}$ is infinitely monotone, then the lower probability $\underline{\mathbb{P}}$, defined as $\underline{\mathbb{P}}\!\left(A\right)=\inf_{\mathbb{P}\in\mathbb{K}}\mathbb{P}\!\left(A\right)$, is exactly the generating infinitely monotone Choquet capacity. We also have the upper probability $\overline{\mathbb{P}}\!\left(A\right)=\sup_{\mathbb{P}\in\mathbb{K}}\mathbb{P}\!\left(A\right)=1-\underline{\mathbb{P}}\!\left(A^{c}\right)$.

Proof of Theorem 14.

Consider a set $\Theta$ containing as states the posssible total choices of the plp. Over this space we have a product measure that is completely specified by the probabilities attached to probabilistic facts. Now consider a multi-valued mapping $\Gamma$ between $\Theta$ and the space $\Omega$ of all possible models of our probabilistic logic program. For each element $\theta\in\Theta$, define $\Gamma(\theta)$ to be the set of stable models associated with the total choice $\theta$ of the probabilistic facts. Now we use the fact that a probability space and a multi-valued mapping induce an infinite monotone Choquet capacity over the range of the mapping (that is, over $\Omega$) (?). ∎

Infinitely monotone credal sets have several useful properties; for one thing they are closed and convex. Convexity here means that if $\mathbb{P}_{1}$ and $\mathbb{P}_{2}$ are in the credal set, then $\alpha\mathbb{P}_{1}+(1-\alpha)\mathbb{P}_{2}$ is also in the credal set for $\alpha\in[0,1]$. Thus, as illustrated by Example 9.

Corollary 15**.**

Given a consistent plp, its credal semantics is a closed and convex set of probability measures.

There are several additional results concerning the representation of infinitely monotone capacities using their Möbius transforms (?, ?); we refrain from mentioning every possible corollary we might produce by rehashing those results. Instead, we focus on a few important results that can be used to great effect in future applications. First, as we have a finite Herbrand base, we can use the symbols in the proof of Theorem 14 to write, for any set of models $\mathcal{M}$ (?, Section 5.3.2):

[TABLE]

Suppose we are interested in the probability of a set $\mathbf{Q}$ of truth assignments for ground atoms in the Herbrand base of the union of program $\mathbf{P}$ with all facts in $\mathbf{PF}$. A direct translation of Expression (5) leads to an algorithm that computes bounds on $\mathbb{P}\!\left(\mathbf{Q}\right)$ as follows:

• $\bullet$

Given a plp $\left<\mathbf{P},\mathbf{PF}\right>$ and $\mathbf{Q}$, initialize $a$ and $b$ with [math].

• $\bullet$

For each total choice $\theta$ of probabilistic facts, compute the set $S$ of all stable models of $\mathbf{P}\cup\mathbf{PF}^{\downarrow\theta}$, and:

• –

if $\mathbf{Q}$ is $\mathsf{true}$ in every stable model in $S$, then $a\leftarrow a+\mathbb{P}\!\left(\theta\right)$;

• –

if $\mathbf{Q}$ is $\mathsf{true}$ in some stable model of $S$, then $b\leftarrow b+\mathbb{P}\!\left(\theta\right)$.

• $\bullet$

Return $[a,b]$ as the interval $[\underline{\mathbb{P}}\!\left(\mathbf{Q}\right),\overline{\mathbb{P}}\!\left(\mathbf{Q}\right)]$.

Note that to find whether $\mathbf{Q}$ is $\mathsf{true}$ in every stable model of a program, we must run cautious inference, and to find whether $\mathbf{Q}$ is $\mathsf{true}$ in some stable model of a program, we must run brave inference. The complexity of these logical inferences have been studied in depth in the literature (?).

For infinitely monotone credal sets we can find easy expressions for lower and upper conditional probabilities (that is, the infimum and supremum of conditional probabilities). Indeed, if $A$ and $B$ are events, then the lower probability of $A$ given $B$ is (where the superscript $c$ denotes complement) (?):

[TABLE]

when $\underline{\mathbb{P}}\!\left(A\cap B\right)+\overline{\mathbb{P}}\!\left(A^{c}\cap B\right)>0$; we then have that $\underline{\mathbb{P}}\!\left(A|B\right)=1$ when $\underline{\mathbb{P}}\!\left(A\cap B\right)+\overline{\mathbb{P}}\!\left(A^{c}\cap B\right)=0$ and $\overline{\mathbb{P}}\!\left(A\cap B\right)>0$; finally, $\underline{\mathbb{P}}\!\left(A|B\right)$ is undefined when $\overline{\mathbb{P}}\!\left(A\cap B\right)=\overline{\mathbb{P}}\!\left(A^{c}\cap B\right)=0$ (as this condition is equivalent to $\overline{\mathbb{P}}\!\left(B\right)=0$). Similarly, the upper probability of $A$ given $B$ is:

[TABLE]

when $\overline{\mathbb{P}}\!\left(A\cap B\right)+\underline{\mathbb{P}}\!\left(A^{c}\cap B\right)>0$; and we have that $\overline{\mathbb{P}}\!\left(A|B\right)=0$ when $\overline{\mathbb{P}}\!\left(A\cap B\right)+\underline{\mathbb{P}}\!\left(A^{c}\cap B\right)=0$ and $\overline{\mathbb{P}}\!\left(A^{c}\cap B\right)>0$; finally, $\overline{\mathbb{P}}\!\left(A|B\right)$ is undefined when $\overline{\mathbb{P}}\!\left(A\cap B\right)=\overline{\mathbb{P}}\!\left(A^{c}\cap B\right)=0$. We also note that the computation of lower and upper expected values with respect to infinitely monotone Choquet capacities admits relatively simple expressions (?). For instance, the lower expectation $\underline{\mathbb{E}}\!\left[f\right]=\inf_{\mathbb{P}\in\mathbb{K}}\mathbb{E}_{\mathbb{P}}\!\left[f\right]$, where $f$ is a function over the truth assignments, and $\mathbb{E}_{\mathbb{P}}\!\left[f\right]$ is the expectation of $f$ with respect to $\mathbb{P}$, is $\underline{\mathbb{E}}\!\left[f\right]=\sum_{\theta\in\Theta}\max_{\omega\in\Gamma(\theta)}f(\omega)$ (?). And there are expressions even for lower and upper conditional expectations that mirror Expression (5).

To translate these expresions into actual computations, suppose we have a sets $(\mathbf{Q}$ and $\mathbf{E}$ of truth assignments for ground atoms in the Herbrand base of the union of program $\mathbf{P}$ with all facts in $\mathbf{PF}$. To obtain bounds on $\mathbb{P}\!\left(\mathbf{Q}|\mathbf{E}\right)$, we can combine the previous algorithm with Expressions (6) and (7), to obtain:

• $\bullet$

Given a plp $\left<\mathbf{P},\mathbf{PF}\right>$ and $\mathbf{Q}$, initialize $a$, $b$, $c$, and $d$ with [math].

• $\bullet$

For each total choice $\theta$ of probabilistic facts, compute the set $S$ of all stable models of $\mathbf{P}\cup\mathbf{PF}^{\downarrow\theta}$, and:

• –

if $\mathbf{Q}\cup\mathbf{E}$ is $\mathsf{true}$ in every stable model in $S$, then $a\leftarrow a+\mathbb{P}\!\left(\theta\right)$;

• –

if $\mathbf{Q}\cup\mathbf{E}$ is $\mathsf{true}$ in some stable model of $S$, then $b\leftarrow b+\mathbb{P}\!\left(\theta\right)$;

• –

if $\mathbf{Q}$ if $\mathsf{false}$ and $\mathbf{E}$ is $\mathsf{true}$ in every stable model of $S$, then $c\leftarrow c+\mathbb{P}\!\left(\theta\right)$;

• –

if $\mathbf{Q}$ if $\mathsf{false}$ and $\mathbf{E}$ is $\mathsf{true}$ in some stable model of $S$, then $d\leftarrow d+\mathbb{P}\!\left(\theta\right)$.

• $\bullet$

Return the interval $[\underline{\mathbb{P}}\!\left(\mathbf{Q}|\mathbf{E}\right),\overline{\mathbb{P}}\!\left(\mathbf{Q},\mathbf{E}\right)]$ as follows, in case $b+d>0$ (otherwise, report failure and stop):

• –

$[0,0]$ if $b+c=0$ and $d>0$;

• –

$[1,1]$ if $a+d=0$ and $b>0$;

• –

$[a/(a+d),b/(b+c)]$ otherwise.

In fact the algorithm above has already been derived by ? (?), using clever optimization techniques (note that Cali et al. use a different strategy to handle the case where $\overline{\mathbb{P}}\!\left(\mathbf{E}\right)=0$). The advantage of our approach is that the algorithm is a transparent consequence of known facts about capacities; other than that, Cali et al. have already presented the algorithm so we do not need to dwell on it. Rather, we later return to this algorithm with a focus on the complexity of computing of the lower probability $\underline{\mathbb{P}}\!\left(\mathbf{Q}|\mathbf{E}\right)$. Algorithms that reproduce some properties of infinitely monotone Choquet capacities are also presented by ? (?) in their work on constraint logic programming.

6 The complexity of inferences: acyclic and stratified probabilistic logic programs

In this section we focus on the computation of inferences for acyclic and stratified plps; in these cases both the credal and the well-founded semantics agree. We focus on the following decision problem:

Input:

A plp $\left<\mathbf{P},\mathbf{PF}\right>$ whose probabilities are rational numbers, a pair $(\mathbf{Q},\mathbf{E})$, called the query, where both $\mathbf{Q}$ and $\mathbf{E}$ are sets of truth assignments to atoms in the Herbrand base of the union of program $\mathbf{P}$ and all facts in $\mathbf{PF}$, and a rational $\gamma\in[0,1]$.

Output:

Whether or not $\mathbb{P}\!\left(\mathbf{Q}|\mathbf{E}\right)>\gamma$; by convention, output is NO (that is, input is rejected) if $\mathbb{P}\!\left(\mathbf{E}\right)=0$.

We refer to this complexity as the inferential complexity of plps. One may also be interested in the complexity of inferences when the plp is fixed, and the only input is the query $(\mathbf{Q},\mathbf{E})$. This is the query complexity of the program; to define it, consider:

Fixed:

A plp $\left<\mathbf{P},\mathbf{PF}\right>$, whose probabilities are rational numbers, that employs a vocabulary $\mathbf{R}$ of predicates.

Input:

A pair $(\mathbf{Q},\mathbf{E})$, called the query, where both $\mathbf{Q}$ and $\mathbf{E}$ are sets of truth assignments to atoms of predicates in $\mathbf{R}$, and a rational $\gamma\in[0,1]$.

Output:

Whether or not $\mathbb{P}\!\left(\mathbf{Q}|\mathbf{E}\right)>\gamma$; by convention, output is NO if $\mathbb{P}\!\left(\mathbf{E}\right)=0$.

Say that the query complexity of a class $\mathcal{P}$ of plps is in a complexity class $\mathsf{C}$ if the complexity of this decision problem is in $\mathsf{C}$ for every plp in $\mathcal{P}$. And say that the query complexity of $\mathcal{P}$ is $\mathsf{C}$-hard if each decision problem in $\mathsf{C}$ can be reduced, with many-one reductions, to a decision problem for at least one plp in $\mathcal{P}$. And say that the query complexity of $\mathcal{P}$ is $\mathsf{C}$-complete if $\mathcal{P}$ is both in $\mathsf{C}$ and $\mathsf{C}$-hard.

In practice, one may face situations where a plp may be small compared to the query, or where a single plp is queried many times; then query complexity is the concept of interest.

The definition of query complexity is clearly related to the concept of data complexity found in database theory (?); indeed we have used “data complexity” in previous related work (?). Here we prefer to use “query” instead of “data” because usually data complexity fixes the rules and varies the number of facts; in this paper we keep both rules and facts fixed. In fact, we have already mentioned the highly relevant work by ? (?), where they study the complexity of various types of probabilistic logic programs; in that work they use data complexity to refer to the complexity of computing probabilistic for fixed queries and fixed programs, as the stock of facts and probabilistic facts varies. Note also that Ceylan et al. consider a much more sophisticated language for queries that we do; for them, a query can be any union of Boolean conjunctive query as usually employed in databases (?). The distinction between “query” and “data” thus seems significant in the context of probabilistic logic programming.

We must further comment on a few parallel results by ? (?). They analyze the complexity of plps under two semantics; one of them is in line with Sato’s distribution semantics, and another one is geared towards inconsistent programs; neither is equivalent to the credal or the well-founded semantics. Moreover, they focus on queries that are Boolean formulas, they do not allow for conditioning evidence, and they use a somewhat different version of probabilistic facts called contexts (that can be reproduced with our probabilistic facts). Despite these differences, in dealing with their first semantics they prove statements that are related to results in Section 6.1. More precisely: by translating the various languages and arguments appropriately, the points made by our Theorems 18 and 19 can be obtained from their results on full acyclic programs; also our Theorem 22 is comparable to their corresponding result, even though our “query” complexity is not their “data” complexity.333We note that our results on acyclic programs appeared (?) almost simultaneously to the publication by ? (?), and were produced independently. We decided to include our proof of Theorem 19 in full here because we need the techniques in later proofs, and because we find that our techniques illuminate the matter adequately.

6.1 Acyclic probabilistic logic programs

We start with acyclic plps. In this case the credal and the well-founded semantics define a single distribution, given by a Bayesian network whose structure is the program’s grounded dependency graph, and whose parameters are obtained from the program’s Clark completion (?, ?).

Example 16**.**

Take a simplified version of the plp in Example 5, without predicates $\mathsf{calls}$ and $\mathsf{neighbor}$:

[TABLE]

We can understand this plp as the specification of the Bayesian network in Figure 4. Note that the structure of the network is just the grounded dependency graph, and the logical sentence comes directly from the Clark completion. $\hfill\Box$

Conversely, any propositional Bayesian network can be specified by an acyclic propositional plp (?, ?). The argument is simple, and we show it by turning Example 16 upside down:

Example 17**.**

Suppose we have the Bayesian network in Figure 5. This Bayesian network is equivalent to the Bayesian network in Figure 4 (that is: the same distribution is defined over $\mathsf{alarm}$, $\mathsf{burglary}$, $\mathsf{earthquake}$). And the latter network is specified by an acyclic plp. $\hfill\Box$

By combining these arguments, we see that inference in acyclic propositional plps has the complexity of inference in Bayesian networks (?, ?):

Theorem 18**.**

The inferential complexity of inference in acyclic propositional plps is $\mathsf{PP}$-complete.

One might suspect that a bound on predicate arity would yield the same $\mathsf{PP}$-completeness, because the grounding of a plp would then produce only polynomially-many ground atoms. Surprisingly, this is not the case here, as shown by the next theorem.

Theorem 19**.**

The inferential complexity of inference in acyclic plps with bounded predicate arity is $\mathsf{PP}^{\mathsf{NP}}$-complete.

Proof.

To prove membership, start with the “unconditional” decision $\mathbb{P}\!\left(\mathbf{Q}\right)>\gamma$. This decision problem is in $\mathsf{PP}^{\mathsf{NP}}$; this follows from the fact that logical reasoning with acyclic normal logic programs is $\Delta_{2}^{P}$-complete (?, Table 5) (that is, $\mathsf{P}^{\mathsf{NP}}$-complete). Consider a nondeterministic Turing machine that goes through all probabilistic facts. For each probabilistic fact $\alpha::A\mbox{\large.}$, where $\alpha$ is a rational such that $\alpha=\mu/\nu$ for some (smallest) integers $\mu$ and $\nu$, the machine nondeterministically decides whether to keep $A$ as a fact or discard it; then the machine creates $\mu$ computation paths if $A$ is to be kept (all these computational paths reach the same point), and $\nu-\mu$ computation paths if $A$ is to be discarded (again, these computation paths reach the same point). Note that these computation paths can be created with polynomial effort even if $\mu$ and $\nu$ are specified in binary notation. Then, for the particular selection of probabilistic facts that are not discarded, the machine processes the resulting acyclic normal logic program: logical reasoning can determine whether any set of truth assignments, for atoms in the Herbrand base, hold or not. The input is in the language if more than half the computation paths of this machine are accepting paths. This decides whether $\mathbb{P}\!\left(\mathbf{Q}\right)>1/2$.

Now consider membership of the decision $\mathbb{P}\!\left(\mathbf{Q}|\mathbf{E}\right)>\gamma$; we process this decision as follows.444We are indebted to Cassio Polpo de Campos for suggesting this technique; the probabilities attached to probabilistic facts are as proposed by Park and described by ? (?, Theorem 11.5). Suppose the query consists of $\mathbf{Q}=\{Q_{1},\dots,Q_{n}\}$ and $\mathbf{E}=\{E_{1},\dots,E_{m}\}$, where each $Q_{i}$ and each $E_{j}$ is a literal. The simpler case is $\gamma\geq 1/2$, so assume it to begin. Then, as the query is processed, introduce $\mathsf{aux1}\colonminus Q_{1},\dots,Q_{n}\mbox{\large.}$, $\mathsf{aux2}\colonminus E_{1},\dots,E_{m}\mbox{\large.}$, $\mathsf{aux3}\colonminus\mathsf{aux1},\mathsf{aux2},\mathsf{aux4}\mbox{\large.}$, $\mathsf{aux3}\colonminus\mathbf{not}\;\mathsf{aux2},\mathsf{aux5}\mbox{\large.}$, where each literal $Q_{i}$ or $E_{j}$ is written as the corresponding subgoal, and $(1/(2\gamma))::\mathsf{aux4}\mbox{\large.}$, $0.5::\mathsf{aux5}\mbox{\large.}$. Thus $\mathbb{P}\!\left(\mathsf{aux3}=\mathsf{true}\right)>1/2\Leftrightarrow(1/(2\gamma))\mathbb{P}\!\left(\mathbf{Q},\mathbf{E}\right)+(1/2)(1-\mathbb{P}\!\left(\mathbf{E}\right))>1/2\Leftrightarrow\mathbb{P}\!\left(\mathbf{Q}|\mathbf{E}\right)>\gamma$ (that is, the decision on $\mathbb{P}\!\left(\mathsf{aux3}=1\right)>1/2$ yields the decision on $\mathbb{P}\!\left(\mathbf{Q}|\mathbf{E}\right)>\gamma$). Now if $\gamma<1/2$, then introduce $\mathsf{aux3}\colonminus\mathbf{not}\;\mathsf{aux1},\mathsf{aux2},\mathsf{aux6}\mbox{\large.}$, remove $\mathsf{aux4}$, and introduce $(1-2\gamma)/(2-2\gamma)::\mathsf{aux6}\mbox{\large.}$. Thus $\mathbb{P}\!\left(\mathsf{aux3}=\mathsf{true}\right)>1/2\Leftrightarrow\mathbb{P}\!\left(\mathbf{Q},\mathbf{E}\right)+(1-2\gamma)/(2-2\gamma)(\mathbb{P}\!\left(\mathbf{E}\right)-\mathbb{P}\!\left(\mathbf{Q},\mathbf{E}\right))+(1/2)(1-\mathbb{P}\!\left(\mathbf{E}\right))>1/2\Leftrightarrow\mathbb{P}\!\left(\mathbf{Q}|\mathbf{E}\right)>\gamma$, as desired. (This technique is used several times in latter proofs.)

Hardness is shown by building a plp that solves the problem $\#_{1}\mathsf{3CNF}(>)$ as defined in Section 2.2; that is, one has a propositional sentence $\phi$ in 3CNF with two sets of logical variables $\mathbf{X}$ and $\mathbf{Y}$, and the goal is to decide whether the number of truth assignments for $\mathbf{X}$ that satisfy $\exists\mathbf{Y}:\phi(\mathbf{X},\mathbf{Y})$ is larger than a given integer $M$. We take $\phi$ to be a conjunction of clauses $c_{1},\dots c_{k}$; each clause $c_{j}$ contains an ordered triplet of propositional variables.

For instance, we might have as input the integer $M=1$ and the formula

[TABLE]

In this case the input is accepted (the number of satisfying assignments is $2$). Note that the last clause is equivalent to $\neg y_{1}$; we pad the clause so as to have three literals in it.

For each propositional variable $y_{i}$, we introduce a corresponding logical variable $Y_{i}$. The ordered tuple of propositional variables in clause $c_{j}$ corresponds to a tuple of propositional variables that is denoted by $\mathbf{Y}_{j}$; these are the propositional variables in $c_{j}$ that belong to $\mathbf{Y}$. In Expression (8), $\mathbf{Y}_{1}=\mathbf{Y}_{2}=\mathbf{Y}_{3}=[Y_{1}]$.

We use a few constants and predicates. Two constants, $\mathsf{0}$ and $\mathsf{1}$, stand for $\mathsf{false}$ and $\mathsf{true}$ respectively. Also, we use [math]-arity predicates $\mathsf{x_{i}}$, each one standing for a propositional variable $x_{i}$ in $\mathbf{X}$. And we use predicates $\mathsf{c_{1}},\ldots,\mathsf{c_{k}}$, each one standing for a clause $c_{j}$. The arity of each $\mathsf{c_{j}}$ is the length of $\mathbf{Y}_{j}$, denoted by $d_{j}$.

For each $\mathsf{c_{j}}$, go over the $2^{d_{j}}$ possible assignments of $\mathbf{Y}_{j}$. That is, if $d_{j}=1$, then go over $\mathsf{c_{j}}(\mathsf{0})$ and $\mathsf{c_{j}}(\mathsf{1})$; if $d_{j}=2$, then go over $\mathsf{c_{j}}(\mathsf{0},\mathsf{0})$, $\mathsf{c_{j}}(\mathsf{0},\mathsf{1})$, $\mathsf{c_{j}}(\mathsf{1},\mathsf{0})$ and $\mathsf{c_{j}}(\mathsf{1},\mathsf{1})$. And if $d_{j}=3$, go over the $8$ assignments. Note that if $d_{j}=0$, there is only one “empty” assignment to visit. Thus there are at most $8k$ assignments to consider.

Suppose then that we have predicate $\mathsf{c_{j}}$, and we take the assignment $\mathbf{y}$ (which may be empty). If $c_{j}$ is $\mathsf{true}$ for $\mathbf{y}$, regardless of the possible assignments for propositional variables $x_{i}$, then just introduce the fact

[TABLE]

If instead $c_{j}$ is $\mathsf{false}$ for $\mathbf{y}$, regardless of the possible assignments for propositional variables $x_{i}$, then just move to another assignment (that is, there are no propositional variables $x_{i}$ in $c_{j}$, and the clause is $\mathsf{false}$ for $\mathbf{y}$; by leaving $\mathsf{c_{j}}(\mathbf{y})$, we guarantee that it is forced to be $\mathsf{false}$ by the semantics). Otherwise, there are propositional variables in $\mathbf{X}$ that affect the truth value of $c_{j}$ when $\mathbf{y}$ is fixed; there may be one, two or three such propositional variables. Take the first one of them, denoted by $x_{j1}$, and introduce the rule

[TABLE]

If there is a second propositional variable $x_{j2}$ that affects the truth value of $c_{j}$ when $\mathbf{y}$ is fixed, add a similar rule $\mathsf{c_{j}}(\mathbf{y})\colonminus[\mathbf{not}]\;\mathsf{x_{j2}}$. And similarly if there is a third propositional variable $x_{j3}$ that affects the truth value of $c_{j}$. Note that these rules create a disjunction for $\mathsf{c_{j}}$, in effect encoding the clause $c_{j}$ for fixed $\mathbf{y}$.

Finally, introduce the rule

[TABLE]

and probabilistic facts (one per predicate $\mathsf{x_{i}}$)

[TABLE]

The Clark completion of the plp just constructed encodes the $\#_{1}\mathsf{3CNF}(>)$ problem of interest, thus proving $\mathsf{PP}^{\mathsf{NP}}$-hardness: to determine whether $\exists\mathbf{Y}:\phi(\mathbf{X},\mathbf{Y})$ has more than $M$ satisfying assignments, decide whether $\mathbb{P}\!\left(\mathsf{cnf}=\mathsf{true}\right)>M/2^{n}$, where $n$ is the number of propositional variables in $\mathbf{X}$.

For instance, given the formula in Expression (8), generate the following plp:

[TABLE]

By determining whether $\mathbb{P}\!\left(\mathsf{cnf}=\mathsf{true}\right)>M/2^{2}$, we decide whether the number of truth assignments for $x_{1}$ and $x_{2}$ such that $\forall y_{1}\varphi(x_{1},x_{2},y_{1})$ holds is larger than $M$. ∎

Intuitively, this results shows that, to produce an inference for a plp with bounded predicate arity, one must go through the truth assignments for polynomially many groundings, guessing one at a time (thus a counting nondeterministic Turing machine), and, for each assignment, it is then necessary to use an $\mathsf{NP}$-oracle to construct the probability values. Theorem 19 suggests that acyclic plps capture a larger set of probabilistic languages than many probabilistic relational models that stay within $\mathsf{PP}$ (?).

The next step is to remove the bound on arity. We obtain:

Theorem 20**.**

The inferential complexity of inference in acyclic plps is $\mathsf{PEXP}$-complete.

Proof.

Membership follows from grounding the plp.555A short proof of membership is obtained by applying the same concise argument used in the proof of Theorem 24(c); here we present a longer but possibly more intuitive argument based on inference on Bayesian networks. If the plp has $n$ constants, then a relation of arity $k$ produces $n^{k}$ groundings. Each one of these exponentially many groudings corresponds to a node of a (necessarily acyclic) Bayesian network. To write down the conditional probabilities associated with each node of the grounded Bayesian network, take the Clark completion of the program, and ground the expresions. For each non-root node we have a first-order formula that can be written as a possibly exponentially-long quantifier-free formula. Now to determine whether $\mathbb{P}\!\left(\mathbf{Q}\right)>1/2$, we can use a probabilistic Turing machine that runs inference for the exponentially large (grounded) Bayesian network (or, rather, there is an exponential-time Turing machine that guesses a truth assignment for all grounded probabilistic facs, and for each such truth assignment, computes the truth assignment for any other atom by going through the possibly exponentially large non-root node completions).

To prove hardness, we encode an exponential-time nondeterministic Turing machine $\mathbb{M}$ using logical formulas that are directly produced by the Clark completion of an acyclic normal logic program $\mathbf{P}$. Assume that $\mathbb{M}$ can solve some $\mathsf{PEXP}$-complete problem; that is, for a $\mathsf{PEXP}$-complete language $\mathcal{L}$, $\ell\in\mathcal{L}$ iff $\mathbb{M}$ halts within time $2^{n}$ with more than half of paths accepting $\ell$, where $n$ is some polynomial on the length of $\ell$. We also add probabilistic facts $\mathbf{PF}$ to $\mathbf{P}$, so that an inference in the resulting plp decides whether the number of acceptings paths of $\mathbb{M}$ is larger than half of the total number of computation paths (thus deciding the same language as $\mathbb{M}$ does). So, consider the encoding of Turing machines that is described by ? (?, Theorem 3.2.4), summarized as follows. Suppose $\mathbb{M}$ has states $q$, with an initial state $q_{0}$, an accepting state $q_{a}$, and a rejecting state $q_{r}$; suppose also that $\mathbb{M}$ uses an alphabet with symbols $\sigma$ (in the alphabet there is a blank symbol $\scriptstyle\sqcup$); finally suppose that $\mathbb{M}$ has a transition function $\delta$ that takes a pair $(q,\sigma)$, understood as state $q$ and symbol $\sigma$ read by the machine head, and returns one of a number of triplets $(q^{\prime},\sigma^{\prime},m)$, where $q^{\prime}$ is the next state, $\sigma^{\prime}$ is the symbol to be written at the tape, and $m$ is either $-1$ (head goes to the left), [math] (head stays at the same position), and $1$ (head goes to the right). Assume that the alphabet is enlarged so that every pair $(q,\sigma)$ is also a possible symbol. The input to the machine is a sequence of symbols $(q_{0},\sigma_{0}^{1}),\sigma_{0}^{2},\dots,\sigma_{0}^{m}$, and a configuration of the tape is $\sigma^{1},\sigma^{2},\dots,(q,\sigma),\dots,\sigma^{2n}$ (note that the “useful” portion of the tape runs from position $1$ to position $2^{n}$).

The encoding of $\mathbb{M}$ is obtained by introducing a number of predicates and a number of first-order sentences $\phi_{i}$; when all these sentences hold, then any interpretation for the predicates is an accepting computation. We omit the logical expressions of this encoding as they can be taken from Grädel’s presentation. In any case, if we decide whether the number of interpretations for the predicates in these sentences is larger than half of the number of possible interpretations, we obtain the desired decision. We enforce each sentence $\phi_{i}$ by introducing a predicate $\mathsf{aux_{i}}$ and a rule $\mathsf{aux_{i}}\colonminus\phi_{i}$ (where we write $\phi_{i}$ in the rule with the understanding that $\phi_{i}$ is obtained as the Clark completion of a set of auxiliary predicates and rules; recall that conjunction, disjunction and negation are available, as well as existential quantifiers; universal quantifiers are produced by negating existential ones); then the sentence $\phi_{i}$ holds when $\{\mathsf{aux_{i}}=\mathsf{true}\}$ holds. We simply collect all these truth assignments in the set $\mathbf{E}$. Now, we must have one of the sentences in $\mathbb{M}$’s encoding as a “detector” for the accepting state; that is, $\exists X:\mathsf{state}_{q_{a}}(X)$, where $X$ indexes the computation steps, and $\mathsf{state}_{q_{a}}(X)$ is a predicate that indicates that at computation step $X$ the state is $\mathsf{state}_{q_{a}}$. Denote by $\mathsf{aux_{a}}$ the auxiliary predicate associated with the latter sentence. At this point we can reproduce the behavior of $\mathbb{M}$ if we focus on interpretations that satisfy $\mathbf{E}$. The next step is to encode the input. Now, the input symbols can be inserted by appropriate facts (these facts refer to predicates introduced in the encoding). And the final step is to count the accepting computations. First we must assume that, once $\mathbb{M}$ reaches $q_{a}$ or $q_{r}$, it stays with the same configuration (it just keeps repeating the state and the tape), so that the number of accepting paths is the same number of interpretations that satisfy $\{\mathsf{aux_{a}}=\mathsf{true}\}$; this assumption is harmless as $\mathbb{M}$ can always be modified to do it. Then we add, for each predicate $\mathsf{r}$ that is introduced in the construction, except the ones in $\mathbf{E}$, the probabilistic fact $0.5::\mathsf{r}(X_{1},\dots,X_{k})$, where $k$ is the arity of $\mathsf{r}$. Given all of this, the decision $\mathbb{P}\!\left(\mathsf{aux_{a}}=\mathsf{true}|\mathbf{E}\right)>1/2$ determines whether the number of “accepting” interpretations for $\mathbb{M}$ is larger than half the number of intepretations for $\mathbb{M}$. Thus hardness obtains. ∎

Consider query complexity. The following result is handy:

Theorem 21**.**

Query complexity is $\mathsf{PP}$-hard for the following plp:

[TABLE]

[TABLE]

Proof.

Consider a CNF formula $\varphi(x_{1},\ldots,x_{n})$ with clauses $c_{1},\ldots,c_{m}$ and propositional variables $x_{1},\dots,x_{n}$. Let $P_{j}$ (resp., $N_{j}$) be a vector denoting the indices of the positive (negative) literals $x_{i}$ ($\neg x_{i}$) in clause $j$. We can encode the truth-value of a clause $c_{j}$ as $\mathsf{c}(j)$, the truth-value of $x_{i}$ as $\mathsf{t}(i)$, and the occurrence of a positive (negative) literal $x_{i}\in P_{j}$ ($x_{i}\in N_{j}$) as $\mathsf{pos}(i,j)$ ($\mathsf{neg}(i,j)$). So assemble a query $\mathbf{Q}$ containing assignments to $\{\mathsf{c}(j)=\mathsf{true}\}$ for $j=1,\ldots,m$, $\{\mathsf{pos}(i,j)=\mathsf{true}\}$ for $i\in P_{j},j=1,\ldots,m$, $\{\mathsf{neg}(i,j)=\mathsf{true}\}$ for $i\in N_{j},j=1\ldots,m$. Now if a grounding of $\mathsf{pos}$ or $\mathsf{neg}$ is not already assigned $\mathsf{true}$, then assign it to $\mathsf{false}$ and add this assignment to $\mathbf{Q}$. The Clark completion defines $\mathsf{c}(j)\Leftrightarrow\bigvee_{i\in P_{j}}\mathsf{t}(i)\vee\bigvee_{i\in N_{j}}\neg\mathsf{t}(i)$ for every $c_{j}$. And the number of assignments to $x_{1},\dots,x_{n}$ that satisfy $\varphi$ is larger than $M$ iff $\mathbb{P}\!\left(\mathbf{Q}\right)>M/2^{2s^{2}+s}$ where $s=\max(m,n)$; hence the desired hardness obtains. ∎

Consequently:

Theorem 22**.**

The query complexity of inference for acyclic plps is $\mathsf{PP}$-complete.

Proof.

Hardness follows from Theorem 21. Membership is obtained using the same reasoning in the proof of Theorem 19, only noting that, once the probabilistic facts are selected, logical reasoning with the resulting acyclic normal logic program can be done with polynomial effort (?, Theorem 5.1); thus $\mathbb{P}\!\left(\mathbf{Q}\right)>\gamma$ can be decided within $\mathsf{PP}$. ∎

There are subclasses of acyclic plps that characterize well-known tractable Bayesian networks. An obvious one is the class of propositional acyclic programs whose grounded dependency graph has bounded treewidth, as Bayesian networks subject to such a bound are tractable (?). As another interesting example, consider the two-level networks that are processed by the Quick-Score algorithm (?); that is, two-level networks where the top level consists of marginally independent “diseases” and the bottom level consists of “findings” that are conditionally independent given the diseases, and that are determined by noisy-or gates. Such a network can be easily encoded using a propositional acyclic plp; these plps inherit the fact that inference is polynomial when $\mathbf{Q}$ contains only negated atoms (that is, only $\mathsf{false}$). Alas, this tractability result is quite fragile, as “positive” evidence breaks polynomial behavior as long as $\mathsf{P}\neq\mathsf{NP}$ (?). Yet another tractable class consists of acyclic definite propositional plps such that each atom is the head of at most one rule: inference in this class is polynomial when $\mathbf{Q}$ contains only $\mathsf{true}$. This is obtained by noting that the Clark completion of these programs produces Bayesian networks that are specified using only conjunction, and a polynomial algorithm obtains from results by ? (?). This is also a fragile result:

Proposition 23**.**

Inference for the class of acyclic propositional plps such that each atom is the head of at most one rule is $\mathsf{PP}$-complete even if (a) $\mathbf{Q}$ contains only $\mathsf{true}$ but the program contains $\mathbf{not}$; (b) the program is definite but $\mathbf{Q}$ contains $\mathsf{false}$.

Proof.

Membership follows, for both (a) and (b), from Theorem 18. So, consider hardness. Any plp in Case (b) produces, as its Clark completion, a Bayesian network that is specified using conjunctions; inference for this sort of Bayesian network is $\mathsf{PP}$-complete when evidence can be “negative” (?). Hardness for Case (a) then obtains easily, because one can use negation to turn “positive” evidence into “negative” evidence. ∎

6.2 Stratified probabilistic logic programs

A stratified normal logic program has the useful property that its universally adopted semantics produces a single interpretation (and is equal to its stable and well-founded semantics). Because every total choice of a stratified plp produces a stratified normal logic program, the credal/well-founded semantics of a stratified plp is a unique distribution.

One might fear that in moving from acyclic to stratified programs we must pay a large penalty. This is not the case: the complexity classes remain the same as in Section 6.1:

Theorem 24**.**

For locally stratified plps, inferential complexity is $\mathsf{PEXP}$-complete; it is $\mathsf{PP}^{\mathsf{NP}}$-complete for plps with bounded predicate arity; it is $\mathsf{PP}$-complete for propositional plps. For locally stratified plps, query complexity is $\mathsf{PP}$-complete.

Proof.

For propositional stratified plps, hardness comes from the fact that a Bayesian network on binary random variables can be encoded by a stratified program (indeed, by an acyclic program), and inference with such networks is $\mathsf{PP}$-complete (?, ?). Membership is obtained using the same reasoning in the proof of Theorem 19, only noting that, once the probabilistic facts are selected, logical reasoning with the resulting stratified normal logic program can be done with polynomial effort (?, Table 2).

For stratified programs with bounded predicate arity, hardness follows from Theorem 19. Membership is obtained using the same reasoning in the proof of Theorem 19; in fact that proof of membership applies directly to stratified programs with bounded arity.

For general stratified plps, hardness is argued as in the proof of Theorem 20. Membership follows from the fact that we can ground the plp into an exponentially large propositional plp. Once the (exponentially-many) probabilistic facts are selected, the Turing machine is left with a stratified propositional normal logic program, and logical inference is polynomial in the size of this program (that is, logical inference requires exponential effort).

Finally, hardness of query complexity follows from Theorem 21. Membership is obtained using the same reasoning in the proof of Theorem 19, only noting that, once the probabilistic facts are selected, logical reasoning with the resulting stratified normal logic program can be done with polynomial effort as guaranteed by the analysis of data complexity of stratified normal logic programs (?). ∎

We noted, at the end of Section 6.1, that some sub-classes of acyclic programs display polynomial behavior. We now show an analogue result for a sub-class of definite (and therefore stratified, but possibly cyclic) programs with unary and binary predicates:

Proposition 25**.**

Inferential complexity is polynomial for queries containing only $\mathsf{true}$, for plps where: (a) every predicate is unary or binary, and facts can be asserted about them; (b) probabilistic facts can be of the form $\alpha::\mathsf{a}(X)\mbox{\large.}$, $\alpha::\mathsf{a}(a)$, $\alpha::\mathsf{r}(X,Y)$ (that is, each unary predicate can be associated with ground or non-ground probabilistic facts, while each binary predicate can be associated to a particular non-ground probabilistic fact); (c) no binary predicate is the head of a rule that has a body; (d) each atom is the head of at most one rule that has a body, and only the three following rule forms are allowed:

[TABLE]

Proof.

We show that the inference can be reduced to a tractable weighted model counting problem. First, ground the program in polynomial time (because each rule has at most two logical variables). Since the resulting program is definite, only atoms that are ancestors of the queries in the grounded dependency graph are relevant for determining the truth-value of the query in any logic program induced by a total choice (this follows as resolution is complete for propositional definite programs). Thus, discard all atoms that are not ancestors of a query atom. For the query to be true, the remaining atoms that are not probabilistic facts are forced to be true by the semantics. So collect all rules of the sort $\mathsf{a}(a)\colonminus\mathsf{r}(a,b)\,\mbox{\large.}$, $\mathsf{a}(a)\colonminus\mathsf{r}(b,a)\,\mbox{\large.}$, plus all facts and all probabilistic facts. This is an acyclic program, so that its Clark completion gives the stable model semantics. This completion is a formula containing a conjunction of subformulas $\mathsf{a}(a)\Leftrightarrow\bigvee_{b}\mathsf{r}(a,b)$, $\mathsf{a}(a)\Leftrightarrow\bigvee_{a}\mathsf{r}(a,b)$, and unit (weighted) clauses corresponding to (probabilistic) facts. The query is satisfied only on models where the lefthand side of the definitions are true, which is equivalent to reducing the subformulas to their righthand side. The resulting weighted model counting problem has been shown to be polynomial-time solvable (?). ∎

7 The complexity of inferences: credal semantics

Now consider plps that may be non-stratified. We have to adapt the definitions of inferential and query complexity to account for the fact that we now have lower and upper probabilities. First we focus on lower probabilities; the lower-probability version of inferential complexity for a class of plps is the complexity of the following decision problem:

Input:

A plp $\left<\mathbf{P},\mathbf{PF}\right>$ whose probabilities are rational numbers, a pair $(\mathbf{Q},\mathbf{E})$, called the query, where both $\mathbf{Q}$ and $\mathbf{E}$ are sets of truth assignments to atoms in the Herbrand base of the union of program $\mathbf{P}$ and all facts in $\mathbf{PF}$, and a rational $\gamma\in[0,1]$.

Output:

Whether or not $\underline{\mathbb{P}}\!\left(\mathbf{Q}|\mathbf{E}\right)>\gamma$; by convention, output is NO (that is, input is rejected) if $\overline{\mathbb{P}}\!\left(\mathbf{E}\right)=0$.

The lower-probability version of query complexity is, accordingly:

Fixed:

A plp $\left<\mathbf{P},\mathbf{PF}\right>$, whose probabilities are rational numbers, that employs a vocabulary $\mathbf{R}$ of predicates.

Input:

A pair $(\mathbf{Q},\mathbf{E})$, called the query, where both $\mathbf{Q}$ and $\mathbf{E}$ are sets of truth assignments to atoms of predicates in $\mathbf{R}$, and a rational $\gamma\in[0,1]$.

Output:

Whether or not $\underline{\mathbb{P}}\!\left(\mathbf{Q}|\mathbf{E}\right)>\gamma$; by convention, output is NO if $\overline{\mathbb{P}}\!\left(\mathbf{E}\right)=0$.

So, we are ready to state our main results on complexity for the credal semantics. To understand these results, consider the computation of lower probabilities by the algorithms in Section 5: the basic idea is to go through all possible configurations of probabilistic facts, and each configuration requires runs of cautious/brave inference (that it, it is necessary to check whether all possible stable models satisfy $\mathbf{Q}\cap\mathbf{E}$, and whether all possible stable models fail to satisfy $\mathbf{Q}$ while satisfying $\mathbf{E}$. Thus the proof strategies employed previously can be adapted to some extent, by using cautious/brave inference in our Turing machines. We have:

Theorem 26**.**

Adopt the credal semantics for plps, and assume that input plps are consistent. The lower-probability version of inferential complexity is $\mathsf{PP}^{\mathsf{NP}^{\mathsf{NP}}}$-complete for plps where all predicates have a bound on arity, and $\mathsf{PP}^{\mathsf{NP}}$-complete for propositional plps. The lower-probability version of query complexity is $\mathsf{PP}^{\mathsf{NP}}$-complete.

Proof.

We first focus on propositional programs.

To prove membership, we describe a polynomial time nondeterministic Turing machine such that more than half of its computation paths, on a given input, end up accepting iff the input is an YES instance. The machine receives the plp $\left<\mathbf{P},\mathbf{PF}\right>$, the pair $(\mathbf{Q},\mathbf{E})$, and the rational $\gamma\in[0,1]$ as input. In case $\underline{\mathbb{P}}\!\left(\mathbf{Q},\mathbf{E}\right)+\overline{\mathbb{P}}\!\left(\neg\mathbf{Q},\mathbf{E}\right)>0$, where we use $\neg\mathbf{Q}$ to indicate that $\mathbf{Q}$ does not hold, we have to decide whether:

[TABLE]

Write $\gamma$ as $\mu/\nu$ for the smallest possible integers $\mu$ and $\nu$, such that $\nu>0$, to conclude that our decision is whether

[TABLE]

In case $\underline{\mathbb{P}}\!\left(\mathbf{Q},\mathbf{E}\right)+\overline{\mathbb{P}}\!\left(\neg\mathbf{Q},\mathbf{E}\right)=0$, there are a few cases to consider, as indicated by the discussion around Expression (6). First, if $\overline{\mathbb{P}}\!\left(\mathbf{Q}\cap\mathbf{E}\right)=0$, the machine must return NO (numeric probability value is not defined); and if $\overline{\mathbb{P}}\!\left(\mathbf{Q}\cap\mathbf{E}\right)>0$, the machine must return NO if $\gamma=1$ and YES if $\gamma<1$. One simple way to capture all these cases is this: if $\overline{\mathbb{P}}\!\left(\mathbf{Q},\mathbf{E}\right)>0$ and $\overline{\mathbb{P}}\!\left(\neg\mathbf{Q},\mathbf{E}\right)=0$ and $\gamma<1$, then return YES and stop; otherwise return YES or NO according to inequality in Expression (9). Thus the machine starts by handling the special case in the previous sentence. If $\gamma<1$, then the machine determines whether $\overline{\mathbb{P}}\!\left(\mathbf{Q}\cap\mathbf{E}\right)>0$ and $\overline{\mathbb{P}}\!\left(\neg\mathbf{Q}\cap\mathbf{E}\right)=0$ using the $\mathsf{NP}$ oracle twice. In each case, the oracle guesses a total choice and determines, using brave inference, whether there is a stable model that satisfies the event of interest. If there is no such total choice, then the upper probability is zero. So, if $\gamma<1$ and $\overline{\mathbb{P}}\!\left(\mathbf{Q}\cap\mathbf{E}\right)>0$ and $\overline{\mathbb{P}}\!\left(\neg\mathbf{Q}\cap\mathbf{E}\right)=0$, move into the accepting state; otherwise, move to some state $q$ and continue.

From $q$, the machine “goes through” the possible selections of probabilistic facts, operating similarly to the second algorithm in Section 5. We will use the fact that cautious logical reasoning is $\mathsf{coNP}$-complete and brave logical reasoning is $\mathsf{NP}$-complete (?, Table 2).

The machine proceeds from $q$ as in the proof of Theorem 19, nondeterministically selecting whether each fact is kept or discarded. Suppose we have $n$ ground probabilistic facts $\alpha_{1}::A_{1}\mbox{\large.},\dots,\alpha_{n}::A_{n}\mbox{\large.}$. For each probabilistic fact $\alpha_{i}::A_{i}\mbox{\large.}$, where $\alpha_{i}=\mu_{i}/\nu_{i}$ for smallest integers $\mu_{i}$ and $\nu_{i}$ such that $\nu_{i}>0$, the machine creates $\mu_{i}$ computation paths out of the decision to keep $A_{i}$, and $\nu_{i}-\mu_{i}$ computation paths out of the decision to discard $A_{i}$. Note that after guessing the status of each probabilistic fact the machine may branch in at most $\nu_{i}$ paths, and the total number of paths out of this sequence of decisions is $\prod_{i=1}^{n}\nu_{i}$. Denote this latter number by $N$. At this point the machine has a normal logic program, and it runs cautious inference to determine whether $\mathbf{Q}\cap\mathbf{E}$ holds in every stable model of this program. Cautious logical reasoning is solved by the $\mathsf{NP}$ oracle. If indeed $\mathbf{Q}\cap\mathbf{E}$ holds in every stable model of this program, the machine moves to state $q_{1}$. Otherwise, the machine runs brave inference to determine whether $\mathbf{Q}$ is $\mathsf{false}$ while $\mathbf{E}$ is $\mathsf{true}$ in some stable model of the program. Brave logical reasoning is solved by the $\mathsf{NP}$ oracle. And if indeed $\mathbf{Q}$ is $\mathsf{false}$ while $\mathbf{E}$ is $\mathsf{true}$ in some stable model of the program, the machine moves to state $q_{2}$. Otherwise, the machine moves to state $q_{3}$. Denote by $N_{1}$ the number of computation paths that arrive at $q_{1}$, and similarly for $N_{2}$ and $N_{3}$. From $q_{1}$ the machine branches into $\nu-\mu$ computation paths that all arrive at the accepting state (thus there are $(\nu-\mu)N_{1}$ paths through $q_{1}$ to the accepting state). And from $q_{2}$ the machine branches into $\mu$ computation paths that all arrive at the rejecting state. Finally, from $q_{3}$ the machine nondeterministically moves either into the accepting or the rejecting state. Thus the number of accepting computation paths is larger than the number of rejecting computation paths iff

[TABLE]

Note that, by construction, $N_{1}/N=\underline{\mathbb{P}}\!\left(\mathbf{Q},\mathbf{E}\right)$ and $N_{2}/N=\overline{\mathbb{P}}\!\left(\neg\mathbf{Q},\mathbf{E}\right)$; thus the number of accepting computation paths is larger than the number of rejecting computation paths iff

[TABLE]

Membership is thus proved.

Hardness is shown by a reduction from the problem $\#_{1}\mathsf{DNF}(>)$: Decide whether the number of assignments of $\mathbf{X}$ such that the formula $\phi(\mathbf{X})=\forall\mathbf{Y}:\varphi(\mathbf{X},\mathbf{Y})$ holds is strictly larger than $M$, where $\varphi$ is a propositional formula in DNF with conjuncts $d_{1},\ldots,d_{k}$ (and $\mathbf{X}=\{x_{1},\dots,x_{n}\}$ and $\mathbf{Y}=\{y_{1},\dots,y_{m}\}$ are sets of propositional variables). Introduce $\mathsf{x_{i}}$ for each $x_{i}$ and $\mathsf{y_{i}}$ for each $y_{i}$, and encode $\phi$ as follows. Each conjunct $d_{j}$ is represented by a predicate $\mathsf{d_{j}}$ and a rule $\mathsf{d_{j}}\colonminus s_{1},\dots,s_{r}\mbox{\large.}$, where $s_{i}$ stands for a properly encoded subgoal: either some $\mathsf{x_{i}}$, or $\mathbf{not}\;\mathsf{x_{i}}$, or some $\mathsf{y_{i}}$, or $\mathbf{not}\;\mathsf{y_{i}}$. And then introduce $k$ rules $\mathsf{dnf}\colonminus\mathsf{d_{j}}\mbox{\large.}$, one per conjunct. Note that for a fixed truth assignment for all $\mathsf{x_{i}}$ and all $\mathsf{y_{i}}$, $\mathsf{dnf}$ is $\mathsf{true}$ iff $\varphi$ holds. Now introduce probabilistic facts $0.5::\mathsf{x_{i}}$, one for each $\mathsf{x_{i}}$. There are then $2^{n}$ possible ways to select probabilistic facts. The remaining problem is to encode the univeral quantifier over the $y_{i}$. To do so, introduce a pair of rules for each $\mathsf{y_{i}}$,

[TABLE]

Thus there are $2^{m}$ stable models running through assignments of $y_{1},\dots,y_{m}$, for each fixed selection of probabilistic facts. By Expression (5) we have that $\underline{\mathbb{P}}\!\left(\mathsf{dnf}=\mathsf{true}\right)$ is equal to $\sum_{\theta}\min f(\theta)/2^{n}$, where $\theta$ denotes a total choice, the minimum is over all stable models produced by $\mathbf{P}\cup\mathbf{PF}^{\downarrow\theta}$, and $f(\theta)$ is a function that yields $1$ if $\mathsf{dnf}$ is $\mathsf{true}$ and [math] otherwise. Now $\min f(\theta)$ yields $1$ iff for all $\mathbf{Y}$ we have that $\varphi(\mathbf{X},\mathbf{Y})$ is $\mathsf{true}$, where $\mathbf{X}$ is fixed by $\theta$. Hence $\underline{\mathbb{P}}\!\left(\mathsf{dnf}=\mathsf{true}\right)>M/2^{n}$ iff the input problem is accepted. Hardness is thus proved.

Now consider plps where predicates have bounded arity.

Membership follows using the same construction described for the propositional case, but using a $\Sigma_{2}^{P}$ oracle as cautious logical reasoning is $\Pi_{2}^{P}$-complete and brave logical reasoning is $\Sigma_{2}^{P}$-complete (?, Table 5).

Hardness is shown by a reduction from $\#_{2}\mathsf{3CNF}(>)$: Decide whether the number of assignments of $\mathbf{X}$ such that the formula $\phi(\mathbf{X})=\forall\mathbf{Z}:\exists\mathbf{Y}:\varphi(\mathbf{X},\mathbf{Y},\mathbf{Z})$ holds is strictly larger than $M$, where $\varphi$ is a propositional formula in 3CNF with clauses $c_{1},\dots,c_{k}$ (and $\mathbf{X}$, $\mathbf{Y}$, and $\mathbf{Z}$ are sets of propositional variables, and $\mathbf{X}$ contains $n$ propositional variables). We proceed exactly as in the proof of hardness for Theorem 19; each propositional variable $y_{i}$ now appears as a logical variable $Y_{i}$, while each propositional variable $x_{i}$ appears as a predicate $\mathsf{x_{i}}$. The novelty is that each propositional variable $z_{i}$ appears as a predicate $\mathsf{z_{i}}$ that receive the same treatment as predicates $\mathsf{y_{i}}$ in the proof for the propositional case. So, just repeat the whole translation of the formula $\varphi$ used in the proof of Theorem 19, with the only difference that now there may be propositional variables $z_{i}$ in the formula, and these propositional variables appear as predicates $\mathsf{z_{i}}$ in the plp. Then introduce, for each $z_{i}$, a pair of rules

[TABLE]

Again, for each fixed selection of probabilistic facts, there are stable models, one per assignment of $\mathbf{Z}$. And $\underline{\mathbb{P}}\!\left(\mathsf{cnf}=\mathsf{true}\right)>M/2^{n}$ iff the input problem is accepted.

Finally, consider query complexity.

Membership follows using the same construction described for the propositional case, but using a $\mathsf{NP}$ oracle as cautious logical reasoning is $\mathsf{coNP}$-complete and brave logical reasoning is $\mathsf{NP}$-complete (?, Theorem 5.8).

Hardness follows again by a reduction from $\#_{1}\mathsf{DNF}(>)$; that is, again we must decide whether the number of assignments of $\mathbf{X}$ such that $\forall\mathbf{Y}:\varphi(\mathbf{X},\mathbf{Y})$ holds is strictly larger than $M$, where $\varphi$ is a formula in DNF (again, the number of propositional variables in $\mathbf{X}$ is $n$, and the number of propositional variables in $\mathbf{Y}$ is $m$). We employ a construction inspired by the proof of Theorem 21, using the following fixed plp. Note that $\mathsf{x}$ stands for the propositional variables in $\mathbf{X}$, where counting operates; $\mathsf{y}$ stands for the propositional variables in $\mathbf{Y}$, where the universal quantifier operates; $\mathsf{c}$ stands for clauses that are then negated to obtain the DNF:

[TABLE]

By providing $\mathsf{select1}$, $\mathsf{select2}$, $\mathsf{select3}$ and $\mathsf{select4}$ as $\mathbf{Q}$, we can encode the formula $\varphi$. Add to $\mathbf{Q}$ the assignment $\{\mathsf{dnf}=1\}$, and then $\underline{\mathbb{P}}\!\left(\mathbf{Q}\right)>M/2^{4s^{2}+s}$, where $s=\max(m,n)$, iff the input problem is accepted. $\Box$ ∎

Theorem 26 focuses on the computation of lower probabilities. We can of course define the upper-probability versions of inferential and query complexities, by replacing the decision $\underline{\mathbb{P}}\!\left(\mathbf{Q}|\mathbf{E}\right)>\gamma$ with $\overline{\mathbb{P}}\!\left(\mathbf{Q}|\mathbf{E}\right)>\gamma$. If anything, this latter decision leads to easier proofs of membership, for all special cases are dealt with by deciding whether

[TABLE]

where again $\gamma=\mu/\nu$. All other points in the membership proofs remain the same, once brave and cautious reasoning are exchanged. Several arguments concerning hardness can also be easily adapted. For instance, $\mathsf{PP}^{\mathsf{NP}}$-hardness for propositional programs can be proved by reducting from $\#_{1}\mathsf{CNF}(>)$, by encoding a formula in CNF. Similarly, $\mathsf{PP}^{\mathsf{NP}}$-hardness for query complexity reduces from $\#_{1}\mathsf{CNF}(>)$ by using the fixed program described in the proof of Theorem 26 without the latter three rules (and query with assignments on groundings of $\mathsf{c}$).

Theorem 26 does not discuss the complexity of plps, under the credal semantics, without a bound on arity. Without such a bound, logical cautious reasoning is $\mathsf{coNEXP}$-complete, so we conjecture that exponentially bounded counting Turing machines will be needed here. We leave this conjecture as an open question.

Finally, our complexity results were obtained assuming that plps were consistent; of course, in practice one must consider the problem of checking consistency. We have:

Proposition 27**.**

Consistency checking is $\Pi_{2}^{P}$-complete for propositional plps and is $\Pi_{3}^{P}$-complete for plps where predicates have a bound on arity.

Proof.

Membership of consistency checking of a propositional plp obtains by verifying whether logical consistency holds for each total choice of probabilistic facts, and this can be accomplished by deciding whether all total choices satisfy logical consistency (logical consistency checking for this language is $\mathsf{NP}$-complete (?, Table 1)). An analogue reasoning leads to membership in $\Pi_{3}^{P}$ for plps with a bound on arity, as logical consistency checking with bounded arity is $\Sigma^{P}_{2}$-complete (?, Table 4).

Now consider hardness in the propositional case. Take a sentence $\phi$ equal to $\forall\mathbf{X}:\exists\mathbf{Z}:\varphi(\mathbf{X},\mathbf{Z})$, where $\phi$ is a propositional formula in 3CNF with clauses $c_{1},\dots,c_{k}$, and vectors of propositional variables $\mathbf{X}$ and $\mathbf{Z}$. Deciding the satisfiability of such a formula is a $\Pi^{P}_{2}$-complete problem (?). So, introduce a predicate $\mathsf{x_{i}}$ for each $x_{i}$, associated with a probabilistic fact $0.5::\mathsf{x_{i}}\mbox{\large.}$, and a predicate $\mathsf{z_{i}}$ for each $z_{i}$, associated with rules

[TABLE]

Now encode the formula $\phi$ as follows. For each clause $c_{j}$ with three literals, add the rules $\mathsf{c_{j}}\colonminus\ell_{1}\mbox{\large.}$, $\mathsf{c_{j}}\colonminus\ell_{2}\mbox{\large.}$, and $\mathsf{c_{j}}\colonminus\ell_{3}\mbox{\large.}$, where each $\ell_{i}$ stands for a subgoal containing a predicate in $\mathsf{x_{1}},\dots,\mathsf{x_{n}}$ or in $\mathsf{z_{1}},\dots,\mathsf{z_{m}}$, perhaps preceded by $\mathbf{not}$, as appropriate (mimicking a similar construction in the proof of Theorem 19). Then add a rule

[TABLE]

to build the formula $\varphi$, and an additional rule

[TABLE]

to force $\mathsf{cnf}$ to be $\mathsf{true}$ in any stable model. The question of whether this program has a stable model for every configuration of $\mathbf{X}$ then solves the original question about satisfiability of $\phi$.

Finally, consider hardness in the relational (bounded arity) case. Take a sentence $\phi$ equal to $\forall\mathbf{X}:\exists\mathbf{Z}:\neg\exists\mathbf{Y}:\varphi(\mathbf{X},\mathbf{Y},\mathbf{Z})$, where $\varphi$ is a propositional formula in 3CNF; deciding the satisfiability of this formula is a $\Pi^{P}_{3}$-complete problem (?). Denote $\neg\exists\mathbf{Y}:\phi(\mathbf{X},\mathbf{Y},\mathbf{Z})$ by $\phi^{\prime}$. The strategy here will be to combine the constructs in the previous paragraph (propositional case) with the proof of hardness for Theorem 19. That is, introduce a predicate $\mathsf{x_{i}}$ for each $x_{i}$, associated with a probabilistic fact $0.5::\mathsf{x_{i}}\mbox{\large.}$, and a predicate $\mathsf{z_{i}}$ for each $z_{i}$, again associated with rules

[TABLE]

And then encode each clause $c_{j}$ of $\varphi$ by introducing a predicate $\mathsf{c_{j}}(\mathbf{Y}_{j})$, where $\mathbf{Y}_{j}$ is exactly as in the proof of Theorem 19. And as in that proof, introduce

[TABLE]

and force $\phi^{\prime}$ to be $\mathsf{false}$ by introducing:

[TABLE]

The question of whether this program has a stable model for every configuration of $\mathbf{X}$ then solves the original question about satisfiability of $\varphi$. ∎

Again we have left open the complexity of consistency checking for plps without a bound on predicate arity. This question should be addressed in future work.

8 The complexity of inference under the well-founded semantics

In this section we investigate the complexity of probabilistic inference under the well-founded semantics. As before, we examine propositional and relational programs, and within the latter we look at programs with a bound on predicate arity. Note that a bound on predicate arity forces each predicate to have a polynomial number of groundings, but the grounding of the program may still be exponential (as there is no bound on the number of atoms that appear in a single rule, each rule may have many logical variables, thus leading to many groundings).

Theorem 28**.**

Adopt the well-founded semantics for plps. The inferential complexity of plps is $\mathsf{PEXP}$-complete; it is $\mathsf{PP}^{\mathsf{NP}}$-complete if the plp has a bound on the arity of its predicates; it is $\mathsf{PP}$-complete if the plp is propositional. The query complexity of plps is $\mathsf{PP}$-complete.

Proof.

Consider first propositional plps. Such a plp can encode any Bayesian network over binary variables (?), so inference is $\mathsf{PP}$-hard. Membership is proved by adapting the arguments in the proof of Theorem 19; whenever a total choice is selected by the nondeterministic Turing machine, logical inference (under the well-founded semantics) is run with polynomial effort in the resulting propositional normal logic program (?).

Consider now plps with logical variables. Membership follows from the same argument in the previous paragraph, using the fact that inference in normal logic programs under the well-founded semantics is in $\mathsf{EXP}$ (?). Hardness follows from the fact that inferential complexity is $\mathsf{PEXP}$-hard already for acyclic programs (Theorem 20).

Now consider plps with a bound on the arity of predicates. Membership follows from the same argument in the previous paragraphs, using the fact that inference in normal logic programs with a bound on the arity of predicates is under the well-founded semantics is in in $\mathsf{P}^{\mathsf{NP}}$, as proved in Theorem 29. Hardness for $\mathsf{PP}^{\mathsf{NP}}$ follows from the fact that inference complexity of plps under the stable model semantics is $\mathsf{PP}^{\mathsf{NP}}$-hard even for stratified programs, and noting that for stratified programs the stable model and the well-founded semantics agree. ∎

The proof of Theorem 28, in the case of plps with bound on predicate arity, uses the following result. Note that this is a result on logical inference; however it does not seem to be found in current literature.

Theorem 29**.**

Consider the class of normal logic programs with a bound on the arity of predicates, and consider the problem of deciding whether a literal is in the well-founded model of the program. This decision problem is $\mathsf{P}^{\mathsf{NP}}$-complete.

Proof.

Hardness follows from the hardness of logical inference with stratified programs under the stable model semantics (?). Membership requires more work. We use the monotone operator $\mathbb{LFT}_{\mathbf{P}}(\mathbb{LFT}_{\mathbf{P}}(\mathcal{I}))$. Consider the algorithm that constructs the well-founded extension by starting with the empty interpretation and by iterating $\mathbb{LFT}_{\mathbf{P}}(\mathbb{LFT}_{\mathbf{P}}(\mathcal{I}))$. As there are only polynomially-many groundings, there are at most a polynomial number of iterations. Thus in essence we need to iterate the operator $\mathbb{LFT}_{\mathbf{P}}(\mathcal{I})$; thus, focus attention on the computation of $\mathbb{LFT}_{\mathbf{P}}(\mathcal{I})$. The latter computation consists of finding the least fixpoint of $\mathbb{T}_{\mathbf{P}^{\mathcal{I}}}$. So we must focus on the effort involved in computing the least fixpoint of $\mathbb{T}_{\mathbf{P}^{\mathcal{I}}}$. Again, there are at most a polynomial number of iterations of $\mathbb{T}_{\mathbf{P}^{\mathcal{I}}}$ to be run. So, focus on a single iteration of $\mathbb{T}_{\mathbf{P}^{\mathcal{I}}}$. Note that any interpretation $\mathcal{I}$ has polynomial size; however, we cannot explicitly generate the reduct $\mathcal{P}^{\mathcal{I}}$ as it may have exponential size. What we need to do then is, for each grounded atom $A$, to decide whether there is a rule whose grounding makes the atom $A$ $\mathsf{true}$ in $\mathbb{T}_{\mathbf{P}^{\mathcal{I}}}$. So we must make a nondeterministic choice per atom (the choice has the size of logical variables in a rule, a polynomial number). Hence by running a polynomial number of nondeterministic choices, we obtain an iteration of $\mathbb{T}_{\mathbf{P}^{\mathcal{I}}}$; by running a polynomial number of such iterations, we obtain a single iteration of $\mathbb{LFT}_{\mathbf{P}}(\mathcal{I})$; and by running a polynomial number of such iterations, we build the well-founded model. Thus we are within $\mathsf{P}^{\mathsf{NP}}$ as desired. ∎

Obviously, for the well-founded semantics there are no concerns about consistency: every normal logic program has a well-founded semantics, so every plp has one and only one well-founded semantics.

9 Conclusion

We can summarize our contributions as follows. First, we have identified and compared the main ideas between the credal and the well-founded semantics for plps based on probabilistic facts and normal logic programs. Other semantics may be studied in future work, but the credal and the well-founded ones seem to be the most important starting point. Second, we have shown that the credal semantis is intimately related to infinitely monotone Choquet capacitites; precisely: the credal semantics of a consistent plp is a credal set that dominates a infinitely monotone Choquet capacity. Third, we have derived the inferential and query complexity of acyclic, stratified and general plps both under the credal and the well-founded semantics. These results on complexity are summarized in Table 2; note that plps reach non-trivial classes in the counting hierarchy. It is interesting to note that acyclic plps with a bound on arity go beyond Bayesian networks in the complexity classes they can reach.

Concerning complexity, acyclic and stratified plps have identical credal and well-founded semantics, while general plps may have different credal and well-founded semantics. For normal logic programs (not probabilistic ones), the well-founded semantics is known to stay within lower complexity classes than the credal semantics (?); the same phenomenon persists in the probabilistic case. Indeed, the well-founded semantics for general plps reaches the same complexity classes as for acyclic plps. One might take this as an argument for the well-founded semantics, on top of the fact that the well-founded semantics is defined for any plp. On the other hand, our analysis in Section 4 favors, at least conceptually, the credal semantics, despite the fact that it may not be defined for some plps (in fact one might argue that no semantics should be defined for such plps). It is much easier to understand the meaning of plps using the credal semantics than the well-founded semantics, as the latter mixes three-valued logic and probabilities in a non-trivial way. We suggest that more study is needed to isolate those programs where $\mathsf{undefined}$ values are justified and can be properly mixed with probabilities. Also, the well-founded semantics may be taken as an approximation of the set of possible probability models. In any case, we find that Lukasiewicz’s credal semantics is quite attractive and not as well known as it deserves to be.

We could include in the analysis of plps a number of useful constructs that have been adopted in answer set programming (?). There, classic negation, such as $\neg\mathsf{wins}(X)$, is allowed on top of $\mathbf{not}$. Also, constraints, such as $\colonminus\phi$, are allowed to mean that $\phi$ is $\mathsf{false}$. More substantial is the presence, in answer set programming, of disjunctive heads. With such a machinery, we can for instance rewrite the rules in Example 9 as a single rule $\mathsf{single}(X)\vee\mathsf{husband}(X)\colonminus\mathsf{man}(X).$, and the rules in Example 11 as the pair:

[TABLE]

Now the point to be made is this. Suppose we have a probabilistic logic program $\left<\mathbf{P},\mathbf{PF}\right>$, where as before we have independent probabilistic facts, but where $\mathbf{P}$ is now a logic program with classic negation, constraints, disjuctive heads, and $\mathbf{P}$ is consistent in that it has stable models for every total choice of probabilistic facts. The proof of Theorem 14 can be reproduced in this setting, and hence the credal semantics (the set of measures over stable models) of these probabilistic answer set programs is again an infinite monotone credal set. The complexity of inference with these constructs is left for future investigation.

Much more is yet to be explored concerning the complexity of plps. Several classes of plps deserve attention, such as definite, tight, strict, order-consistent programs, and programs with aggregates and other constructs. The inclusion of functions (with appropriate restrictions to ensure decidability) is another challenge. Concerning complexity theory itself, it seems that approximability should be investigated, as well as questions surrounding learnability and expressivity of plps.

Acknowledgements

The first author is partially supported by CNPq, grant 308433/2014-9. The second author received financial support from the São Paulo Research Foundation (FAPESP), grant 2016/01055-1.