On Constraint Definability in Tractable Probabilistic Models

TL;DR

This paper investigates the mathematical foundations of incorporating various constraints into tractable probabilistic models like sum-product networks, addressing a significant open problem in probabilistic machine learning.

Contribution

It provides a theoretical analysis of how constraints can be integrated into the learning process of tractable probabilistic models.

Findings

Identifies the conditions under which constraints can be incorporated.

Provides a framework for understanding constraint definability.

Highlights open challenges in constraint integration.

Abstract

Incorporating constraints is a major concern in probabilistic machine learning. A wide variety of problems require predictions to be integrated with reasoning about constraints, from modelling routes on maps to approving loan predictions. In the former, we may require the prediction model to respect the presence of physical paths between the nodes on the map, and in the latter, we may require that the prediction model respect fairness constraints that ensure that outcomes are not subject to bias. Broadly speaking, constraints may be probabilistic, logical or causal, but the overarching challenge is to determine if and how a model can be learnt that handles all the declared constraints. To the best of our knowledge, this is largely an open problem. In this paper, we consider a mathematical inquiry on how the learning of tractable probabilistic models, such as sum-product networks, is possible while incorporating constraints.