The Theory and Practice of MAP Inference over Non-Convex Constraints

TL;DR

This paper addresses the challenge of performing MAP inference under non-convex constraints in probabilistic ML, proposing scalable algorithms that outperform existing methods on synthetic and real-world benchmarks.

Contribution

It introduces conditions for exact, efficient constrained MAP inference and develops scalable message-passing and domain partitioning algorithms for non-convex constraints.

Findings

Algorithms outperform constraint-agnostic baselines.

Methods scale to complex, intractable densities.

Approaches are validated on synthetic and real-world benchmarks.

Abstract

In many safety-critical settings, probabilistic ML systems have to make predictions subject to algebraic constraints, e.g., predicting the most likely trajectory that does not cross obstacles. These real-world constraints are rarely convex, nor the densities considered are (log-)concave. This makes computing this constrained maximum a posteriori (MAP) prediction efficiently and reliably extremely challenging. In this paper, we first investigate under which conditions we can perform constrained MAP inference over continuous variables exactly and efficiently and devise a scalable message-passing algorithm for this tractable fragment. Then, we devise a general constrained MAP strategy that interleaves partitioning the domain into convex feasible regions with numerical constrained optimization. We evaluate both methods on synthetic and real-world benchmarks, showing our approaches outperform constraint-agnostic baselines, and scale to complex densities intractable for SoTA exact solvers.