The Gradient of Algebraic Model Counting

TL;DR

This paper extends algebraic model counting to learning by generalizing gradients and backpropagation across semirings, enabling more memory-efficient algorithms and faster optimization in probabilistic logical models.

Contribution

It introduces a unified framework for learning using algebraic model counting with semirings, generalizing gradients and backpropagation, and explores efficiency improvements.

Findings

Algebraic backpropagation achieves significant speed-ups.

Semiring properties can optimize memory usage.

Unified approach applies to various probabilistic logical models.

Abstract

Algebraic model counting unifies many inference tasks on logic formulas by exploiting semirings. Rather than focusing on inference, we consider learning, especially in statistical-relational and neurosymbolic AI, which combine logical, probabilistic and neural representations. Concretely, we show that the very same semiring perspective of algebraic model counting also applies to learning. This allows us to unify various learning algorithms by generalizing gradients and backpropagation to different semirings. Furthermore, we show how cancellation and ordering properties of a semiring can be exploited for more memory-efficient backpropagation. This allows us to obtain some interesting variations of state-of-the-art gradient-based optimisation methods for probabilistic logical models. We also discuss why algebraic model counting on tractable circuits does not lead to more efficient second-order optimization. Empirically, our algebraic backpropagation exhibits considerable speed-ups as compared to existing approaches.