Learning Logistic Circuits

TL;DR

This paper introduces logistic circuits, a new classification model that outperforms larger neural networks on MNIST and Fashion datasets, combining symbolic AI origins with convex parameter learning.

Contribution

It presents logistic circuits as a novel discriminative model with convex parameter learning and a local search algorithm for structure induction.

Findings

Outperforms neural networks with more parameters on MNIST and Fashion datasets

Parameter learning is convex optimization

Simple local search induces effective model structures

Abstract

This paper proposes a new classification model called logistic circuits. On MNIST and Fashion datasets, our learning algorithm outperforms neural networks that have an order of magnitude more parameters. Yet, logistic circuits have a distinct origin in symbolic AI, forming a discriminative counterpart to probabilistic-logical circuits such as ACs, SPNs, and PSDDs. We show that parameter learning for logistic circuits is convex optimization, and that a simple local search algorithm can induce strong model structures from data.