Deep Compression of Sum-Product Networks on Tensor Networks

TL;DR

This paper introduces tensor SPNs, a highly efficient representation of sum-product networks that achieves significant parameter compression with minimal accuracy loss by leveraging their connection to tensor networks.

Contribution

It establishes a novel connection between SPNs and tensor networks and develops optimization techniques for effective compression of SPNs.

Findings

Significant parameter reduction achieved

Negligible accuracy loss demonstrated

Efficient inference with compressed models

Abstract

Sum-product networks (SPNs) represent an emerging class of neural networks with clear probabilistic semantics and superior inference speed over graphical models. This work reveals a strikingly intimate connection between SPNs and tensor networks, thus leading to a highly efficient representation that we call tensor SPNs (tSPNs). For the first time, through mapping an SPN onto a tSPN and employing novel optimization techniques, we demonstrate remarkable parameter compression with negligible loss in accuracy.