How to Square Tensor Networks and Circuits Without Squaring Them

TL;DR

This paper introduces a novel parameterization of squared tensor networks and circuits that simplifies marginalization, enabling efficient distribution estimation without losing expressiveness.

Contribution

It proposes a new way to parameterize squared circuits inspired by canonical forms, reducing computational complexity in marginalization tasks.

Findings

Efficient marginalization achieved in squared circuits.

No loss of expressiveness in distribution estimation.

Improved learning efficiency demonstrated in experiments.

Abstract

Squared tensor networks (TNs) and their extension as computational graphs--squared circuits--have been used as expressive distribution estimators, yet supporting closed-form marginalization. However, the squaring operation introduces additional complexity when computing the partition function or marginalizing variables, which hinders their applicability in ML. To solve this issue, canonical forms of TNs are parameterized via unitary matrices to simplify the computation of marginals. However, these canonical forms do not apply to circuits, as they can represent factorizations that do not directly map to a known TN. Inspired by the ideas of orthogonality in canonical forms and determinism in circuits enabling tractable maximization, we show how to parameterize squared circuits to overcome their marginalization overhead. Our parameterizations unlock efficient marginalization even in factorizations different from TNs, but encoded as circuits, whose structure would otherwise make marginalization computationally hard. Finally, our experiments on distribution estimation show how our proposed conditions in squared circuits come with no expressiveness loss, while enabling more efficient learning.