What is the Relationship between Tensor Factorizations and Circuits (and How Can We Exploit it)?

TL;DR

This paper establishes a rigorous connection between circuit representations and tensor factorizations, unifying and generalizing models, and introduces a modular framework for building and optimizing tensorized circuit architectures with promising empirical results.

Contribution

It generalizes tensor factorizations within circuit language and unifies various circuit learning algorithms under a hierarchical framework.

Findings

Framework effectively constructs diverse tensorized circuit architectures.

Empirical evaluations demonstrate the framework's effectiveness.

Highlights new research opportunities in probabilistic modeling.

Abstract

This paper establishes a rigorous connection between circuit representations and tensor factorizations, two seemingly distinct yet fundamentally related areas. By connecting these fields, we highlight a series of opportunities that can benefit both communities. Our work generalizes popular tensor factorizations within the circuit language, and unifies various circuit learning algorithms under a single, generalized hierarchical factorization framework. Specifically, we introduce a modular "Lego block" approach to build tensorized circuit architectures. This, in turn, allows us to systematically construct and explore various circuit and tensor factorization models while maintaining tractability. This connection not only clarifies similarities and differences in existing models, but also enables the development of a comprehensive pipeline for building and optimizing new circuit/tensor factorization architectures. We show the effectiveness of our framework through extensive empirical evaluations, and highlight new research opportunities for tensor factorizations in probabilistic modeling.