Tensor Network Structure Search with Program Synthesis

TL;DR

This paper introduces a novel program synthesis approach to tensor network structure search, significantly improving efficiency and compression performance while enabling scaling to larger tensors.

Contribution

It formulates tensor network structure search as a program synthesis problem, proposing a constraint-based assessment method and new operations to efficiently find optimal structures.

Findings

Search speed improved by up to 10x

Achieves 1.5x to 3x better compression ratios than state-of-the-art

Scales to larger tensors unattainable by prior methods

Abstract

Tensor networks provide a powerful framework for compressing multi-dimensional data. The optimal tensor network structure for a given data tensor depends on both data characteristics and specific optimality criteria, making tensor network structure search a difficult problem. Existing solutions typically rely on sampling and compressing numerous candidate structures; these procedures are computationally expensive and therefore limiting for practical applications. We address this challenge by viewing tensor network structure search as a program synthesis problem and introducing an efficient constraint-based assessment method that avoids costly tensor decomposition. Specifically, we establish a correspondence between transformation programs and network structures. We also design a novel operation named output-directed splits to reduce the search space without hindering expressiveness. We then propose a synthesis algorithm to identify promising network candidates through constraint solving, and avoid tensor decomposition for all but the most promising candidates. Experimental results show that our approach improves search speed by up to $10\times$ and achieves compression ratios $1.5\times$ to $3\times$ better than state-of-the-art. Notably, our approach scales to larger tensors that are unattainable by prior work. Furthermore, the discovered topologies generalize well to similar data, yielding compression ratios up to $ 2.4\times$ better than a generic structure while the runtime remains around $110$ seconds.