Tucker Diffusion Model for High-dimensional Tensor Generation

TL;DR

This paper introduces a Tucker diffusion model for high-dimensional tensor generation, leveraging low Tucker rank structure for efficient and accurate distribution learning, with theoretical guarantees and empirical success.

Contribution

It proposes a novel Tucker diffusion framework that decomposes the score function for tensor data, enabling efficient high-dimensional tensor generation with theoretical convergence guarantees.

Findings

The score function admits a structured decomposition under low Tucker rank.

The distribution of generated tensors converges at a rate depending on maximum tensor mode dimensions.

The model achieves comparable or superior performance with reduced training and sampling costs.

Abstract

Statistical inference on large-dimensional tensor data has been extensively studied in the literature and widely used in economics, biology, machine learning, and other fields, but how to generate a structured tensor with a target distribution is still a new problem. As profound AI generators, diffusion models have achieved remarkable success in learning complex distributions. However, their extension to generating multi-linear tensor-valued observations remains underexplored. In this work, we propose a novel Tucker diffusion model for learning high-dimensional tensor distributions. We show that the score function admits a structured decomposition under the low Tucker rank assumption, allowing it to be both accurately approximated and efficiently estimated using a carefully tailored tensor-shaped architecture named Tucker-Unet. Furthermore, the distribution of generated tensors, induced by the estimated score function, converges to the true data distribution at a rate depending on the maximum of tensor mode dimensions, thereby offering a clear theoretical advantage over the naive vectorized approach, which has a product dependence. Empirically, compared to existing approaches, the Tucker diffusion model demonstrates strong practical potential in synthetic and real-world tensor generation tasks, achieving comparable and sometimes even superior statistical performance with significantly reduced training and sampling costs.