Score-Based Model for Low-Rank Tensor Recovery

TL;DR

This paper introduces a score-based neural network model for low-rank tensor recovery that avoids predefined structural assumptions, improving accuracy in tensor completion and denoising tasks across diverse data types.

Contribution

It proposes a novel score-based approach that learns tensor-factor compatibility without relying on fixed structural assumptions, enhancing flexibility and performance.

Findings

Significant performance improvements in tensor completion and denoising

Effective handling of sparse and continuous-time tensors

Outperforms traditional methods in various tensor recovery tasks

Abstract

Low-rank tensor decompositions (TDs) provide an effective framework for multiway data analysis. Traditional TD methods rely on predefined structural assumptions, such as CP or Tucker decompositions. From a probabilistic perspective, these can be viewed as using Dirac delta distributions to model the relationships between shared factors and the low-rank tensor. However, such prior knowledge is rarely available in practical scenarios, particularly regarding the optimal rank structure and contraction rules. The optimization procedures based on fixed contraction rules are complex, and approximations made during these processes often lead to accuracy loss. To address this issue, we propose a score-based model that eliminates the need for predefined structural or distributional assumptions, enabling the learning of compatibility between tensors and shared factors. Specifically, a neural network is designed to learn the energy function, which is optimized via score matching to capture the gradient of the joint log-probability of tensor entries and shared factors. Our method allows for modeling structures and distributions beyond the Dirac delta assumption. Moreover, integrating the block coordinate descent (BCD) algorithm with the proposed smooth regularization enables the model to perform both tensor completion and denoising. Experimental results demonstrate significant performance improvements across various tensor types, including sparse and continuous-time tensors, as well as visual data.