Low-Rank Tensor Recovery via Variational Schatten-p Quasi-Norm and Jacobian Regularization

TL;DR

This paper introduces a novel low-rank tensor recovery method using a variational Schatten-p quasi-norm and Jacobian regularization, leveraging neural networks for improved interpretability and performance in multi-dimensional data tasks.

Contribution

It proposes a CP-based neural tensor model with a variational Schatten-p quasi-norm for sparsity and a Jacobian spectral norm regularization for smoothness, with theoretical guarantees and practical effectiveness.

Findings

Outperforms state-of-the-art in image inpainting and denoising

Effective in point cloud upsampling tasks

Provides theoretical bounds and practical algorithms

Abstract

Higher-order tensors are well-suited for representing multi-dimensional data, such as images and videos, which typically characterize low-rank structures. Low-rank tensor decomposition has become essential in machine learning and computer vision, but existing methods like Tucker decomposition offer flexibility at the expense of interpretability. The CANDECOMP/PARAFAC (CP) decomposition provides a natural and interpretable structure, while obtaining a sparse solutions remains challenging. Leveraging the rich properties of CP decomposition, we propose a CP-based low-rank tensor function parameterized by neural networks (NN) for implicit neural representation. This approach can model the tensor both on-grid and beyond grid, fully utilizing the non-linearity of NN with theoretical guarantees on excess risk bounds. To achieve sparser CP decomposition, we introduce a variational Schatten-p quasi-norm to prune redundant rank-1 components and prove that it serves as a common upper bound for the Schatten-p quasi-norms of arbitrary unfolding matrices. For smoothness, we propose a regularization term based on the spectral norm of the Jacobian and Hutchinson's trace estimator. The proposed smoothness regularization is SVD-free and avoids explicit chain rule derivations. It can serve as an alternative to Total Variation (TV) regularization in image denoising tasks and is naturally applicable to implicit neural representation. Extensive experiments on multi-dimensional data recovery tasks, including image inpainting, denoising, and point cloud upsampling, demonstrate the superiority and versatility of our method compared to state-of-the-art approaches. The code is available at https://github.com/CZY-Code/CP-Pruner.