Superpixel-informed Continuous Low-Rank Tensor Representation for Multi-Dimensional Data Recovery

TL;DR

This paper introduces a superpixel-informed continuous low-rank tensor representation (SCTR) that models multi-dimensional data more flexibly and accurately by leveraging superpixels and neural network parameterization, outperforming traditional methods.

Contribution

The paper proposes a novel SCTR framework that uses superpixels and asymmetric low-rank tensor factorization with neural networks to improve data modeling beyond grid constraints.

Findings

Achieves 3-5 dB PSNR improvements over existing methods.

Effectively captures semantic regions with superpixels.

Demonstrates versatility across multispectral images, videos, and color images.

Abstract

Low-rank tensor representation (LRTR) has emerged as a powerful tool for multi-dimensional data processing. However, classical LRTR-based methods face two critical limitations: (1) they typically assume that the holistic data is low-rank, this assumption is often violated in real-world scenarios with significant spatial variations; and (2) they are constrained to discrete meshgrid data, limiting their flexibility and applicability. To overcome these limitations, we propose a Superpixel-informed Continuous low-rank Tensor Representation (SCTR) framework, which enables continuous and flexible modeling of multi-dimensional data beyond traditional grid-based constraints. Our approach introduces two main innovations: First, motivated by the observation that semantically coherent regions exhibit stronger low-rank characteristics than holistic data, we employ superpixels as the basic modeling units. This design not only encodes rich semantic information, but also enhances adaptability to diverse forms of data streams. Second, we propose a novel asymmetric low-rank tensor factorization (ALTF) where superpixel-specific factor matrices are parameterized by a shared neural network with specialized heads. By strategically separating global pattern learning from local adaptation, this framework efficiently captures both cross-superpixel commonalities and within-superpixel variations. This yields a representation that is both highly expressive and compact, balancing model efficiency with adaptability. Extensive experiments on several benchmark datasets demonstrate that SCTR achieves 3-5 dB PSNR improvements over existing LRTR-based methods across multispectral images, videos, and color images.