3DeepRep: 3D Deep Low-rank Tensor Representation for Hyperspectral Image Inpainting

TL;DR

This paper introduces 3DeepRep, a novel 3-directional deep low-rank tensor model for hyperspectral image inpainting that leverages low-rank structures along all tensor modes, improving over existing spectral-only methods.

Contribution

The paper proposes a new 3-directional deep low-rank tensor representation with a learnable fusion and an efficient optimization algorithm for hyperspectral image inpainting.

Findings

Outperforms state-of-the-art inpainting methods on real datasets.

Effectively captures low-rank structures along all tensor modes.

Demonstrates superior qualitative and quantitative results.

Abstract

Recent approaches based on transform-based tensor nuclear norm (TNN) have demonstrated notable effectiveness in hyperspectral image (HSI) inpainting by leveraging low-rank structures in latent representations. Recent developments incorporate deep transforms to improve low-rank tensor representation; however, existing approaches typically restrict the transform to the spectral mode, neglecting low-rank properties along other tensor modes. In this paper, we propose a novel 3-directional deep low-rank tensor representation (3DeepRep) model, which performs deep nonlinear transforms along all three modes of the HSI tensor. To enforce low-rankness, the model minimizes the nuclear norms of mode-i frontal slices in the corresponding latent space for each direction (i=1,2,3), forming a 3-directional TNN regularization. The outputs from the three directional branches are subsequently fused via a learnable aggregation module to produce the final result. An efficient gradient-based optimization algorithm is developed to solve the model in a self-supervised manner. Extensive experiments on real-world HSI datasets demonstrate that the proposed method achieves superior inpainting performance compared to existing state-of-the-art techniques, both qualitatively and quantitatively.