RPCANet++: Deep Interpretable Robust PCA for Sparse Object Segmentation

TL;DR

RPCANet++ is a novel deep learning framework that combines the interpretability of RPCA with efficient neural network architectures to improve sparse object segmentation across diverse imaging scenarios.

Contribution

It introduces a structured network that unfolds a relaxed RPCA model, incorporating modules for background approximation, object extraction, and image restoration, with enhancements for interpretability and efficiency.

Findings

Achieves state-of-the-art segmentation performance.

Enhances interpretability with low-rankness and sparsity measurements.

Demonstrates robustness across various datasets and imaging conditions.

Abstract

Robust principal component analysis (RPCA) decomposes an observation matrix into low-rank background and sparse object components. This capability has enabled its application in tasks ranging from image restoration to segmentation. However, traditional RPCA models suffer from computational burdens caused by matrix operations, reliance on finely tuned hyperparameters, and rigid priors that limit adaptability in dynamic scenarios. To solve these limitations, we propose RPCANet++, a sparse object segmentation framework that fuses the interpretability of RPCA with efficient deep architectures. Our approach unfolds a relaxed RPCA model into a structured network comprising a Background Approximation Module (BAM), an Object Extraction Module (OEM), and an Image Restoration Module (IRM). To mitigate inter-stage transmission loss in the BAM, we introduce a Memory-Augmented Module (MAM) to enhance background feature preservation, while a Deep Contrast Prior Module (DCPM) leverages saliency cues to expedite object extraction. Extensive experiments on diverse datasets demonstrate that RPCANet++ achieves state-of-the-art performance under various imaging scenarios. We further improve interpretability via visual and numerical low-rankness and sparsity measurements. By combining the theoretical strengths of RPCA with the efficiency of deep networks, our approach sets a new baseline for reliable and interpretable sparse object segmentation. Codes are available at our Project Webpage https://fengyiwu98.github.io/rpcanetx.