Enhanced Self-Supervised Multi-Image Super-Resolution for Camera Array Images

TL;DR

This paper introduces a novel self-supervised multi-image super-resolution framework tailored for camera array images, leveraging a dual Transformer network and combining Multi-to-Single and Multi-to-Multi SSL methods for enhanced detail recovery.

Contribution

It proposes a new Multi-to-Single-Guided Multi-to-Multi SSL framework and a dual Transformer network specifically designed for camera array super-resolution tasks.

Findings

Outperforms existing methods on synthetic and real datasets.

Effectively recovers high-frequency details and textures.

Demonstrates robustness against complex degradations and occlusions.

Abstract

Conventional multi-image super-resolution (MISR) methods, such as burst and video SR, rely on sequential frames from a single camera. Consequently, they suffer from complex image degradation and severe occlusion, increasing the difficulty of accurate image restoration. In contrast, multi-aperture camera-array imaging captures spatially distributed views with sampling offsets forming a stable disk-like distribution, which enhances the non-redundancy of observed data. Existing MISR algorithms fail to fully exploit these unique properties. Supervised MISR methods tend to overfit the degradation patterns in training data, and current self-supervised learning (SSL) techniques struggle to recover fine-grained details. To address these issues, this paper thoroughly investigates the strengths, limitations and applicability boundaries of multi-image-to-single-image (Multi-to-Single) and multi-image-to-multi-image (Multi-to-Multi) SSL methods. We propose the Multi-to-Single-Guided Multi-to-Multi SSL framework that combines the advantages of Multi-to-Single and Multi-to-Multi to generate visually appealing and high-fidelity images rich in texture details. The Multi-to-Single-Guided Multi-to-Multi SSL framework provides a new paradigm for integrating deep neural network with classical physics-based variational methods. To enhance the ability of MISR network to recover high-frequency details from aliased artifacts, this paper proposes a novel camera-array SR network called dual Transformer suitable for SSL. Experiments on synthetic and real-world datasets demonstrate the superiority of the proposed method.