Sub-Aperture Feature Adaptation in Single Image Super-resolution Model for Light Field Imaging

TL;DR

This paper introduces an adaptation module for pretrained SISR networks to enhance light field image super-resolution, leveraging existing models to improve resolution without domain-specific engineering.

Contribution

It proposes a novel adaptation module with Sub aperture Shift and fusion blocks that improves super-resolution performance on light field images using pretrained SISR models.

Findings

Outperforms existing light field super-resolution algorithms.

Achieves PSNR gains of over 1 dB at scale factor 2.

Improves PSNR by 0.6 to 1 dB at scale factor 4.

Abstract

With the availability of commercial Light Field (LF) cameras, LF imaging has emerged as an up and coming technology in computational photography. However, the spatial resolution is significantly constrained in commercial microlens based LF cameras because of the inherent multiplexing of spatial and angular information. Therefore, it becomes the main bottleneck for other applications of light field cameras. This paper proposes an adaptation module in a pretrained Single Image Super Resolution (SISR) network to leverage the powerful SISR model instead of using highly engineered light field imaging domain specific Super Resolution models. The adaption module consists of a Sub aperture Shift block and a fusion block. It is an adaptation in the SISR network to further exploit the spatial and angular information in LF images to improve the super resolution performance. Experimental validation shows that the proposed method outperforms existing light field super resolution algorithms. It also achieves PSNR gains of more than 1 dB across all the datasets as compared to the same pretrained SISR models for scale factor 2, and PSNR gains 0.6 to 1 dB for scale factor 4.