High-Order Residual Network for Light Field Super-Resolution

TL;DR

This paper introduces a high-order residual network that effectively learns geometric features from light field data for super-resolution, improving reconstruction quality especially near occlusions and non-Lambertian regions.

Contribution

The proposed high-order residual network leverages the structural properties of light field data with novel residual blocks for hierarchical geometric feature learning.

Findings

Outperforms state-of-the-art methods in quantitative metrics

Reduces artifacts near occlusion and non-Lambertian regions

Enhances spatial details through a refinement network

Abstract

Plenoptic cameras usually sacrifice the spatial resolution of their SAIs to acquire geometry information from different viewpoints. Several methods have been proposed to mitigate such spatio-angular trade-off, but seldom make use of the structural properties of the light field (LF) data efficiently. In this paper, we propose a novel high-order residual network to learn the geometric features hierarchically from the LF for reconstruction. An important component in the proposed network is the high-order residual block (HRB), which learns the local geometric features by considering the information from all input views. After fully obtaining the local features learned from each HRB, our model extracts the representative geometric features for spatio-angular upsampling through the global residual learning. Additionally, a refinement network is followed to further enhance the spatial details by minimizing a perceptual loss. Compared with previous work, our model is tailored to the rich structure inherent in the LF, and therefore can reduce the artifacts near non-Lambertian and occlusion regions. Experimental results show that our approach enables high-quality reconstruction even in challenging regions and outperforms state-of-the-art single image or LF reconstruction methods with both quantitative measurements and visual evaluation.