MiNL: Micro-images based Neural Representation for Light Fields

TL;DR

MiNL introduces a compact and efficient neural representation for light fields using micro-images, achieving faster decoding and better visual quality compared to traditional pixel-wise implicit methods.

Contribution

The paper proposes MiNL, a novel MI-wise implicit neural representation that improves efficiency and quality over existing pixel-wise methods by combining MLP and CNN.

Findings

80-180 times faster decoding speed

1-4 dB PSNR improvement on average

More compact representation with better visual quality

Abstract

Traditional representations for light fields can be separated into two types: explicit representation and implicit representation. Unlike explicit representation that represents light fields as Sub-Aperture Images (SAIs) based arrays or Micro-Images (MIs) based lenslet images, implicit representation treats light fields as neural networks, which is inherently a continuous representation in contrast to discrete explicit representation. However, at present almost all the implicit representations for light fields utilize SAIs to train an MLP to learn a pixel-wise mapping from 4D spatial-angular coordinate to pixel colors, which is neither compact nor of low complexity. Instead, in this paper we propose MiNL, a novel MI-wise implicit neural representation for light fields that train an MLP + CNN to learn a mapping from 2D MI coordinates to MI colors. Given the micro-image's coordinate, MiNL outputs the corresponding micro-image's RGB values. Light field encoding in MiNL is just training a neural network to regress the micro-images and the decoding process is a simple feedforward operation. Compared with common pixel-wise implicit representation, MiNL is more compact and efficient that has faster decoding speed (\textbf{$\times$80$\sim$180} speed-up) as well as better visual quality (\textbf{1$\sim$4dB} PSNR improvement on average).