Light Field Compression by Residual CNN Assisted JPEG

TL;DR

This paper introduces a fast, learning-based light field compression method that leverages JPEG and neural networks to efficiently reconstruct high-quality light fields with minimal compression time and high structural similarity.

Contribution

It proposes a novel JPEG-assisted neural network pipeline for light field compression and reconstruction, significantly improving speed while maintaining quality compared to traditional video compression methods.

Findings

Achieves 18x faster decompression than video-based methods.

Maintains high SSIM and comparable PSNR at low bitrates.

Reconstructs light fields with minimal compression time cost.

Abstract

Light field (LF) imaging has gained significant attention due to its recent success in 3-dimensional (3D) displaying and rendering as well as augmented and virtual reality usage. Nonetheless, because of the two extra dimensions, LFs are much larger than conventional images. We develop a JPEG-assisted learning-based technique to reconstruct an LF from a JPEG bitstream with a bit per pixel ratio of 0.0047 on average. For compression, we keep the LF's center view and use JPEG compression with 50% quality. Our reconstruction pipeline consists of a small JPEG enhancement network (JPEG-Hance), a depth estimation network (Depth-Net), followed by view synthesizing by warping the enhanced center view. Our pipeline is significantly faster than using video compression on pseudo-sequences extracted from an LF, both in compression and decompression, while maintaining effective performance. We show that with a 1% compression time cost and 18x speedup for decompression, our methods reconstructed LFs have better structural similarity index metric (SSIM) and comparable peak signal-to-noise ratio (PSNR) compared to the state-of-the-art video compression techniques used to compress LFs.