Light Field Image Coding Using VVC standard and View Synthesis based on Dual Discriminator GAN

TL;DR

This paper introduces a novel light field image coding method that combines VVC standard encoding, view synthesis with a dual discriminator GAN, and quality enhancement to achieve superior compression efficiency for VR content.

Contribution

It proposes a new LF image coding approach that transmits only key views, synthesizes others with a dual discriminator GAN, and enhances quality for smooth navigation, outperforming existing methods.

Findings

Achieves -36.22% BD-BR reduction compared to state-of-the-art.

Improves BD-PSNR by 1.35 dB over previous methods.

Demonstrates effective view synthesis and quality enhancement for VR applications.

Abstract

Light field (LF) technology is considered as a promising way for providing a high-quality virtual reality (VR) content. However, such an imaging technology produces a large amount of data requiring efficient LF image compression solutions. In this paper, we propose a LF image coding method based on a view synthesis and view quality enhancement techniques. Instead of transmitting all the LF views, only a sparse set of reference views are encoded and transmitted, while the remaining views are synthesized at the decoder side. The transmitted views are encoded using the versatile video coding (VVC) standard and are used as reference views to synthesize the dropped views. The selection of non-reference dropped views is performed using a rate-distortion optimization based on the VVC temporal scalability. The dropped views are reconstructed using the LF dual discriminator GAN (LF-D2GAN) model. In addition, to ensure that the quality of the views is consistent, at the decoder, a quality enhancement procedure is performed on the reconstructed views allowing smooth navigation across views. Experimental results show that the proposed method provides high coding performance and overcomes the state-of-the-art LF image compression methods by -36.22% in terms of BD-BR and 1.35 dB in BD-PSNR. The web page of this work is available at https://naderbakir79.github.io/LFD2GAN.html.