A Novel Light Field Coding Scheme Based on Deep Belief Network & Weighted Binary Images for Additive Layered Displays

TL;DR

This paper introduces a new light field coding framework using Deep Belief Networks and weighted binary images, enabling scalable, high-quality light field representation for augmented reality displays.

Contribution

It presents a novel light field coding scheme combining DBN and weighted binary images, optimized for additive layered AR displays with adaptive bitrate capabilities.

Findings

Effective capture of light field redundancies

Achieves scalable bitrate encoding with H.265

Validated on real and synthetic datasets

Abstract

Light-field displays create an immersive experience by providing binocular depth sensation and motion parallax. Stacking light attenuating layers is one approach to implement a light field display with a broader depth of field, wide viewing angles and high resolution. Due to the transparent holographic optical element (HOE) layers, additive layered displays can be integrated into augmented reality (AR) wearables to overlay virtual objects onto the real world, creating a seamless mixed reality (XR) experience. This paper proposes a novel framework for light field representation and coding that utilizes Deep Belief Network (DBN) and weighted binary images suitable for additive layered displays. The weighted binary representation of layers makes the framework more flexible for adaptive bitrate encoding. The framework effectively captures intrinsic redundancies in the light field data, and thus provides a scalable solution for light field coding suitable for XR display applications. The latent code is encoded by H.265 codec generating a rate-scalable bit-stream. We achieve adaptive bitrate decoding by varying the number of weighted binary images and the H.265 quantization parameter, while maintaining an optimal reconstruction quality. The framework is tested on real and synthetic benchmark datasets, and the results validate the rate-scalable property of the proposed scheme.