A System for Acquiring, Processing, and Rendering Panoramic Light Field Stills for Virtual Reality

TL;DR

This paper introduces a comprehensive system for capturing, processing, and rendering panoramic light field images for VR, featuring novel hardware, real-time algorithms, and efficient compression, enabling high-quality, accessible VR experiences.

Contribution

The work presents new light field camera rigs, a real-time reconstruction algorithm, a light field prefiltering method, and a modified VP9 codec for efficient compression, all integrated into a complete VR system.

Findings

Achieved 90Hz stereo rendering on commodity VR hardware.

Developed a portable, efficient light field acquisition system.

Created a publicly available VR light field application with over 15,000 downloads.

Abstract

We present a system for acquiring, processing, and rendering panoramic light field still photography for display in Virtual Reality (VR). We acquire spherical light field datasets with two novel light field camera rigs designed for portable and efficient light field acquisition. We introduce a novel real-time light field reconstruction algorithm that uses a per-view geometry and a disk-based blending field. We also demonstrate how to use a light field prefiltering operation to project from a high-quality offline reconstruction model into our real-time model while suppressing artifacts. We introduce a practical approach for compressing light fields by modifying the VP9 video codec to provide high quality compression with real-time, random access decompression. We combine these components into a complete light field system offering convenient acquisition, compact file size, and high-quality rendering while generating stereo views at 90Hz on commodity VR hardware. Using our system, we built a freely available light field experience application called Welcome to Light Fields featuring a library of panoramic light field stills for consumer VR which has been downloaded over 15,000 times.