Generalized Scene Reconstruction

TL;DR

The paper introduces Generalized Scene Reconstruction (GSR), a passive method for reconstructing complex, boundless scenes with diverse materials using a novel plenoptic octree data structure, demonstrated on highly reflective surfaces.

Contribution

It presents GSR and the plenoptic octree data structure, enabling efficient reconstruction of complex scenes with diverse material properties in various devices.

Findings

Successful reconstruction of highly reflective automobile panels

Efficient scene representation with the plenoptic octree

Demonstrated applicability on mobile and AR devices

Abstract

A new passive approach called Generalized Scene Reconstruction (GSR) enables "generalized scenes" to be effectively reconstructed. Generalized scenes are defined to be "boundless" spaces that include non-Lambertian, partially transmissive, textureless and finely-structured matter. A new data structure called a plenoptic octree is introduced to enable efficient (database-like) light and matter field reconstruction in devices such as mobile phones, augmented reality (AR) glasses and drones. To satisfy threshold requirements for GSR accuracy, scenes are represented as systems of partially polarized light, radiometrically interacting with matter. To demonstrate GSR, a prototype imaging polarimeter is used to reconstruct (in generalized light fields) highly reflective, hail-damaged automobile body panels. Follow-on GSR experiments are described.