N-Dimensional Gaussians for Fitting of High Dimensional Functions

TL;DR

This paper introduces efficient methods for fitting and evaluating high-dimensional Gaussian mixture models to represent complex, multi-parameter 3D content, enabling rapid rendering and adaptive refinement.

Contribution

It presents novel high-dimensional culling and density control schemes for Gaussian mixtures, facilitating scalable, compact, and adaptive explicit representations of high-dimensional functions.

Findings

Efficient fitting and evaluation of N-D Gaussian mixtures within minutes and milliseconds.

Enables representation of complex appearance dependent on many input dimensions.

Provides a scalable approach for high-dimensional content reconstruction.

Abstract

In the wake of many new ML-inspired approaches for reconstructing and representing high-quality 3D content, recent hybrid and explicitly learned representations exhibit promising performance and quality characteristics. However, their scaling to higher dimensions is challenging, e.g. when accounting for dynamic content with respect to additional parameters such as material properties, illumination, or time. In this paper, we tackle these challenges for an explicit representations based on Gaussian mixture models. With our solutions, we arrive at efficient fitting of compact N-dimensional Gaussian mixtures and enable efficient evaluation at render time: For fast fitting and evaluation, we introduce a high-dimensional culling scheme that efficiently bounds N-D Gaussians, inspired by Locality Sensitive Hashing. For adaptive refinement yet compact representation, we introduce a loss-adaptive density control scheme that incrementally guides the use of additional capacity towards missing details. With these tools we can for the first time represent complex appearance that depends on many input dimensions beyond position or viewing angle within a compact, explicit representation optimized in minutes and rendered in milliseconds.