GATE: Geometry-Aware Trained Encoding

TL;DR

GATE introduces a geometry-aware encoding method that stores feature vectors on triangular meshes, improving neural rendering efficiency and overcoming limitations of previous hash-based schemes by decoupling feature density from geometry density.

Contribution

The paper presents GATE, a novel encoding scheme that leverages mesh surfaces for feature storage, enhancing neural network training and adaptive level-of-detail control.

Findings

Reduces hash collision issues in neural encoding.

Allows decoupling of feature density from scene geometry.

Improves neural rendering performance and flexibility.

Abstract

The encoding of input parameters is one of the fundamental building blocks of neural network algorithms. Its goal is to map the input data to a higher-dimensional space, typically supported by trained feature vectors. The mapping is crucial for the efficiency and approximation quality of neural networks. We propose a novel geometry-aware encoding called GATE that stores feature vectors on the surface of triangular meshes. Our encoding is suitable for neural rendering-related algorithms, for example, neural radiance caching. It also avoids limitations of previous hash-based encoding schemes, such as hash collisions, selection of resolution versus scene size, and divergent memory access. Our approach decouples feature vector density from geometry density using mesh colors, while allowing for finer control over neural network training and adaptive level-of-detail.