Local Positional Encoding for Multi-Layer Perceptrons

TL;DR

This paper introduces local positional encoding for MLPs, combining positional and grid encodings to enhance high-frequency signal representation with fewer frequencies, improving 2D and 3D regression results.

Contribution

It proposes a novel local positional encoding method that extends existing encodings, enabling small MLPs to better learn high-frequency signals efficiently.

Findings

Outperforms positional and grid encodings in quality

Comparable to multi-resolution grid encoding in results

Effective in 2D and 3D regression tasks

Abstract

A multi-layer perceptron (MLP) is a type of neural networks which has a long history of research and has been studied actively recently in computer vision and graphics fields. One of the well-known problems of an MLP is the capability of expressing high-frequency signals from low-dimensional inputs. There are several studies for input encodings to improve the reconstruction quality of an MLP by applying pre-processing against the input data. This paper proposes a novel input encoding method, local positional encoding, which is an extension of positional and grid encodings. Our proposed method combines these two encoding techniques so that a small MLP learns high-frequency signals by using positional encoding with fewer frequencies under the lower resolution of the grid to consider the local position and scale in each grid cell. We demonstrate the effectiveness of our proposed method by applying it to common 2D and 3D regression tasks where it shows higher-quality results compared to positional and grid encodings, and comparable results to hierarchical variants of grid encoding such as multi-resolution grid encoding with equivalent memory footprint.