Factorized Multi-Resolution HashGrid for Efficient Neural Radiance Fields: Execution on Edge-Devices

TL;DR

Fact-Hash is a novel parameter-encoding method that combines tensor factorization and hash-encoding to enable efficient on-device training of neural radiance fields, reducing memory usage while maintaining quality.

Contribution

The paper introduces Fact-Hash, a new encoding technique that improves memory efficiency and robustness for on-device NeRF training, suitable for resource-constrained environments.

Findings

Fact-Hash reduces memory usage by over one-third compared to previous methods.

It maintains PSNR quality while improving computational efficiency and energy consumption.

Evaluations demonstrate superior performance of Fact-Hash in on-device scenarios.

Abstract

We introduce Fact-Hash, a novel parameter-encoding method for training on-device neural radiance fields. Neural Radiance Fields (NeRF) have proven pivotal in 3D representations, but their applications are limited due to large computational resources. On-device training can open large application fields, providing strength in communication limitations, privacy concerns, and fast adaptation to a frequently changing scene. However, challenges such as limited resources (GPU memory, storage, and power) impede their deployment. To handle this, we introduce Fact-Hash, a novel parameter-encoding merging Tensor Factorization and Hash-encoding techniques. This integration offers two benefits: the use of rich high-resolution features and the few-shot robustness. In Fact-Hash, we project 3D coordinates into multiple lower-dimensional forms (2D or 1D) before applying the hash function and then aggregate them into a single feature. Comparative evaluations against state-of-the-art methods demonstrate Fact-Hash's superior memory efficiency, preserving quality and rendering speed. Fact-Hash saves memory usage by over one-third while maintaining the PSNR values compared to previous encoding methods. The on-device experiment validates the superiority of Fact-Hash compared to alternative positional encoding methods in computational efficiency and energy consumption. These findings highlight Fact-Hash as a promising solution to improve feature grid representation, address memory constraints, and improve quality in various applications. Project page: https://facthash.github.io/