CAwa-NeRF: Instant Learning of Compression-Aware NeRF Features

TL;DR

CAwa-NeRF introduces a method for instant compression-aware learning of NeRF features, significantly reducing storage size with negligible training overhead, applicable to various scene types without sacrificing quality.

Contribution

The paper presents a novel approach enabling immediate compression of NeRF feature grids during training, compatible with existing models like Instant-NGP, without additional training time or architecture changes.

Findings

Achieves 94% compression of feature grids with no quality loss.

Reduces feature grid size to as low as 2.4% of original while maintaining high PSNR.

Applicable to diverse static scene types, including real-life captured scenes.

Abstract

Modeling 3D scenes by volumetric feature grids is one of the promising directions of neural approximations to improve Neural Radiance Fields (NeRF). Instant-NGP (INGP) introduced multi-resolution hash encoding from a lookup table of trainable feature grids which enabled learning high-quality neural graphics primitives in a matter of seconds. However, this improvement came at the cost of higher storage size. In this paper, we address this challenge by introducing instant learning of compression-aware NeRF features (CAwa-NeRF), that allows exporting the zip compressed feature grids at the end of the model training with a negligible extra time overhead without changing neither the storage architecture nor the parameters used in the original INGP paper. Nonetheless, the proposed method is not limited to INGP but could also be adapted to any model. By means of extensive simulations, our proposed instant learning pipeline can achieve impressive results on different kinds of static scenes such as single object masked background scenes and real-life scenes captured in our studio. In particular, for single object masked background scenes CAwa-NeRF compresses the feature grids down to 6% (1.2 MB) of the original size without any loss in the PSNR (33 dB) or down to 2.4% (0.53 MB) with a slight virtual loss (32.31 dB).