Compressible-composable NeRF via Rank-residual Decomposition

TL;DR

This paper introduces a novel explicit neural field representation for NeRF that enables efficient manipulation, dynamic compression, and scene composition through a rank-residual decomposition approach, avoiding large model sizes and shared renderer constraints.

Contribution

It proposes a hybrid tensor rank decomposition method with rank-residual learning for NeRF, allowing flexible model manipulation, compression, and composition without neural network-based representations.

Findings

Achieves comparable rendering quality to state-of-the-art NeRF methods.

Enables dynamic control of model detail via rank truncation.

Supports scene composition and object compression through rank concatenation.

Abstract

Neural Radiance Field (NeRF) has emerged as a compelling method to represent 3D objects and scenes for photo-realistic rendering. However, its implicit representation causes difficulty in manipulating the models like the explicit mesh representation. Several recent advances in NeRF manipulation are usually restricted by a shared renderer network, or suffer from large model size. To circumvent the hurdle, in this paper, we present an explicit neural field representation that enables efficient and convenient manipulation of models. To achieve this goal, we learn a hybrid tensor rank decomposition of the scene without neural networks. Motivated by the low-rank approximation property of the SVD algorithm, we propose a rank-residual learning strategy to encourage the preservation of primary information in lower ranks. The model size can then be dynamically adjusted by rank truncation to control the levels of detail, achieving near-optimal compression without extra optimization. Furthermore, different models can be arbitrarily transformed and composed into one scene by concatenating along the rank dimension. The growth of storage cost can also be mitigated by compressing the unimportant objects in the composed scene. We demonstrate that our method is able to achieve comparable rendering quality to state-of-the-art methods, while enabling extra capability of compression and composition. Code will be made available at https://github.com/ashawkey/CCNeRF.