3D Scene Compression through Entropy Penalized Neural Representation Functions

TL;DR

This paper introduces a neural network-based method for compressing 3D scenes by directly encoding scene functions, resulting in better quality and lower bitrates compared to traditional view-based compression methods.

Contribution

The authors propose an end-to-end neural scene compression approach using entropy penalization, unifying scene representation and compression, and demonstrating improved performance over existing methods.

Findings

Outperforms traditional view-based compression in quality and bitrate.

Joint multi-scene representation enhances low-bitrate performance.

Neural implicit functions enable efficient 3D scene encoding.

Abstract

Some forms of novel visual media enable the viewer to explore a 3D scene from arbitrary viewpoints, by interpolating between a discrete set of original views. Compared to 2D imagery, these types of applications require much larger amounts of storage space, which we seek to reduce. Existing approaches for compressing 3D scenes are based on a separation of compression and rendering: each of the original views is compressed using traditional 2D image formats; the receiver decompresses the views and then performs the rendering. We unify these steps by directly compressing an implicit representation of the scene, a function that maps spatial coordinates to a radiance vector field, which can then be queried to render arbitrary viewpoints. The function is implemented as a neural network and jointly trained for reconstruction as well as compressibility, in an end-to-end manner, with the use of an entropy penalty on the parameters. Our method significantly outperforms a state-of-the-art conventional approach for scene compression, achieving simultaneously higher quality reconstructions and lower bitrates. Furthermore, we show that the performance at lower bitrates can be improved by jointly representing multiple scenes using a soft form of parameter sharing.