Plenoptic PNG: Real-Time Neural Radiance Fields in 150 KB

TL;DR

This paper introduces Plenoptic PNG, a compact 3D scene encoding method that enables real-time rendering in just 150 KB, making free-viewpoint content more accessible across platforms.

Contribution

It proposes a novel sinusoidal dense volume representation that significantly reduces model size and simplifies rendering, facilitating real-time, platform-independent 3D scene visualization.

Findings

Model size as small as 150 KB per scene

Real-time rendering with a 300-line pipeline

Universal compatibility across platforms

Abstract

The goal of this paper is to encode a 3D scene into an extremely compact representation from 2D images and to enable its transmittance, decoding and rendering in real-time across various platforms. Despite the progress in NeRFs and Gaussian Splats, their large model size and specialized renderers make it challenging to distribute free-viewpoint 3D content as easily as images. To address this, we have designed a novel 3D representation that encodes the plenoptic function into sinusoidal function indexed dense volumes. This approach facilitates feature sharing across different locations, improving compactness over traditional spatial voxels. The memory footprint of the dense 3D feature grid can be further reduced using spatial decomposition techniques. This design combines the strengths of spatial hashing functions and voxel decomposition, resulting in a model size as small as 150 KB for each 3D scene. Moreover, PPNG features a lightweight rendering pipeline with only 300 lines of code that decodes its representation into standard GL textures and fragment shaders. This enables real-time rendering using the traditional GL pipeline, ensuring universal compatibility and efficiency across various platforms without additional dependencies.