Nix and Fix: Targeting 1000x Compression of 3D Gaussian Splatting with Diffusion Models

TL;DR

NiFi employs diffusion-based restoration to achieve unprecedented 1000x compression of 3D Gaussian Splatting, maintaining high perceptual quality at extremely low data rates.

Contribution

Introducing NiFi, a novel artifact-aware diffusion method for ultra-high compression of 3DGS, significantly reducing space requirements while preserving visual quality.

Findings

Achieves state-of-the-art perceptual quality at 0.1 MB compression rate.

Demonstrates up to 1000x rate improvement over original 3DGS.

Maintains real-time rendering capabilities with extreme compression.

Abstract

3D Gaussian Splatting (3DGS) revolutionized novel view rendering. Instead of inferring from dense spatial points, as implicit representations do, 3DGS uses sparse Gaussians. This enables real-time performance but increases space requirements, hindering rate-constrained applications. 3DGS compression emerged as a field aimed at alleviating this issue. While impressive progress has been made, at low rates, compression introduces artifacts that degrade visual quality significantly. We introduce NiFi, a method for extreme 3DGS compression through restoration via artifact-aware, diffusion-based one-step distillation. We show that our method achieves state-of-the-art perceptual quality at extremely low rates, down to 0.1 MB, and towards 1000x rate improvement over 3DGS at comparable perceptual performance. Code is available at: https://github.com/ceteke/nifi