Geometric Algebra Planes: Convex Implicit Neural Volumes

TL;DR

This paper introduces GA-Planes, a novel class of implicit neural volume representations that can be trained via convex optimization, improving training stability and efficiency while maintaining high expressiveness for 3D and 2D tasks.

Contribution

GA-Planes is the first implicit neural volume model trainable by convex optimization, generalizing existing representations and applicable to various inverse problems.

Findings

GA-Planes in 2D is equivalent to low-rank plus low-resolution matrix factorization.

GA-Planes outperforms classic low-rank plus sparse decomposition in natural image fitting.

In 3D, GA-Planes demonstrates competitive performance in radiance field, segmentation, and video segmentation tasks.

Abstract

Volume parameterizations abound in recent literature, from the classic voxel grid to the implicit neural representation and everything in between. While implicit representations have shown impressive capacity and better memory efficiency compared to voxel grids, to date they require training via nonconvex optimization. This nonconvex training process can be slow to converge and sensitive to initialization and hyperparameter choices that affect the final converged result. We introduce a family of models, GA-Planes, that is the first class of implicit neural volume representations that can be trained by convex optimization. GA-Planes models include any combination of features stored in tensor basis elements, followed by a neural feature decoder. They generalize many existing representations and can be adapted for convex, semiconvex, or nonconvex training as needed for different inverse problems. In the 2D setting, we prove that GA-Planes is equivalent to a low-rank plus low-resolution matrix factorization; we show that this approximation outperforms the classic low-rank plus sparse decomposition for fitting a natural image. In 3D, we demonstrate GA-Planes' competitive performance in terms of expressiveness, model size, and optimizability across three volume fitting tasks: radiance field reconstruction, 3D segmentation, and video segmentation.