Semantic 3D Reconstruction with Finite Element Bases

TL;DR

This paper introduces a finite element method framework for discretizing multi-label problems on continuous domains, enhancing semantic 3D reconstruction with improved efficiency and flexibility.

Contribution

It bridges FEM discretizations with labeling problems in computer vision, allowing for anisotropic, non-metric regularizers in a convex relaxation framework.

Findings

FEM discretization is valid for general labeling problems.

The approach reduces memory usage and computation time.

It enables adaptive resolution in 3D reconstruction.

Abstract

We propose a novel framework for the discretisation of multi-label problems on arbitrary, continuous domains. Our work bridges the gap between general FEM discretisations, and labeling problems that arise in a variety of computer vision tasks, including for instance those derived from the generalised Potts model. Starting from the popular formulation of labeling as a convex relaxation by functional lifting, we show that FEM discretisation is valid for the most general case, where the regulariser is anisotropic and non-metric. While our findings are generic and applicable to different vision problems, we demonstrate their practical implementation in the context of semantic 3D reconstruction, where such regularisers have proved particularly beneficial. The proposed FEM approach leads to a smaller memory footprint as well as faster computation, and it constitutes a very simple way to enable variable, adaptive resolution within the same model.