Smooth Mesh Estimation from Depth Data using Non-Smooth Convex Optimization

TL;DR

This paper introduces a real-time method for directly reconstructing smooth, accurate 3D meshes from depth data and sparse landmarks by solving a non-smooth convex optimization problem, bypassing volumetric representations.

Contribution

It proposes a novel convex optimization framework for direct mesh estimation from depth maps, improving accuracy and efficiency over existing volumetric methods.

Findings

Outperforms state-of-the-art in mesh reconstruction accuracy

Runs in real-time with efficient primal-dual optimization

Produces smooth 3D meshes from sparse depth and landmarks

Abstract

Meshes are commonly used as 3D maps since they encode the topology of the scene while being lightweight. Unfortunately, 3D meshes are mathematically difficult to handle directly because of their combinatorial and discrete nature. Therefore, most approaches generate 3D meshes of a scene after fusing depth data using volumetric or other representations. Nevertheless, volumetric fusion remains computationally expensive both in terms of speed and memory. In this paper, we leapfrog these intermediate representations and build a 3D mesh directly from a depth map and the sparse landmarks triangulated with visual odometry. To this end, we formulate a non-smooth convex optimization problem that we solve using a primal-dual method. Our approach generates a smooth and accurate 3D mesh that substantially improves the state-of-the-art on direct mesh reconstruction while running in real-time.