Learning Geometrically Consistent Mesh Corrections

TL;DR

This paper introduces a CNN-based method for post-hoc correction of low-quality 3D meshes, leveraging learned priors to improve accuracy while addressing smoothing issues and geometrical inconsistencies.

Contribution

It presents a novel neural network architecture with a loss-weighting mechanism and a specialized loss function to enhance mesh correction quality.

Findings

Reduces gross errors by up to 77.5%.

Outperforms previous methods by up to five times.

Addresses smoothing and geometric inconsistency issues effectively.

Abstract

Building good 3D maps is a challenging and expensive task, which requires high-quality sensors and careful, time-consuming scanning. We seek to reduce the cost of building good reconstructions by correcting views of existing low-quality ones in a post-hoc fashion using learnt priors over surfaces and appearance. We train a CNN model to predict the difference in inverse-depth from varying viewpoints of two meshes -- one of low quality that we wish to correct, and one of high-quality that we use as a reference. In contrast to previous work, we pay attention to the problem of excessive smoothing in corrected meshes. We address this with a suitable network architecture, and introduce a loss-weighting mechanism that emphasises edges in the prediction. Furthermore, smooth predictions result in geometrical inconsistencies. To deal with this issue, we present a loss function which penalises re-projection differences that are not due to occlusions. Our model reduces gross errors by 45.3%--77.5%, up to five times more than previous work.