Dense image registration and deformable surface reconstruction in presence of occlusions and minimal texture

TL;DR

This paper introduces a novel framework for dense 3D surface reconstruction from monocular images that effectively handles occlusions and minimal textures by scaling dense features with relevancy scores, outperforming existing methods.

Contribution

The work presents a new template-matching based approach that improves robustness in reconstructing poorly textured, occluded surfaces, with extensive comparisons demonstrating state-of-the-art performance.

Findings

Achieves superior results on standard datasets.

Performs well on a new sparsely textured, occluded surface dataset.

Outperforms existing local feature matching and dense template alignment methods.

Abstract

Deformable surface tracking from monocular images is well-known to be under-constrained. Occlusions often make the task even more challenging, and can result in failure if the surface is not sufficiently textured. In this work, we explicitly address the problem of 3D reconstruction of poorly textured, occluded surfaces, proposing a framework based on a template-matching approach that scales dense robust features by a relevancy score. Our approach is extensively compared to current methods employing both local feature matching and dense template alignment. We test on standard datasets as well as on a new dataset (that will be made publicly available) of a sparsely textured, occluded surface. Our framework achieves state-of-the-art results for both well and poorly textured, occluded surfaces.