Using Curvilinear Features in Focus for Registering a Single Image to a 3D Object

TL;DR

This paper presents a novel method for 2D/3D registration by using curvilinear features and focus-based representations to match photographs with 3D models, improving pose estimation accuracy.

Contribution

It introduces a new curvilinear saliency concept and a ridge and valley detector for depth images, combined with focus curves, to enhance feature matching across modalities.

Findings

High repeatability of detected features

Effective registration and pose estimation results

Improved matching accuracy across modalities

Abstract

In the context of 2D/3D registration, this paper introduces an approach that allows to match features detected in two different modalities: photographs and 3D models, by using a common 2D reprensentation. More precisely, 2D images are matched with a set of depth images, representing the 3D model. After introducing the concept of curvilinear saliency, related to curvature estimation, we propose a new ridge and valley detector for depth images rendered from 3D model. A variant of this detector is adapted to photographs, in particular by applying it in multi-scale and by combining this feature detector with the principle of focus curves. Finally, a registration algorithm for determining the correct viewpoint of the 3D model and thus the pose is proposed. It is based on using histogram of gradients features adapted to the features manipulated in 2D and in 3D, and the introduction of repeatability scores. The results presented highlight the quality of the features detected, in term of repeatability, and also the interest of the approach for registration and pose estimation.