KVN: Keypoints Voting Network with Differentiable RANSAC for Stereo Pose Estimation

TL;DR

This paper introduces a differentiable RANSAC layer and a multi-view PnP solver to improve stereo object pose estimation, achieving state-of-the-art results on challenging datasets.

Contribution

It presents a novel end-to-end trainable stereo pose estimation framework integrating differentiable RANSAC and multi-view PnP, advancing prior non-differentiable RANSAC-based methods.

Findings

Differentiable RANSAC significantly improves pose accuracy.

The multi-view PnP enhances robustness across views.

State-of-the-art results on public and custom datasets.

Abstract

Object pose estimation is a fundamental computer vision task exploited in several robotics and augmented reality applications. Many established approaches rely on predicting 2D-3D keypoint correspondences using RANSAC (Random sample consensus) and estimating the object pose using the PnP (Perspective-n-Point) algorithm. Being RANSAC non-differentiable, correspondences cannot be directly learned in an end-to-end fashion. In this paper, we address the stereo image-based object pose estimation problem by i) introducing a differentiable RANSAC layer into a well-known monocular pose estimation network; ii) exploiting an uncertainty-driven multi-view PnP solver which can fuse information from multiple views. We evaluate our approach on a challenging public stereo object pose estimation dataset and a custom-built dataset we call Transparent Tableware Dataset (TTD), yielding state-of-the-art results against other recent approaches. Furthermore, in our ablation study, we show that the differentiable RANSAC layer plays a significant role in the accuracy of the proposed method. We release with this paper the code of our method and the TTD dataset.