Localization and Mapping using Instance-specific Mesh Models

TL;DR

This paper introduces an online optimized instance-specific mesh model for object shape and pose estimation from monocular images, enhancing semantic mapping for robotic applications.

Contribution

It presents a novel method to optimize object mesh models using semantic cues from camera images, enabling accurate shape and pose estimation in real-time.

Findings

Effective shape and pose estimation demonstrated in simulation.

Accurate object mapping on the KITTI dataset.

Improved semantic mapping for robotic navigation.

Abstract

This paper focuses on building semantic maps, containing object poses and shapes, using a monocular camera. This is an important problem because robots need rich understanding of geometry and context if they are to shape the future of transportation, construction, and agriculture. Our contribution is an instance-specific mesh model of object shape that can be optimized online based on semantic information extracted from camera images. Multi-view constraints on the object shape are obtained by detecting objects and extracting category-specific keypoints and segmentation masks. We show that the errors between projections of the mesh model and the observed keypoints and masks can be differentiated in order to obtain accurate instance-specific object shapes. We evaluate the performance of the proposed approach in simulation and on the KITTI dataset by building maps of car poses and shapes.