Semantic Mapping with Simultaneous Object Detection and Localization

TL;DR

This paper introduces CT-Map, a filtering-based semantic mapping method that simultaneously detects objects and estimates their 6-DoF poses using a CRF model and particle filtering, demonstrated on a robot with RGB-D sensor.

Contribution

The paper presents a novel CRF-based semantic mapping approach with particle filtering for joint object detection and pose estimation, improving over baseline methods.

Findings

Enhanced object detection accuracy

Improved pose estimation consistency

Effective in real robot environment

Abstract

We present a filtering-based method for semantic mapping to simultaneously detect objects and localize their 6 degree-of-freedom pose. For our method, called Contextual Temporal Mapping (or CT-Map), we represent the semantic map as a belief over object classes and poses across an observed scene. Inference for the semantic mapping problem is then modeled in the form of a Conditional Random Field (CRF). CT-Map is a CRF that considers two forms of relationship potentials to account for contextual relations between objects and temporal consistency of object poses, as well as a measurement potential on observations. A particle filtering algorithm is then proposed to perform inference in the CT-Map model. We demonstrate the efficacy of the CT-Map method with a Michigan Progress Fetch robot equipped with a RGB-D sensor. Our results demonstrate that the particle filtering based inference of CT-Map provides improved object detection and pose estimation with respect to baseline methods that treat observations as independent samples of a scene.