Learning Category-level Last-meter Navigation from RGB Demonstrations of a Single-instance

TL;DR

This paper presents a novel RGB-based imitation learning approach for precise last-meter navigation of a mobile manipulator, enabling accurate object positioning without depth or map priors, and generalizing across unseen objects and environments.

Contribution

It introduces an object-centric imitation learning framework conditioned on goal images, multi-view RGB observations, and text prompts, with explicit object grounding and pose reasoning, for category-level last-meter navigation.

Findings

Achieves 73.47% success in edge-alignment

Achieves 96.94% success in object-alignment

Generalizes to unseen objects across diverse environments

Abstract

Achieving precise positioning of the mobile manipulator's base is essential for successful manipulation actions that follow. Most of the RGB-based navigation systems only guarantee coarse, meter-level accuracy, making them less suitable for the precise positioning phase of mobile manipulation. This gap prevents manipulation policies from operating within the distribution of their training demonstrations, resulting in frequent execution failures. We address this gap by introducing an object-centric imitation learning framework for last-meter navigation, enabling a quadruped mobile manipulator robot to achieve manipulation-ready positioning using only RGB observations from its onboard cameras. Our method conditions the navigation policy on three inputs: goal images, multi-view RGB observations from the onboard cameras, and a text prompt specifying the target object. A language-driven segmentation module and a spatial score-matrix decoder then supply explicit object grounding and relative pose reasoning. Using real-world data from a single object instance within a category, the system generalizes to unseen object instances across diverse environments with challenging lighting and background conditions. To comprehensively evaluate this, we introduce two metrics: an edge-alignment metric, which uses ground truth orientation, and an object-alignment metric, which evaluates how well the robot visually faces the target. Under these metrics, our policy achieves 73.47% success in edge-alignment and 96.94% success in object-alignment when positioning relative to unseen target objects. These results show that precise last-meter navigation can be achieved at a category-level without depth, LiDAR, or map priors, enabling a scalable pathway toward unified mobile manipulation. Project page: https://rpm-lab-umn.github.io/category-level-last-meter-nav/