Image-Based Roadmaps for Vision-Only Planning and Control of Robotic Manipulators

TL;DR

This paper introduces a novel image-based motion planning framework for robotic manipulators that constructs collision-free paths directly in image space, enabling vision-only control without explicit robot models.

Contribution

It presents a new approach to generate collision-free paths in image space using a roadmap built from visual features, eliminating the need for geometric models or proprioception.

Findings

Learned-distance roadmap achieved 100% success in control convergence.

Predefined image-space distance roadmap enabled faster transient responses.

The approach was validated through experimental robotic arm path following.

Abstract

This work presents a motion planning framework for robotic manipulators that computes collision-free paths directly in image space. The generated paths can then be tracked using vision-based control, eliminating the need for an explicit robot model or proprioceptive sensing. At the core of our approach is the construction of a roadmap entirely in image space. To achieve this, we explicitly define sampling, nearest-neighbor selection, and collision checking based on visual features rather than geometric models. We first collect a set of image-space samples by moving the robot within its workspace, capturing keypoints along its body at different configurations. These samples serve as nodes in the roadmap, which we construct using either learned or predefined distance metrics. At runtime, the roadmap generates collision-free paths directly in image space, removing the need for a robot model or joint encoders. We validate our approach through an experimental study in which a robotic arm follows planned paths using an adaptive vision-based control scheme to avoid obstacles. The results show that paths generated with the learned-distance roadmap achieved 100% success in control convergence, whereas the predefined image-space distance roadmap enabled faster transient responses but had a lower success rate in convergence.