Integration of Vision-based Object Detection and Grasping for Articulated Manipulator in Lunar Conditions

TL;DR

This paper develops a vision-based system integrating object detection, segmentation, and grasping for lunar robots, successfully performing rock stacking in challenging lighting with a 92% success rate and demonstrating assembly of robot parts.

Contribution

It introduces a versatile vision-based pipeline for lunar robot manipulation tasks, capable of handling various applications and challenging environmental conditions.

Findings

Achieved 92% success in rock stacking under difficult lighting.

Demonstrated assembly of 3D printed robot components.

Proposed a generic task pipeline adaptable to different lunar tasks.

Abstract

The integration of vision-based frameworks to achieve lunar robot applications faces numerous challenges such as terrain configuration or extreme lighting conditions. This paper presents a generic task pipeline using object detection, instance segmentation and grasp detection, that can be used for various applications by using the results of these vision-based systems in a different way. We achieve a rock stacking task on a non-flat surface in difficult lighting conditions with a very good success rate of 92%. Eventually, we present an experiment to assemble 3D printed robot components to initiate more complex tasks in the future.