Markerless visual servoing on unknown objects for humanoid robot platforms

TL;DR

This paper presents a comprehensive markerless visual servoing framework for humanoid robots to accurately grasp unknown objects using stereo vision, Bayesian filtering, and real-time control, demonstrated on the iCub platform.

Contribution

It introduces a novel integrated approach combining object volume estimation, pose tracking, grasp planning, and visual servoing without markers, tailored for unknown objects.

Findings

Effective real-time performance on iCub robot

Achieved sub-pixel precision in object grasping

Demonstrated robustness and smooth trajectories

Abstract

To precisely reach for an object with a humanoid robot, it is of central importance to have good knowledge of both end-effector, object pose and shape. In this work we propose a framework for markerless visual servoing on unknown objects, which is divided in four main parts: I) a least-squares minimization problem is formulated to find the volume of the object graspable by the robot's hand using its stereo vision; II) a recursive Bayesian filtering technique, based on Sequential Monte Carlo (SMC) filtering, estimates the 6D pose (position and orientation) of the robot's end-effector without the use of markers; III) a nonlinear constrained optimization problem is formulated to compute the desired graspable pose about the object; IV) an image-based visual servo control commands the robot's end-effector toward the desired pose. We demonstrate effectiveness and robustness of our approach with extensive experiments on the iCub humanoid robot platform, achieving real-time computation, smooth trajectories and sub-pixel precisions.