Multimodal feedback for active robot-object interaction

TL;DR

This paper introduces a multimodal feedback system combining laser-based SLAM, RGBD imaging, and contact sensors to enhance active robot-object interaction, enabling better manipulation and collision avoidance.

Contribution

The work presents a novel multimodal feedback approach integrating visual and tactile data for improved robot-object interaction and manipulation.

Findings

The multimodal system outperforms visual-only and tactile-only feedback methods.

The system effectively adjusts robot pose for collision-free grasping.

Experimental results validate the approach's robustness in dynamic environments.

Abstract

In this work, we present a multimodal system for active robot-object interaction using laser-based SLAM, RGBD images, and contact sensors. In the object manipulation task, the robot adjusts its initial pose with respect to obstacles and target objects through RGBD data so it can perform object grasping in different configuration spaces while avoiding collisions, and updates the information related to the last steps of the manipulation process using the contact sensors in its hand. We perform a series of experiment to evaluate the performance of the proposed system following the the RoboCup2018 international competition regulations. We compare our approach with a number of baselines, namely a no-feedback method and visual-only and tactile-only feedback methods, where our proposed visual-and-tactile feedback method performs best.