Visual-tactile manipulation to collect household waste in outdoor

TL;DR

This paper introduces a perception system combining vision and tactile sensors for outdoor household waste collection by robots, achieving high accuracy and fast processing.

Contribution

It presents an integrated perception system with CNN-based modules for waste localization, recognition, and grasping adjustment in outdoor environments.

Findings

Localization error around 6%

Recognition accuracy of 98%

Grasping stability of 91%

Abstract

This work presents a perception system applied to robotic manipulation, that is able to assist in navigation, household waste classification and collection in outdoor environments. This system is made up of optical tactile sensors, RGBD cameras and a LiDAR. These sensors are integrated on a mobile platform with a robot manipulator and a robotic gripper. Our system is divided in three software modules, two of them are vision-based and the last one is tactile-based. The vision-based modules use CNNs to localize and recognize solid household waste, together with the grasping points estimation. The tactile-based module, which also uses CNNs and image processing, adjusts the gripper opening to control the grasping from touch data. Our proposal achieves localization errors around 6 %, a recognition accuracy of 98% and ensures the grasping stability the 91% of the attempts. The sum of runtimes of the three modules is less than 750 ms.