Adaptive Whole-body Robotic Tool-use Learning on Low-rigidity Plastic-made Humanoids Using Vision and Tactile Sensors

TL;DR

This paper introduces a neural network-based method for adaptive whole-body tool-use control on low-rigidity humanoid robots, accounting for body deflections and tool variations using vision and tactile sensors.

Contribution

It presents a novel neural network model that captures the mutual relationship among joint angles, visual, and tactile data, incorporating Parametric Bias for tool variation adaptation.

Findings

Effective tool-tip control on low-rigidity humanoid robot

Neural network captures mutual relationships among sensor data

Adaptation to different tool weights and lengths

Abstract

Various robots have been developed so far; however, we face challenges in modeling the low-rigidity bodies of some robots. In particular, the deflection of the body changes during tool-use due to object grasping, resulting in significant shifts in the tool-tip position and the body's center of gravity. Moreover, this deflection varies depending on the weight and length of the tool, making these models exceptionally complex. However, there is currently no control or learning method that takes all of these effects into account. In this study, we propose a method for constructing a neural network that describes the mutual relationship among joint angle, visual information, and tactile information from the feet. We aim to train this network using the actual robot data and utilize it for tool-tip control. Additionally, we employ Parametric Bias to capture changes in this mutual relationship caused by variations in the weight and length of tools, enabling us to understand the characteristics of the grasped tool from the current sensor information. We apply this approach to the whole-body tool-use on KXR, a low-rigidity plastic-made humanoid robot, to validate its effectiveness.