GeMuCo: Generalized Multisensory Correlational Model for Body Schema Learning

TL;DR

This paper introduces GeMuCo, a unified model enabling robots to autonomously learn and adapt their body schema from multisensory data, improving control, state estimation, and anomaly detection across various robotic platforms.

Contribution

The paper presents a novel generalized correlational model that allows robots to learn and adapt their body schema online from multisensory experiences, unifying control and estimation tasks.

Findings

Effective in tool-use scenarios with changing grasp states

Successful joint-muscle mapping for musculoskeletal robots

Demonstrated full-body manipulation on a low-rigidity humanoid

Abstract

Humans can autonomously learn the relationship between sensation and motion in their own bodies, estimate and control their own body states, and move while continuously adapting to the current environment. On the other hand, current robots control their bodies by learning the network structure described by humans from their experiences, making certain assumptions on the relationship between sensors and actuators. In addition, the network model does not adapt to changes in the robot's body, the tools that are grasped, or the environment, and there is no unified theory, not only for control but also for state estimation, anomaly detection, simulation, and so on. In this study, we propose a Generalized Multisensory Correlational Model (GeMuCo), in which the robot itself acquires a body schema describing the correlation between sensors and actuators from its own experience, including model structures such as network input/output. The robot adapts to the current environment by updating this body schema model online, estimates and controls its body state, and even performs anomaly detection and simulation. We demonstrate the effectiveness of this method by applying it to tool-use considering changes in grasping state for an axis-driven robot, to joint-muscle mapping learning for a musculoskeletal robot, and to full-body tool manipulation for a low-rigidity plastic-made humanoid.