Identifying Important Sensory Feedback for Learning Locomotion Skills

TL;DR

This paper introduces a systematic saliency analysis to identify essential sensory feedback states for learning quadruped robot locomotion skills via deep reinforcement learning, enabling robust performance with minimal sensing.

Contribution

It presents a quantitative method to determine key feedback states for motor skill learning, reducing sensory dependencies while maintaining performance.

Findings

Key states include joint positions, gravity vector, velocities.

Robots achieve robust locomotion with only key states.

Removing key states significantly impairs learning success.

Abstract

Robot motor skills can be learned through deep reinforcement learning (DRL) by neural networks as state-action mappings. While the selection of state observations is crucial, there has been a lack of quantitative analysis to date. Here, we present a systematic saliency analysis that quantitatively evaluates the relative importance of different feedback states for motor skills learned through DRL. Our approach can identify the most essential feedback states for locomotion skills, including balance recovery, trotting, bounding, pacing and galloping. By using only key states including joint positions, gravity vector, base linear and angular velocities, we demonstrate that a simulated quadruped robot can achieve robust performance in various test scenarios across these distinct skills. The benchmarks using task performance metrics show that locomotion skills learned with key states can achieve comparable performance to those with all states, and the task performance or learning success rate will drop significantly if key states are missing. This work provides quantitative insights into the relationship between state observations and specific types of motor skills, serving as a guideline for robot motor learning. The proposed method is applicable to differentiable state-action mapping, such as neural network based control policies, enabling the learning of a wide range of motor skills with minimal sensing dependencies.