Sim-to-Real Transfer of Compliant Bipedal Locomotion on Torque Sensor-Less Gear-Driven Humanoid

TL;DR

This paper presents a novel sim-to-real transfer method for compliant bipedal robots that addresses actuator modeling and system identification challenges, enabling stable walking without force sensors.

Contribution

It introduces a new gear simulation model and a system identification approach that leverages failed attempts, improving sim-to-real transfer for torque sensor-less humanoid robots.

Findings

Successfully transferred policies to ROBOTIS-OP3 robot

Achieved stable walking on uneven surfaces

Robot withstands severe disturbances

Abstract

Sim-to-real is a mainstream method to cope with the large number of trials needed by typical deep reinforcement learning methods. However, transferring a policy trained in simulation to actual hardware remains an open challenge due to the reality gap. In particular, the characteristics of actuators in legged robots have a considerable influence on sim-to-real transfer. There are two challenges: 1) High reduction ratio gears are widely used in actuators, and the reality gap issue becomes especially pronounced when backdrivability is considered in controlling joints compliantly. 2) The difficulty in achieving stable bipedal locomotion causes typical system identification methods to fail to sufficiently transfer the policy. For these two challenges, we propose 1) a new simulation model of gears and 2) a method for system identification that can utilize failed attempts. The method's effectiveness is verified using a biped robot, the ROBOTIS-OP3, and the sim-to-real transferred policy can stabilize the robot under severe disturbances and walk on uneven surfaces without using force and torque sensors.