Sim-to-Real Transfer for Quadrupedal Locomotion via Terrain Transformer

TL;DR

This paper introduces Terrain Transformer (TERT), a high-capacity Transformer-based model for quadrupedal locomotion that improves sim-to-real transfer, enabling robots to navigate complex terrains more effectively.

Contribution

The paper presents TERT, a novel Transformer architecture for locomotion control, and a two-stage training framework that enhances sim-to-real transfer in complex environments.

Findings

TERT outperforms baselines in simulation across various terrains.

TERT successfully navigates nine challenging real-world terrains.

The two-stage training improves transfer robustness and control quality.

Abstract

Deep reinforcement learning has recently emerged as an appealing alternative for legged locomotion over multiple terrains by training a policy in physical simulation and then transferring it to the real world (i.e., sim-to-real transfer). Despite considerable progress, the capacity and scalability of traditional neural networks are still limited, which may hinder their applications in more complex environments. In contrast, the Transformer architecture has shown its superiority in a wide range of large-scale sequence modeling tasks, including natural language processing and decision-making problems. In this paper, we propose Terrain Transformer (TERT), a high-capacity Transformer model for quadrupedal locomotion control on various terrains. Furthermore, to better leverage Transformer in sim-to-real scenarios, we present a novel two-stage training framework consisting of an offline pretraining stage and an online correction stage, which can naturally integrate Transformer with privileged training. Extensive experiments in simulation demonstrate that TERT outperforms state-of-the-art baselines on different terrains in terms of return, energy consumption and control smoothness. In further real-world validation, TERT successfully traverses nine challenging terrains, including sand pit and stair down, which can not be accomplished by strong baselines.