Masked Sensory-Temporal Attention for Sensor Generalization in Quadruped Locomotion

TL;DR

This paper introduces Masked Sensory-Temporal Attention (MSTA), a transformer-based approach that improves sensor data handling and generalization in quadruped locomotion, even with missing information or unseen sensor combinations.

Contribution

The paper proposes MSTA, a novel masking transformer mechanism that enhances sensory-temporal understanding and generalization across different quadruped robot models and sensor inputs.

Findings

MSTA effectively manages missing sensor data.

It generalizes well to unseen sensor combinations.

It is deployable on physical quadruped systems.

Abstract

With the rising focus on quadrupeds, a generalized policy capable of handling different robot models and sensor inputs becomes highly beneficial. Although several methods have been proposed to address different morphologies, it remains a challenge for learning-based policies to manage various combinations of proprioceptive information. This paper presents Masked Sensory-Temporal Attention (MSTA), a novel transformer-based mechanism with masking for quadruped locomotion. It employs direct sensor-level attention to enhance the sensory-temporal understanding and handle different combinations of sensor data, serving as a foundation for incorporating unseen information. MSTA can effectively understand its states even with a large portion of missing information, and is flexible enough to be deployed on physical systems despite the long input sequence.