Goal-Conditioned Decision Transformer for Multi-Goal Offline Reinforcement Learning

TL;DR

This paper presents a goal-conditioned decision transformer tailored for offline multi-goal robotics, improving task performance and robustness using pre-collected data.

Contribution

It introduces a novel transformer-based framework that explicitly incorporates goal states for efficient multi-goal offline reinforcement learning in robotics.

Findings

Outperforms state-of-the-art online methods on complex tasks

Maintains robustness in sparse-reward scenarios

Effective with limited expert demonstrations

Abstract

Reinforcement learning (RL) in robotics faces significant hurdles regarding sample efficiency and generalization across varying goals. While Offline RL mitigates the need for costly online interactions, its integration with goal-conditioned policies and transformer-based architectures remains underexplored. We introduce a Goal-Conditioned Decision Transformer adapted for offline multi-goal robotics. By explicitly incorporating goal states into the sequence modeling framework, our approach efficiently solves varying tasks using only pre-collected data. We validate this method on a newly released offline dataset for the Franka Emika Panda platform. Experimental results demonstrate that our approach outperforms state-of-the-art online baselines in complex tasks and maintains robustness in sparse-reward settings, even with limited expert demonstrations.