Elastic Decision Transformer
Yueh-Hua Wu, Xiaolong Wang, Masashi Hamaya

TL;DR
The Elastic Decision Transformer (EDT) enhances trajectory stitching during action inference by adjusting history length, leading to improved performance over existing methods in multi-task benchmarks like D4RL locomotion and Atari games.
Contribution
EDT introduces a novel approach to trajectory stitching by dynamically adjusting history length during inference, improving over standard Decision Transformers.
Findings
EDT outperforms Q Learning-based methods on D4RL locomotion tasks.
EDT achieves better results in Atari game benchmarks.
Trajectory stitching improves with dynamic history adjustment.
Abstract
This paper introduces Elastic Decision Transformer (EDT), a significant advancement over the existing Decision Transformer (DT) and its variants. Although DT purports to generate an optimal trajectory, empirical evidence suggests it struggles with trajectory stitching, a process involving the generation of an optimal or near-optimal trajectory from the best parts of a set of sub-optimal trajectories. The proposed EDT differentiates itself by facilitating trajectory stitching during action inference at test time, achieved by adjusting the history length maintained in DT. Further, the EDT optimizes the trajectory by retaining a longer history when the previous trajectory is optimal and a shorter one when it is sub-optimal, enabling it to "stitch" with a more optimal trajectory. Extensive experimentation demonstrates EDT's ability to bridge the performance gap between DT-based and Q…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAnomaly Detection Techniques and Applications · Reinforcement Learning in Robotics · Human Pose and Action Recognition
MethodsMulti-Head Attention · Attention Is All You Need · Dense Connections · Dropout · Byte Pair Encoding · Softmax · Layer Normalization · Position-Wise Feed-Forward Layer · Linear Layer · Absolute Position Encodings
