Prioritized offline Goal-swapping Experience Replay
Wenyan Yang, Joni Pajarinen, Dinging Cai, Joni K\"am\"ar\"ainen

TL;DR
This paper introduces PGSER, a method that enhances offline goal-conditioned reinforcement learning by prioritizing goal-swapped experiences using a pre-trained Q function, leading to improved performance across various benchmarks.
Contribution
The paper proposes a novel prioritized goal-swapping experience replay method that effectively filters and emphasizes valuable goal-swapped transitions in offline reinforcement learning.
Findings
PGSER outperforms baseline methods in benchmark tasks.
Significant improvements in dexterous in-hand manipulation tasks.
Effective prioritization reduces invalid trajectories.
Abstract
In goal-conditioned offline reinforcement learning, an agent learns from previously collected data to go to an arbitrary goal. Since the offline data only contains a finite number of trajectories, a main challenge is how to generate more data. Goal-swapping generates additional data by switching trajectory goals but while doing so produces a large number of invalid trajectories. To address this issue, we propose prioritized goal-swapping experience replay (PGSER). PGSER uses a pre-trained Q function to assign higher priority weights to goal swapped transitions that allow reaching the goal. In experiments, PGSER significantly improves over baselines in a wide range of benchmark tasks, including challenging previously unsuccessful dexterous in-hand manipulation tasks.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsReinforcement Learning in Robotics · Explainable Artificial Intelligence (XAI) · Adversarial Robustness in Machine Learning
MethodsExperience Replay
