CoMPS: Continual Meta Policy Search
Glen Berseth, Zhiwei Zhang, Grace Zhang, Chelsea Finn, Sergey Levine

TL;DR
CoMPS is a continual meta-learning method that incrementally trains on task sequences, enabling agents to learn new tasks quickly without revisiting previous tasks, outperforming prior methods in continuous control tasks.
Contribution
Introduces CoMPS, a novel continual meta-policy search algorithm that trains incrementally over task sequences without revisiting past tasks, advancing continual reinforcement learning.
Findings
CoMPS outperforms prior continual learning methods.
CoMPS surpasses off-policy meta-reinforcement algorithms.
Effective in challenging continuous control tasks.
Abstract
We develop a new continual meta-learning method to address challenges in sequential multi-task learning. In this setting, the agent's goal is to achieve high reward over any sequence of tasks quickly. Prior meta-reinforcement learning algorithms have demonstrated promising results in accelerating the acquisition of new tasks. However, they require access to all tasks during training. Beyond simply transferring past experience to new tasks, our goal is to devise continual reinforcement learning algorithms that learn to learn, using their experience on previous tasks to learn new tasks more quickly. We introduce a new method, continual meta-policy search (CoMPS), that removes this limitation by meta-training in an incremental fashion, over each task in a sequence, without revisiting prior tasks. CoMPS continuously repeats two subroutines: learning a new task using RL and using the…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
Taxonomy
TopicsDomain Adaptation and Few-Shot Learning · Reinforcement Learning in Robotics · Human Pose and Action Recognition
