Constrained Meta Agnostic Reinforcement Learning

TL;DR

This paper introduces C-MAML, a meta-learning algorithm that integrates constrained optimization to enable rapid adaptation to new tasks while respecting environmental constraints, demonstrated on robotic locomotion tasks.

Contribution

The paper presents C-MAML, a novel meta-learning framework that incorporates task-specific constraints during training to improve safety and adaptability in real-world environments.

Findings

C-MAML achieves faster adaptation to new tasks with constraints.

It produces safer initial policies for task learning.

Demonstrated effectiveness on simulated robotic locomotion tasks.

Abstract

Meta-Reinforcement Learning (Meta-RL) aims to acquire meta-knowledge for quick adaptation to diverse tasks. However, applying these policies in real-world environments presents a significant challenge in balancing rapid adaptability with adherence to environmental constraints. Our novel approach, Constraint Model Agnostic Meta Learning (C-MAML), merges meta learning with constrained optimization to address this challenge. C-MAML enables rapid and efficient task adaptation by incorporating task-specific constraints directly into its meta-algorithm framework during the training phase. This fusion results in safer initial parameters for learning new tasks. We demonstrate the effectiveness of C-MAML in simulated locomotion with wheeled robot tasks of varying complexity, highlighting its practicality and robustness in dynamic environments.