ACDC: Adaptive Curriculum Planning with Dynamic Contrastive Control for Goal-Conditioned Reinforcement Learning in Robotic Manipulation

TL;DR

This paper introduces ACDC, a novel reinforcement learning framework for robotic manipulation that combines adaptive curriculum planning with dynamic contrastive control to improve learning efficiency and success rates.

Contribution

The paper presents a new integrated approach, ACDC, that dynamically balances exploration and exploitation and guides experience selection for better robotic manipulation learning.

Findings

ACDC outperforms state-of-the-art methods in sample efficiency.

ACDC achieves higher final task success rates.

The approach demonstrates robustness across diverse manipulation tasks.

Abstract

Goal-conditioned reinforcement learning has shown considerable potential in robotic manipulation; however, existing approaches remain limited by their reliance on prioritizing collected experience, resulting in suboptimal performance across diverse tasks. Inspired by human learning behaviors, we propose a more comprehensive learning paradigm, ACDC, which integrates multidimensional Adaptive Curriculum (AC) Planning with Dynamic Contrastive (DC) Control to guide the agent along a well-designed learning trajectory. More specifically, at the planning level, the AC component schedules the learning curriculum by dynamically balancing diversity-driven exploration and quality-driven exploitation based on the agent's success rate and training progress. At the control level, the DC component implements the curriculum plan through norm-constrained contrastive learning, enabling magnitude-guided experience selection aligned with the current curriculum focus. Extensive experiments on challenging robotic manipulation tasks demonstrate that ACDC consistently outperforms the state-of-the-art baselines in both sample efficiency and final task success rate.