COBRA: Data-Efficient Model-Based RL through Unsupervised Object Discovery and Curiosity-Driven Exploration

TL;DR

COBRA is a modular, data-efficient model-based reinforcement learning approach that leverages unsupervised object discovery and curiosity-driven exploration to improve robustness and sample efficiency in continuous control tasks.

Contribution

It introduces a novel unsupervised, object-based modeling framework combined with intrinsic motivation for exploration, enabling rapid learning of diverse tasks without supervision.

Findings

Achieves high data efficiency in continuous control environments.

Demonstrates robustness to task-irrelevant perturbations.

Performs well on structured hold-out tests of policy robustness.

Abstract

Data efficiency and robustness to task-irrelevant perturbations are long-standing challenges for deep reinforcement learning algorithms. Here we introduce a modular approach to addressing these challenges in a continuous control environment, without using hand-crafted or supervised information. Our Curious Object-Based seaRch Agent (COBRA) uses task-free intrinsically motivated exploration and unsupervised learning to build object-based models of its environment and action space. Subsequently, it can learn a variety of tasks through model-based search in very few steps and excel on structured hold-out tests of policy robustness.