Robust Robotic Control from Pixels using Contrastive Recurrent State-Space Models

TL;DR

This paper introduces a contrastive recurrent state-space model for robotic control from pixel observations, demonstrating robustness to distractions and achieving state-of-the-art results on a challenging benchmark.

Contribution

It proposes a novel contrastive recurrent latent dynamics model that improves robustness in pixel-based robotic control tasks with distractions.

Findings

Outperforms bisimulation methods in distracted environments

Achieves state-of-the-art results on the Distracting Control Suite

Robust control despite camera and background distractions

Abstract

Modeling the world can benefit robot learning by providing a rich training signal for shaping an agent's latent state space. However, learning world models in unconstrained environments over high-dimensional observation spaces such as images is challenging. One source of difficulty is the presence of irrelevant but hard-to-model background distractions, and unimportant visual details of task-relevant entities. We address this issue by learning a recurrent latent dynamics model which contrastively predicts the next observation. This simple model leads to surprisingly robust robotic control even with simultaneous camera, background, and color distractions. We outperform alternatives such as bisimulation methods which impose state-similarity measures derived from divergence in future reward or future optimal actions. We obtain state-of-the-art results on the Distracting Control Suite, a challenging benchmark for pixel-based robotic control.