Keep it Simple: Data-efficient Learning for Controlling Complex Systems with Simple Models

TL;DR

LVSPC is a data-efficient control method that combines simple models with perception learning to manipulate complex systems from images, achieving high success with minimal data and real-world trials.

Contribution

The paper introduces LVSPC, a novel approach that integrates simple models, perception, and online learning for efficient control of complex systems from images.

Findings

LVSPC performs comparably to state-of-the-art RL methods with much less data.

LVSPC achieves 80% success in real robot rope manipulation after only 10 trials.

The perception system trained in simulation generalizes effectively to real-world tasks.

Abstract

When manipulating a novel object with complex dynamics, a state representation is not always available, for example for deformable objects. Learning both a representation and dynamics from observations requires large amounts of data. We propose Learned Visual Similarity Predictive Control (LVSPC), a novel method for data-efficient learning to control systems with complex dynamics and high-dimensional state spaces from images. LVSPC leverages a given simple model approximation from which image observations can be generated. We use these images to train a perception model that estimates the simple model state from observations of the complex system online. We then use data from the complex system to fit the parameters of the simple model and learn where this model is inaccurate, also online. Finally, we use Model Predictive Control and bias the controller away from regions where the simple model is inaccurate and thus where the controller is less reliable. We evaluate LVSPC on two tasks; manipulating a tethered mass and a rope. We find that our method performs comparably to state-of-the-art reinforcement learning methods with an order of magnitude less data. LVSPC also completes the rope manipulation task on a real robot with 80% success rate after only 10 trials, despite using a perception system trained only on images from simulation.