Embodied Active Learning of Generative Sensor-Object Models

TL;DR

This paper introduces an embodied active learning approach for robots to autonomously learn visual features of unknown objects, using active coverage and theoretical guarantees, demonstrated on hardware and simulation.

Contribution

It presents a novel method combining active coverage, ergodic stability, and PAC-Bayes theory to enable robots to learn object representations with statistical guarantees.

Findings

Successfully learned visual features of novel objects on hardware

Extended PAC-Bayes guarantees to embodied agents

Open-source implementation available online

Abstract

When a robot encounters a novel object, how should it respond$\unicode{x2014}$what data should it collect$\unicode{x2014}$so that it can find the object in the future? In this work, we present a method for learning image features of an unknown number of novel objects. To do this, we use active coverage with respect to latent uncertainties of the novel descriptions. We apply ergodic stability and PAC-Bayes theory to extend statistical guarantees for VAEs to embodied agents. We demonstrate the method in hardware with a robotic arm; the pipeline is also implemented in a simulated environment. Algorithms and simulation are available open source, see http://sites.google.com/u.northwestern.edu/embodied-learning-hardware .