Stronger Generalization Guarantees for Robot Learning by Combining Generative Models and Real-World Data

TL;DR

This paper introduces a framework that combines generative models with real-world data to provide stronger guarantees for robotic policy generalization in unseen environments, validated through simulations and hardware experiments.

Contribution

The paper proposes a novel method that uses generative models as priors and updates them with real data to improve generalization guarantees in robot learning.

Findings

Outperforms prior methods in simulation tests.

Provides validated bounds on generalization in hardware experiments.

Effective in complex sensory and dynamic environments.

Abstract

We are motivated by the problem of learning policies for robotic systems with rich sensory inputs (e.g., vision) in a manner that allows us to guarantee generalization to environments unseen during training. We provide a framework for providing such generalization guarantees by leveraging a finite dataset of real-world environments in combination with a (potentially inaccurate) generative model of environments. The key idea behind our approach is to utilize the generative model in order to implicitly specify a prior over policies. This prior is updated using the real-world dataset of environments by minimizing an upper bound on the expected cost across novel environments derived via Probably Approximately Correct (PAC)-Bayes generalization theory. We demonstrate our approach on two simulated systems with nonlinear/hybrid dynamics and rich sensing modalities: (i) quadrotor navigation with an onboard vision sensor, and (ii) grasping objects using a depth sensor. Comparisons with prior work demonstrate the ability of our approach to obtain stronger generalization guarantees by utilizing generative models. We also present hardware experiments for validating our bounds for the grasping task.