Guarantees on Robot System Performance Using Stochastic Simulation Rollouts

TL;DR

This paper introduces a method to provide finite-sample safety and performance guarantees for stochastic robotic control policies through simulation rollouts, applicable to various policies and robust to distribution shifts.

Contribution

It offers a novel, distribution-agnostic approach to certify safety and performance of robotic policies using only simulation data, including robustness to distribution shifts and multi-hypothesis policy selection.

Findings

Validates bounds in MuJoCo environments including Ant and Half-cheetah.

Demonstrates safety verification on MuJoCo's Ant robot.

Shows importance of multi-hypothesis correction for complex systems.

Abstract

We provide finite-sample performance guarantees for control policies executed on stochastic robotic systems. Given an open- or closed-loop policy and a finite set of trajectory rollouts under the policy, we bound the expected value, value-at-risk, and conditional-value-at-risk of the trajectory cost, and the probability of failure in a sparse cost setting. The bounds hold, with user-specified probability, for any policy synthesis technique and can be seen as a post-design safety certification. Generating the bounds only requires sampling simulation rollouts, without assumptions on the distribution or complexity of the underlying stochastic system. We adapt these bounds to also give a constraint satisfaction test to verify safety of the robot system. We provide a thorough analysis of the bound sensitivity to sim-to-real distribution shifts and provide results for constructing robust bounds that can tolerate some specified amount of distribution shift. Furthermore, we extend our method to apply when selecting the best policy from a set of candidates, requiring a multi-hypothesis correction. We show the statistical validity of our bounds in the Ant, Half-cheetah, and Swimmer MuJoCo environments and demonstrate our constraint satisfaction test with the Ant. Finally, using the 20 degree-of-freedom MuJoCo Shadow Hand, we show the necessity of the multi-hypothesis correction.