Context-Specific Validation of Data-Driven Models

TL;DR

This paper introduces a context-specific validation framework for data-driven models in robotics, ensuring models perform reliably in real-world control tasks without prior system knowledge.

Contribution

It presents a novel validation method based on distance measures and active sampling, tailored to relevant behaviors for specific control purposes.

Findings

Framework effectively validates models in simulations

Active sampling provides sample-efficient bounds

Applicable to real-world system models

Abstract

With an increasing use of data-driven models to control robotic systems, it has become important to develop a methodology for validating such models before they can be deployed to design a controller for the actual system. Specifically, it must be ensured that the controller designed for a learned model would perform as expected on the actual physical system. We propose a context-specific validation framework to quantify the quality of a learned model based on a distance measure between the closed-loop actual system and the learned model. We then propose an active sampling scheme to compute a probabilistic upper bound on this distance in a sample-efficient manner. The proposed framework validates the learned model against only those behaviors of the system that are relevant for the purpose for which we intend to use this model, and does not require any a priori knowledge of the system dynamics. Several simulations illustrate the practicality of the proposed framework for validating the models of real-world systems, and consequently, for controller synthesis.