Distributionally Robust Imitation Learning: Layered Control Architecture for Certifiable Autonomy

TL;DR

This paper introduces a layered control architecture that combines distributionally robust imitation learning methods to enable certifiable autonomy in uncertain dynamical systems.

Contribution

It proposes the Distributionally Robust Imitation Policy (DRIP) architecture that integrates TaSIL and ext{ extlone}DRAC, providing certifiable guarantees for autonomous systems.

Findings

The layered control architecture guarantees certificates for the entire control pipeline.

Integration of learning-based perception with certifiable decision-making is feasible.

The approach enhances robustness against distribution shifts and uncertainties.

Abstract

Imitation learning (IL) enables autonomous behavior by learning from expert demonstrations. While more sample-efficient than comparative alternatives like reinforcement learning, IL is sensitive to compounding errors induced by distribution shifts. There are two significant sources of distribution shifts when using IL-based feedback laws on systems: distribution shifts caused by policy error and distribution shifts due to exogenous disturbances and endogenous model errors due to lack of learning. Our previously developed approaches, Taylor Series Imitation Learning (TaSIL) and $\mathcal{L}_1$ -Distributionally Robust Adaptive Control (\ellonedrac), address the challenge of distribution shifts in complementary ways. While TaSIL offers robustness against policy error-induced distribution shifts, \ellonedrac offers robustness against distribution shifts due to aleatoric and epistemic uncertainties. To enable certifiable IL for learned and/or uncertain dynamical systems, we formulate \textit{Distributionally Robust Imitation Policy (DRIP)} architecture, a Layered Control Architecture (LCA) that integrates TaSIL and~\ellonedrac. By judiciously designing individual layer-centric input and output requirements, we show how we can guarantee certificates for the entire control pipeline. Our solution paves the path for designing fully certifiable autonomy pipelines, by integrating learning-based components, such as perception, with certifiable model-based decision-making through the proposed LCA approach.