Distributionally Robust Path Integral Control

TL;DR

This paper introduces Distributionally Robust Path Integral (DRPI), a novel control method that enhances policy robustness in stochastic control problems with limited data by leveraging distributionally robust optimization within the path integral framework.

Contribution

The paper develops DRPI, integrating distributionally robust optimization with path integral control, providing theoretical guarantees and improved performance over risk-neutral policies.

Findings

DRPI outperforms risk-neutral PIC in uncertain diffusion scenarios.

Theoretical guarantees align with risk-sensitive control parameters.

DRPI offers a robust alternative when diffusion process knowledge is limited.

Abstract

We consider a continuous-time continuous-space stochastic optimal control problem, where the controller lacks exact knowledge of the underlying diffusion process, relying instead on a finite set of historical disturbance trajectories. In situations where data collection is limited, the controller synthesized from empirical data may exhibit poor performance. To address this issue, we introduce a novel approach named Distributionally Robust Path Integral (DRPI). The proposed method employs distributionally robust optimization (DRO) to robustify the resulting policy against the unknown diffusion process. Notably, the DRPI scheme shows similarities with risk-sensitive control, which enables us to utilize the path integral control (PIC) framework as an efficient solution scheme. We derive theoretical performance guarantees for the DRPI scheme, which closely aligns with selecting a risk parameter in risk-sensitive control. We validate the efficacy of our scheme and showcase its superiority when compared to risk-neutral PIC policies in the absence of the true diffusion process.