Path Integral Control and State Dependent Feedback

TL;DR

This paper develops a generalized path integral control framework for state-dependent feedback, linking control quality to importance sampling efficiency and demonstrating how optimal control minimizes variance in estimates.

Contribution

It introduces a generalized formula for path integral control to construct state-dependent feedback controllers and reveals a novel relation between control quality and importance sampling efficiency.

Findings

Generalized path integral control formula for state-dependent feedback

Better control improves importance sampling efficiency

Optimal control achieves zero-variance estimates

Abstract

In this paper we address the problem to compute state dependent feedback controls for path integral control problems. To this end we generalize the path integral control formula and utilize this to construct parameterized state dependent feedback controllers. In addition, we show a novel relation between control and importance sampling: better control, in terms of control cost, yields more efficient importance sampling, in terms of effective sample size. The optimal control provides a zero-variance estimate.