On a Probabilistic Approach for Inverse Data-Driven Optimal Control

TL;DR

This paper introduces a probabilistic method to estimate an agent's possibly non-convex cost function by observing its interactions in complex environments, using convex optimization techniques for both forward and inverse problems.

Contribution

It presents a novel convex optimization framework for inverse optimal control in stochastic, nonlinear, and non-stationary settings, integrating probabilistic descriptions from data and first principles.

Findings

Convex formulation for inverse cost estimation.

Effective algorithm demonstrated through pendulum stabilization simulations.

Applicable to complex, non-convex control problems.

Abstract

We consider the problem of estimating the possibly non-convex cost of an agent by observing its interactions with a nonlinear, non-stationary and stochastic environment. For this inverse problem, we give a result that allows to estimate the cost by solving a convex optimization problem. To obtain this result we also tackle a forward problem. This leads to the formulation of a finite-horizon optimal control problem for which we show convexity and find the optimal solution. Our approach leverages certain probabilistic descriptions that can be obtained both from data and/or from first-principles. The effectiveness of our results, which are turned in an algorithm, is illustrated via simulations on the problem of estimating the cost of an agent that is stabilizing the unstable equilibrium of a pendulum.