A Convex Optimization Approach to Model-Free Inverse Optimal Control with Provable Convergence

TL;DR

This paper introduces a convex optimization framework for model-free inverse optimal control in continuous-time LQR systems, providing the first convergence guarantees and demonstrating superior speed and accuracy over existing methods.

Contribution

It reformulates the joint system and cost estimation problem into a convex second-order cone program with provable convergence, enabling efficient and reliable IOC solutions.

Findings

Achieves a sublinear convergence rate of O(1/k).

Demonstrates an order-of-magnitude faster convergence than benchmarks.

Shows improved accuracy and robustness in simulations.

Abstract

Inverse Optimal Control (IOC) aims to infer the underlying cost functional of an agent from observations of its expert behavior. This paper focuses on the IOC problem within the continuous-time linear quadratic regulator framework, specifically addressing the challenging scenario where both the system dynamics and the cost functional weighting matrices are unknown. A significant limitation of existing methods for this joint estimation problem is the lack of rigorous theoretical guarantees on the convergence and convergence rate of their optimization algorithms, which restricts their application in safety-critical systems. To bridge this theoretical gap, we propose an analytical framework for IOC that provides such guarantees. The core contribution lies in the equivalent reformulation of this non-convex problem of jointly estimating system and cost parameters into a convex second-order cone programming problem. Building on this transformation, we design an efficient iterative solver based on the block successive upper-bound minimization algorithm. We rigorously prove that the proposed algorithm achieves a sublinear convergence rate of $\mathcal{O}(1/k)$. To the best of our knowledge, this is the first solution for the model-free IOC problem that comes with an explicit convergence rate guarantee. Finally, comparative simulation experiments against a state-of-the-art benchmark algorithm validate the superiority of our proposed method. The results demonstrate that our algorithm achieves an order-of-magnitude improvement in convergence speed while also exhibiting significant advantages in reconstruction accuracy and robustness.