DATA-DRIVEN PRONTO: a Model-free Solution for Numerical Optimal Control

TL;DR

This paper introduces DATA-DRIVEN PRONTO, a model-free, iterative algorithm for data-driven numerical optimal control of unknown nonlinear systems, leveraging local trajectory perturbations and convergence analysis.

Contribution

It proposes a novel model-free optimal control method that iteratively refines solutions using data-driven linearizations from perturbed trajectories.

Findings

Algorithm converges locally to an optimal solution within a bounded radius.

Effective in controlling an underactuated robot.

Demonstrates theoretical convergence and practical applicability.

Abstract

This article addresses the problem of data-driven numerical optimal control for unknown nonlinear systems. In our scenario, we suppose to have the possibility of performing multiple experiments (or simulations) on the system. Experiments are performed by relying on a data-driven tracking controller able to steer the system towards a desired reference. Our proposed DATA-DRIVEN PRONTO algorithm iteratively refines a tentative solution of the optimal control problem by computing an approximate descent direction via a local trajectory perturbation. At each iteration, multiple trajectories are gathered by perturbing the current trajectory with a suitable dither signal, and then used to obtain a data-driven, time-varying linearization. The exploration is guided by the tracking controller, so that perturbed trajectories are obtained in closed loop. We show local convergence of DATA-DRIVEN PRONTO to a ball about an isolated optimal solution, whose radius depends on the amplitude of the dither signal. We corroborate the theoretical results by applying it to an underactuated robot.