On the Effectiveness of Iterative Learning Control

TL;DR

This paper provides a theoretical analysis of iterative learning control (ILC), demonstrating its superior performance over misspecified model-based control in high-error regimes through both analysis and empirical experiments.

Contribution

It offers the first theoretical comparison of ILC and misspecified model control on LQR problems with unknown dynamics, highlighting ILC's robustness to modeling errors.

Findings

ILC has a lower suboptimality gap than MM in high-error regimes.

Empirical results show ILC significantly outperforms MM with high modeling errors.

Theoretical bounds are derived for the Ricatti equation in finite horizon settings.

Abstract

Iterative learning control (ILC) is a powerful technique for high performance tracking in the presence of modeling errors for optimal control applications. There is extensive prior work showing its empirical effectiveness in applications such as chemical reactors, industrial robots and quadcopters. However, there is little prior theoretical work that explains the effectiveness of ILC even in the presence of large modeling errors, where optimal control methods using the misspecified model (MM) often perform poorly. Our work presents such a theoretical study of the performance of both ILC and MM on Linear Quadratic Regulator (LQR) problems with unknown transition dynamics. We show that the suboptimality gap, as measured with respect to the optimal LQR controller, for ILC is lower than that for MM by higher order terms that become significant in the regime of high modeling errors. A key part of our analysis is the perturbation bounds for the discrete Ricatti equation in the finite horizon setting, where the solution is not a fixed point and requires tracking the error using recursive bounds. We back our theoretical findings with empirical experiments on a toy linear dynamical system with an approximate model, a nonlinear inverted pendulum system with misspecified mass, and a nonlinear planar quadrotor system in the presence of wind. Experiments show that ILC outperforms MM significantly, in terms of the cost of computed trajectories, when modeling errors are high.