On the Computational Consequences of Cost Function Design in Nonlinear Optimal Control

TL;DR

This paper investigates how the design of cost functions in nonlinear optimal control influences computational complexity and stability, revealing fundamental differences based on system phase behavior and providing bounds for control algorithms.

Contribution

It offers a theoretical analysis linking cost function design to computational effort and stability, highlighting the impact of system phase properties on control complexity.

Findings

Minimum-phase outputs reduce computational complexity

Non-minimum-phase outputs increase prediction horizon and iterations

Reinforcement learning results align with theoretical predictions

Abstract

Optimal control is an essential tool for stabilizing complex nonlinear systems. However, despite the extensive impacts of methods such as receding horizon control, dynamic programming and reinforcement learning, the design of cost functions for a particular system often remains a heuristic-driven process of trial and error. In this paper we seek to gain insights into how the choice of cost function interacts with the underlying structure of the control system and impacts the amount of computation required to obtain a stabilizing controller. We treat the cost design problem as a two-step process where the designer specifies outputs for the system that are to be penalized and then modulates the relative weighting of the inputs and the outputs in the cost. To characterize the computational burden associated to obtaining a stabilizing controller with a particular cost, we bound the prediction horizon required by receding horizon methods and the number of iterations required by dynamic programming methods to meet this requirement. Our theoretical results highlight a qualitative separation between what is possible, from a design perspective, when the chosen outputs induce either minimum-phase or non-minimum-phase behavior. Simulation studies indicate that this separation also holds for modern reinforcement learning methods.