Exploiting homogeneity for the optimal control of discrete-time systems: application to value iteration

TL;DR

This paper leverages homogeneity properties of discrete-time systems to develop a scalable, efficient value iteration method that provides bounds on the optimal value function, reducing computational complexity.

Contribution

It introduces a novel approach exploiting system homogeneity to construct (near-)optimal controls across the state space with fewer computations.

Findings

The proposed method guarantees lower and upper bounds on the value function.

It reduces the computational effort of value iteration.

Numerical results demonstrate the effectiveness of the new algorithm.

Abstract

To investigate solutions of (near-)optimal control problems, we extend and exploit a notion of homogeneity recently proposed in the literature for discrete-time systems. Assuming the plant dynamics is homogeneous, we first derive a scaling property of its solutions along rays provided the sequence of inputs is suitably modified. We then consider homogeneous cost functions and reveal how the optimal value function scales along rays. This result can be used to construct (near-)optimal inputs on the whole state space by only solving the original problem on a given compact manifold of a smaller dimension. Compared to the related works of the literature, we impose no conditions on the homogeneity degrees. We demonstrate the strength of this new result by presenting a new approximate scheme for value iteration, which is one of the pillars of dynamic programming. The new algorithm provides guaranteed lower and upper estimates of the true value function at any iteration and has several appealing features in terms of reduced computation. A numerical case study is provided to illustrate the proposed algorithm.