Symmetry reduction for dynamic programming

TL;DR

This paper introduces a symmetry-based reduction technique for dynamic programming in optimal control, significantly lowering computational complexity by exploiting system symmetries without needing detailed algebraic manipulations.

Contribution

It provides a general, easy-to-apply framework for symmetry reduction in dynamic programming that works with minimal knowledge of the system beyond its symmetries.

Findings

Reduces dimensionality of dynamic programming iterations

Applicable to systems with continuous or discrete symmetry groups

Demonstrated on two six-dimensional control problems

Abstract

We present a method of exploiting symmetries of discrete-time optimal control problems to reduce the dimensionality of dynamic programming iterations. The results are derived for systems with continuous state variables, and can be applied to systems with continuous or discrete symmetry groups. We prove that symmetries of the state update equation and stage costs induce corresponding symmetries of the optimal cost function and the optimal policies. We then provide a general framework for computing the optimal cost function based on gridding a space of lower dimension than the original state space. This method does not require algebraic manipulation of the state update equations; it only requires knowledge of the symmetries that the state update equations possess. Since the method can be performed without any knowledge of the state update map beyond being able to evaluate it and verify its symmetries, this enables the method to be applied in a wide range of application problems. We illustrate these results on two six-dimensional optimal control problems that are computationally difficult to solve by dynamic programming without symmetry reduction.