Markov Chain Monte Carlo for Koopman-based Optimal Control: Technical Report

TL;DR

This paper introduces an MCMC algorithm utilizing Gibbs sampling with parallel tempering to efficiently solve nonlinear optimal control problems by leveraging Koopman operator theory, demonstrated through a video game example.

Contribution

It presents a novel MCMC approach that exploits Koopman linearity for scalable nonlinear control, enabling efficient computation for high-dimensional systems.

Findings

Efficient control solution for high-dimensional nonlinear systems.

Significant computational savings using Koopman-based MCMC.

Successful demonstration on a video game control problem.

Abstract

We propose a Markov Chain Monte Carlo (MCMC) algorithm based on Gibbs sampling with parallel tempering to solve nonlinear optimal control problems. The algorithm is applicable to nonlinear systems with dynamics that can be approximately represented by a finite dimensional Koopman model, potentially with high dimension. This algorithm exploits linearity of the Koopman representation to achieve significant computational saving for large lifted states. We use a video-game to illustrate the use of the method.