Data-driven Rollout for Deterministic Optimal Control

TL;DR

This paper introduces a data-driven rollout algorithm for deterministic infinite horizon optimal control problems, leveraging sampled data and extending to complex scenarios like constraints and multiagent systems.

Contribution

It proposes a novel rollout method based on value and policy iteration that applies broadly to deterministic control problems with arbitrary dynamics and spaces.

Findings

Algorithm effectively utilizes sampled data for control optimization.

Extensible to problems with trajectory constraints and multiagent systems.

Applicable to a wide range of deterministic control scenarios.

Abstract

We consider deterministic infinite horizon optimal control problems with nonnegative stage costs. We draw inspiration from learning model predictive control scheme designed for continuous dynamics and iterative tasks, and propose a rollout algorithm that relies on sampled data generated by some base policy. The proposed algorithm is based on value and policy iteration ideas, and applies to deterministic problems with arbitrary state and control spaces, and arbitrary dynamics. It admits extensions to problems with trajectory constraints, and a multiagent structure.