Increasing Information for Model Predictive Control with Semi-Markov Decision Processes

TL;DR

This paper enhances Learning-Based Model Predictive Control by integrating Semi-Markov Decision Processes to increase information gathering, thereby reducing sample complexity and improving learning efficiency in dynamical systems.

Contribution

It introduces a temporal abstraction framework using Semi-Markov Decision Processes to augment information collection in model predictive control.

Findings

Increased total information from data within fixed sampling budgets.

Reduced sample complexity in learning-based control.

Enhanced exploration capabilities through temporal abstraction.

Abstract

Recent works in Learning-Based Model Predictive Control of dynamical systems show impressive sample complexity performances using criteria from Information Theory to accelerate the learning procedure. However, the sequential exploration opportunities are limited by the system local state, restraining the amount of information of the observations from the current exploration trajectory. This article resolves this limitation by introducing temporal abstraction through the framework of Semi-Markov Decision Processes. The framework increases the total information of the gathered data for a fixed sampling budget, thus reducing the sample complexity.