Data-Driven Strategies for Hierarchical Predictive Control in Unknown Environments

TL;DR

This paper introduces a hierarchical learning framework that leverages stored trajectories to develop safe, data-driven control policies for unknown environments, ensuring system safety and performance in diverse tasks.

Contribution

It presents a novel method to learn generalizable control strategies from past data and integrate them into a model predictive control scheme for unknown environments.

Findings

Proven safety and feasibility of the control policy.

Successful application to robotic path planning, racing, and gaming.

Effective use of stored trajectories for new task environments.

Abstract

This article proposes a hierarchical learning architecture for safe data-driven control in unknown environments. We consider a constrained nonlinear dynamical system and assume the availability of state-input trajectories solving control tasks in different environments. In addition to task-invariant system state and input constraints, a parameterized environment model generates task-specific state constraints, which are satisfied by the stored trajectories. Our goal is to use these trajectories to find a safe and high-performing policy for a new task in a new, unknown environment. We propose using the stored data to learn generalizable control strategies. At each time step, based on a local forecast of the new task environment, the learned strategy consists of a target region in the state space and input constraints to guide the system evolution to the target region. These target regions are used as terminal sets by a low-level model predictive controller. We show how to i) design the target sets from past data and then ii) incorporate them into a model predictive control scheme with shifting horizon that ensures safety of the closed-loop system when performing the new task. We prove the feasibility of the resulting control policy, and apply the proposed method to robotic path planning, racing, and computer game applications.