Successive convex optimization for transformer encoder model predictive control

TL;DR

This paper introduces a novel data-driven MPC framework using transformer encoders, employing successive convex programming to handle nonconvexities and ensure convergence to local optima.

Contribution

It develops a new SCP-based method for transformer-based MPC, guaranteeing recursive feasibility and convergence, with application to nonlinear control problems.

Findings

Guarantees recursive feasibility and convergence of the SCP iterations.

Transforms nonconvex transformer components into convex approximations.

Successfully applied to a benchmark nonlinear control problem.

Abstract

We propose a data-driven Model Predictive Control (MPC) framework that employs a transformer encoder to generate multi-step predictions. To handle the nonconvex attention mechanism, we derive difference of convex (DC) representations of the transformer encoder components and embed them in a successive convex programming (SCP) iteration. Recursive feasibility and convergence of the SCP iterates are guaranteed, and each iterate yields a solution estimate satisfying the problem constraints. Under mild assumptions, the SCP iteration converges to a locally optimal solution of the MPC problem. The approach is illustrated on a benchmark nonlinear control problem.