Fine-Tuning of Neural Network Approximate MPC without Retraining via Bayesian Optimization

TL;DR

This paper introduces a Bayesian optimization approach to fine-tune neural network-based approximate MPC policies without retraining, enabling efficient adaptation to new systems and cost functions, demonstrated on hardware control tasks.

Contribution

It presents a novel method combining Bayesian optimization with AMPC to automatically and efficiently tune control policies without retraining, improving practicality for complex systems.

Findings

Superior hardware performance compared to nominal AMPC

Minimal experimentation needed for adaptation

Effective tuning on high-dimensional control problems

Abstract

Approximate model-predictive control (AMPC) aims to imitate an MPC's behavior with a neural network, removing the need to solve an expensive optimization problem at runtime. However, during deployment, the parameters of the underlying MPC must usually be fine-tuned. This often renders AMPC impractical as it requires repeatedly generating a new dataset and retraining the neural network. Recent work addresses this problem by adapting AMPC without retraining using approximated sensitivities of the MPC's optimization problem. Currently, this adaption must be done by hand, which is labor-intensive and can be unintuitive for high-dimensional systems. To solve this issue, we propose using Bayesian optimization to tune the parameters of AMPC policies based on experimental data. By combining model-based control with direct and local learning, our approach achieves superior performance to nominal AMPC on hardware, with minimal experimentation. This allows automatic and data-efficient adaptation of AMPC to new system instances and fine-tuning to cost functions that are difficult to directly implement in MPC. We demonstrate the proposed method in hardware experiments for the swing-up maneuver on an inverted cartpole and yaw control of an under-actuated balancing unicycle robot, a challenging control problem.