MPC with Sensor-Based Online Cost Adaptation

TL;DR

This paper presents a novel MPC approach that uses a neural network to adapt the cost function online based on sensory inputs, enabling real-time, safe, and robust robot control with high-dimensional visual data.

Contribution

It introduces a neural network-based cost adaptation method for MPC that handles high-dimensional sensor data without solving non-convex problems online.

Findings

Efficiently solves complex non-convex problems with visual inputs

Demonstrates robustness to external disturbances

Enables real-time safe robot control

Abstract

Model predictive control is a powerful tool to generate complex motions for robots. However, it often requires solving non-convex problems online to produce rich behaviors, which is computationally expensive and not always practical in real time. Additionally, direct integration of high dimensional sensor data (e.g. RGB-D images) in the feedback loop is challenging with current state-space methods. This paper aims to address both issues. It introduces a model predictive control scheme, where a neural network constantly updates the cost function of a quadratic program based on sensory inputs, aiming to minimize a general non-convex task loss without solving a non-convex problem online. By updating the cost, the robot is able to adapt to changes in the environment directly from sensor measurement without requiring a new cost design. Furthermore, since the quadratic program can be solved efficiently with hard constraints, a safe deployment on the robot is ensured. Experiments with a wide variety of reaching tasks on an industrial robot manipulator demonstrate that our method can efficiently solve complex non-convex problems with high-dimensional visual sensory inputs, while still being robust to external disturbances.