Safe MPC Alignment with Human Directional Feedback

TL;DR

This paper introduces a novel certifiable method for robots to learn safety constraints in MPC from human directional feedback, ensuring safety and efficiency in real-world tasks with minimal feedback.

Contribution

It is the first approach to learn safety constraints from human feedback, providing certifiability and efficiency in safety-critical robot control.

Findings

Successfully learned safety constraints with tens of human corrections.

Validated on simulation games and real-world robot tasks.

Demonstrated efficacy and efficiency of the method.

Abstract

In safety-critical robot planning or control, manually specifying safety constraints or learning them from demonstrations can be challenging. In this article, we propose a certifiable alignment method for a robot to learn a safety constraint in its model predictive control (MPC) policy from human online directional feedback. To our knowledge, it is the first method to learn safety constraints from human feedback. The proposed method is based on an empirical observation: human directional feedback, when available, tends to guide the robot toward safer regions. The method only requires the direction of human feedback to update the learning hypothesis space. It is certifiable, providing an upper bound on the total number of human feedback in the case of successful learning, or declaring the hypothesis misspecification, i.e., the true safety constraint cannot be found within the specified hypothesis space. We evaluated the proposed method in numerical examples and user studies with two simulation games. Additionally, we tested the proposed method on a real-world Franka robot arm performing mobile water-pouring tasks. The results demonstrate the efficacy and efficiency of our method, showing that it enables a robot to successfully learn safety constraints with a small handful (tens) of human directional corrections.