Human-in-the-Loop Robot Planning with Non-Contextual Bandit Feedback

TL;DR

This paper introduces a semi-supervised Bayesian Optimization approach for robot trajectory planning in human-populated environments, effectively using non-contextual human feedback to optimize satisfaction while ensuring safety and feasibility.

Contribution

It proposes a novel combination of autoencoder-based dimensionality reduction and biased Bayesian Optimization to efficiently plan human-aware robot trajectories with minimal human feedback.

Findings

Efficiently finds collision-free, human-satisfactory trajectories

Reduces high-dimensional planning to low-dimensional latent space

Demonstrates effectiveness in diverse human scenarios

Abstract

In this paper, we consider a robot navigation problem in environments populated by humans. The goal is to determine collision-free and dynamically feasible trajectories that also maximize human satisfaction. This is because they may drive the robot close to humans that need help with their work or because they may keep the robot away from humans when it can interfere with human sight or work. In practice, human satisfaction is subjective and hard to describe mathematically. As a result, the planning problem we consider in this paper may lack important contextual information. To address this challenge, we propose a semi-supervised Bayesian Optimization (BO) method to design globally optimal robot trajectories using non-contextual bandit human feedback in the form of complaints or satisfaction ratings that express how satisfactory a trajectory is, without revealing the reason. Since trajectory planning is typically a high-dimensional optimization problem in the space of waypoints that define a trajectory, BO may require prohibitively many queries for human feedback to return a good solution. To this end, we use an autoencoder to reduce the high-dimensional problem space into a low dimensional latent space, which we update using human feedback. Moreover, we improve the exploration efficiency of BO by biasing the search for new trajectories towards dynamically feasible and collision-free trajectories obtained using off-the-shelf motion planners. We demonstrate the efficiency of our proposed trajectory planning method in a scenario with humans that have diversified and unknown demands.