Leveraging Neural Network Gradients within Trajectory Optimization for Proactive Human-Robot Interactions

TL;DR

This paper introduces a novel framework that integrates neural network-based human behavior prediction with trajectory optimization, enabling robots to plan safer, more efficient, and proactive interactions in crowded environments.

Contribution

It presents a method that fuses neural network gradients with trajectory optimization to explicitly model human-robot interaction dynamics during motion planning.

Findings

Safer and more efficient navigation in crowded scenarios.

Proactive behaviors like waiting for pedestrians.

Outperforms existing planning methods in safety and efficiency.

Abstract

To achieve seamless human-robot interactions, robots need to intimately reason about complex interaction dynamics and future human behaviors within their motion planning process. However, there is a disconnect between state-of-the-art neural network-based human behavior models and robot motion planners -- either the behavior models are limited in their consideration of downstream planning or a simplified behavior model is used to ensure tractability of the planning problem. In this work, we present a framework that fuses together the interpretability and flexibility of trajectory optimization (TO) with the predictive power of state-of-the-art human trajectory prediction models. In particular, we leverage gradient information from data-driven prediction models to explicitly reason about human-robot interaction dynamics within a gradient-based TO problem. We demonstrate the efficacy of our approach in a multi-agent scenario whereby a robot is required to safely and efficiently navigate through a crowd of up to ten pedestrians. We compare against a variety of planning methods, and show that by explicitly accounting for interaction dynamics within the planner, our method offers safer and more efficient behaviors, even yielding proactive and nuanced behaviors such as waiting for a pedestrian to pass before moving.