Socially-Aware Robot Navigation Enhanced by Bidirectional Natural Language Conversations Using Large Language Models

TL;DR

This paper presents HSAC-LLM, a novel framework combining deep reinforcement learning and large language models to enable socially aware, bidirectional natural language interactions for robot navigation in shared human environments.

Contribution

It introduces a new hybrid framework that integrates LLMs with reinforcement learning to improve human-robot interaction and navigation in dynamic, shared spaces.

Findings

HSAC-LLM outperforms existing DRL methods in simulations and real-world tests.

The framework enables proactive communication with pedestrians during navigation.

Experiments demonstrate improved obstacle avoidance and interaction quality.

Abstract

Robot navigation is crucial across various domains, yet traditional methods focus on efficiency and obstacle avoidance, often overlooking human behavior in shared spaces. With the rise of service robots, socially aware navigation has gained prominence. However, existing approaches primarily predict pedestrian movements or issue alerts, lacking true human-robot interaction. We introduce Hybrid Soft Actor-Critic with Large Language Model (HSAC-LLM), a novel framework for socially aware navigation. By integrating deep reinforcement learning with large language models, HSAC-LLM enables bidirectional natural language interactions, predicting both continuous and discrete navigation actions. When potential collisions arise, the robot proactively communicates with pedestrians to determine avoidance strategies. Experiments in 2D simulation, Gazebo, and real-world environments demonstrate that HSAC-LLM outperforms state-of-the-art DRL methods in interaction, navigation, and obstacle avoidance. This paradigm advances effective human-robot interactions in dynamic settings. Videos are available at https://hsacllm.github.io/.