EvoNav: Evolutionary Reward Function Design for Robot Navigation with Large Language Models

TL;DR

EvoNav introduces an evolutionary framework utilizing large language models to automate reward function design for robot navigation, improving policy effectiveness and efficiency over manual methods.

Contribution

The paper presents EvoNav, a novel LLM-based evolutionary approach that automates reward design with a multi-stage evaluation process, reducing manual effort and enhancing policy performance.

Findings

EvoNav outperforms manually designed rewards in navigation tasks.

The framework achieves better policies than existing reward design methods.

EvoNav reduces computational costs through staged evaluation.

Abstract

Robot navigation is a crucial task with applications to social robots in dynamic human environments. While Reinforcement Learning (RL) has shown great promise for this problem, the policy quality is highly sensitive to the specification of reward functions. Hand-crafted rewards require substantial domain expertise and embed inductive biases that are difficult to audit or adapt, limiting their effectiveness and leading to suboptimal performance. In this paper, we propose EvoNav, an evolutionary framework that automates the design of robot navigation reward functions via large language models (LLMs). To overcome prohibitively costly policy training, EvoNav evaluates each candidate proposal from the LLM via a progressive three-stage warm-up-boost procedure. EvoNav advances from analytical proxies with low-cost surrogates, such as small datasets and analytic rules, to lightweight rollouts and, finally, to full policy training, enabling computationally efficient exploration under effective feedback. Experiment results show that EvoNav produces more effective navigation policies than manually designed RL rewards and state-of-the-art reward design methods.