Improving Generalization in Reinforcement Learning Training Regimes for Social Robot Navigation

TL;DR

This paper introduces a curriculum learning approach to enhance the generalization of reinforcement learning-based social robot navigation, tested on diverse and larger environments to better reflect real-world scenarios.

Contribution

It proposes a curriculum learning method with multiple environment types and pedestrian models to improve RL generalization in social navigation tasks.

Findings

Curriculum learning improves generalization performance.

Existing methods often lack out-of-distribution evaluation.

Validation on larger, crowded environments shows better robustness.

Abstract

In order for autonomous mobile robots to navigate in human spaces, they must abide by our social norms. Reinforcement learning (RL) has emerged as an effective method to train sequential decision-making policies that are able to respect these norms. However, a large portion of existing work in the field conducts both RL training and testing in simplistic environments. This limits the generalization potential of these models to unseen environments, and the meaningfulness of their reported results. We propose a method to improve the generalization performance of RL social navigation methods using curriculum learning. By employing multiple environment types and by modeling pedestrians using multiple dynamics models, we are able to progressively diversify and escalate difficulty in training. Our results show that the use of curriculum learning in training can be used to achieve better generalization performance than previous training methods. We also show that results presented in many existing state-of-the-art RL social navigation works do not evaluate their methods outside of their training environments, and thus do not reflect their policies' failure to adequately generalize to out-of-distribution scenarios. In response, we validate our training approach on larger and more crowded testing environments than those used in training, allowing for more meaningful measurements of model performance.