A short methodological review on social robot navigation benchmarking

TL;DR

This paper reviews recent trends in benchmarking methods for social robot navigation, highlighting the lack of standardization and analyzing 85 papers to understand current practices and challenges.

Contribution

It provides a focused review of benchmarking approaches in social robot navigation from 2020 to 2025, identifying key metrics, algorithms, and evaluation methods used.

Findings

Diverse metrics are used for benchmarking social robot navigation.

Various algorithms are employed in benchmarking studies.

Human surveys are increasingly incorporated into evaluation processes.

Abstract

Social Robot Navigation is the skill that allows robots to move efficiently in human-populated environments while ensuring safety, comfort, and trust. Unlike other areas of research, the scientific community has not yet achieved an agreement on how Social Robot Navigation should be benchmarked. This is notably important, as the lack of a de facto standard to benchmark Social Robot Navigation can hinder the progress of the field and may lead to contradicting conclusions. Motivated by this gap, we contribute with a short review focused exclusively on benchmarking trends in the period from January 2020 to July 2025. Of the 130 papers identified by our search using IEEE Xplore, we analysed the 85 papers that met the criteria of the review. This review addresses the metrics used in the literature for benchmarking purposes, the algorithms employed in such benchmarks, the use of human surveys for benchmarking, and how conclusions are drawn from the benchmarking results, when applicable.