SocRATES: Towards Automated Scenario-based Testing of Social Navigation Algorithms
Shashank Rao Marpally, Pranav Goyal, Harold Soh

TL;DR
This paper introduces an automated pipeline leveraging large language models to generate realistic, scenario-based social navigation tests, enabling more comprehensive evaluation of robot social competence in human environments.
Contribution
It presents a novel, automated method for creating detailed social navigation scenarios using LLMs, reducing manual effort and enhancing evaluation realism.
Findings
Pipeline produces realistic scenarios
Significantly improves scenario translation quality
Initial positive feedback from social navigation experts
Abstract
Current social navigation methods and benchmarks primarily focus on proxemics and task efficiency. While these factors are important, qualitative aspects such as perceptions of a robot's social competence are equally crucial for successful adoption and integration into human environments. We propose a more comprehensive evaluation of social navigation through scenario-based testing, where specific human-robot interaction scenarios can reveal key robot behaviors. However, creating such scenarios is often labor-intensive and complex. In this work, we address this challenge by introducing a pipeline that automates the generation of context-, and location-appropriate social navigation scenarios, ready for simulation. Our pipeline transforms simple scenario metadata into detailed textual scenarios, infers pedestrian and robot trajectories, and simulates pedestrian behaviors, which enables…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsEvacuation and Crowd Dynamics · Speech and dialogue systems · Context-Aware Activity Recognition Systems
MethodsFocus
