The RobotSlang Benchmark: Dialog-guided Robot Localization and Navigation
Shurjo Banerjee, Jesse Thomason, Jason J. Corso

TL;DR
RobotSlang introduces a benchmark for natural language dialog-based robot localization and navigation, enabling research on cooperative communication between humans and robots in real-world tasks.
Contribution
The paper presents RobotSlang, a new benchmark dataset with dialog-based navigation tasks, and an initial model demonstrating the feasibility of dialog-guided robot control.
Findings
A new dataset with 169 dialogs and 5k utterances for robot localization and navigation.
An initial model can follow dialog instructions to control a physical robot.
Simulation-trained agents can operate in real-world robot platforms.
Abstract
Autonomous robot systems for applications from search and rescue to assistive guidance should be able to engage in natural language dialog with people. To study such cooperative communication, we introduce Robot Simultaneous Localization and Mapping with Natural Language (RobotSlang), a benchmark of 169 natural language dialogs between a human Driver controlling a robot and a human Commander providing guidance towards navigation goals. In each trial, the pair first cooperates to localize the robot on a global map visible to the Commander, then the Driver follows Commander instructions to move the robot to a sequence of target objects. We introduce a Localization from Dialog History (LDH) and a Navigation from Dialog History (NDH) task where a learned agent is given dialog and visual observations from the robot platform as input and must localize in the global map or navigate towards the…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsMultimodal Machine Learning Applications · Speech and dialogue systems · Topic Modeling
