TL;DR
This paper introduces the task of grounding linguistic commands to navigable regions for autonomous vehicles, extending the concept of referring image segmentation to navigation, and provides a new dataset and benchmark for this purpose.
Contribution
It proposes the novel task of Referring Navigable Regions, creates the Talk2Car-RegSeg dataset with segmentation masks, and benchmarks a transformer-based model for this task.
Findings
The transformer-based model outperforms baselines on multiple metrics.
The dataset enables practical evaluation with maneuver-oriented commands.
Path planning based on RNR improves navigation accuracy.
Abstract
Humans have a natural ability to effortlessly comprehend linguistic commands such as "park next to the yellow sedan" and instinctively know which region of the road the vehicle should navigate. Extending this ability to autonomous vehicles is the next step towards creating fully autonomous agents that respond and act according to human commands. To this end, we propose the novel task of Referring Navigable Regions (RNR), i.e., grounding regions of interest for navigation based on the linguistic command. RNR is different from Referring Image Segmentation (RIS), which focuses on grounding an object referred to by the natural language expression instead of grounding a navigable region. For example, for a command "park next to the yellow sedan," RIS will aim to segment the referred sedan, and RNR aims to segment the suggested parking region on the road. We introduce a new dataset,…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
