Dialogue Object Search
Monica Roy, Kaiyu Zheng, Jason Liu, Stefanie Tellex

TL;DR
This paper introduces the dialogue object search task, where robots communicate with humans via dialogue and video to locate objects in environments, advancing collaborative robot capabilities.
Contribution
It defines a new task combining dialogue and visual search, providing initial analysis and discussing future challenges for developing intelligent collaborative robots.
Findings
Collected pilot data demonstrating task feasibility
Analyzed examples of robot-human dialogue interactions
Outlined key challenges for future research in dialogue-based search
Abstract
We envision robots that can collaborate and communicate seamlessly with humans. It is necessary for such robots to decide both what to say and how to act, while interacting with humans. To this end, we introduce a new task, dialogue object search: A robot is tasked to search for a target object (e.g. fork) in a human environment (e.g., kitchen), while engaging in a "video call" with a remote human who has additional but inexact knowledge about the target's location. That is, the robot conducts speech-based dialogue with the human, while sharing the image from its mounted camera. This task is challenging at multiple levels, from data collection, algorithm and system development,to evaluation. Despite these challenges, we believe such a task blocks the path towards more intelligent and collaborative robots. In this extended abstract, we motivate and introduce the dialogue object search…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsMultimodal Machine Learning Applications · Social Robot Interaction and HRI · Speech and dialogue systems
