Ask Before You Act: Generalising to Novel Environments by Asking Questions
Ross Murphy, Sergey Mosesov, Javier Leguina Peral, Thymo ter Doest

TL;DR
This paper explores how reinforcement learning agents can improve their ability to generalize in new environments by learning to ask yes-no questions to an oracle, thereby gaining useful information and reasoning about environment dynamics.
Contribution
The study introduces a method for RL agents to learn natural language question-asking as a tool for better environment understanding and generalization in temporally-extended tasks.
Findings
Agents asking questions outperform baselines in novel environments.
Question asking enhances understanding of environment dynamics.
Grounded language understanding enables relevant question generation.
Abstract
Solving temporally-extended tasks is a challenge for most reinforcement learning (RL) algorithms [arXiv:1906.07343]. We investigate the ability of an RL agent to learn to ask natural language questions as a tool to understand its environment and achieve greater generalisation performance in novel, temporally-extended environments. We do this by endowing this agent with the ability of asking "yes-no" questions to an all-knowing Oracle. This allows the agent to obtain guidance regarding the task at hand, while limiting the access to new information. To study the emergence of such natural language questions in the context of temporally-extended tasks we first train our agent in a Mini-Grid environment. We then transfer the trained agent to a different, harder environment. We observe a significant increase in generalisation performance compared to a baseline agent unable to ask questions.…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsReinforcement Learning in Robotics · Speech and dialogue systems · Multimodal Machine Learning Applications
