How to Avoid Being Eaten by a Grue: Structured Exploration Strategies for Textual Worlds
Prithviraj Ammanabrolu, Ethan Tien, Matthew Hausknecht, Mark O. Riedl

TL;DR
This paper presents Q*BERT and MC!Q*BERT, innovative agents that utilize knowledge graphs and intrinsic motivation to improve exploration and overcome bottlenecks in text-based games, achieving state-of-the-art results including passing the Grue in Zork.
Contribution
Introduction of Q*BERT and MC!Q*BERT agents that enhance exploration in text games through knowledge graphs and intrinsic motivation, surpassing previous methods.
Findings
Outperforms state-of-the-art on nine text games
First agent to pass the Grue in Zork
Demonstrates effectiveness of knowledge-graph-based exploration
Abstract
Text-based games are long puzzles or quests, characterized by a sequence of sparse and potentially deceptive rewards. They provide an ideal platform to develop agents that perceive and act upon the world using a combinatorially sized natural language state-action space. Standard Reinforcement Learning agents are poorly equipped to effectively explore such spaces and often struggle to overcome bottlenecks---states that agents are unable to pass through simply because they do not see the right action sequence enough times to be sufficiently reinforced. We introduce Q*BERT, an agent that learns to build a knowledge graph of the world by answering questions, which leads to greater sample efficiency. To overcome bottlenecks, we further introduce MC!Q*BERT an agent that uses an knowledge-graph-based intrinsic motivation to detect bottlenecks and a novel exploration strategy to efficiently…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsTopic Modeling · Natural Language Processing Techniques · Artificial Intelligence in Games
