A Mixture-of-Experts Approach to Few-Shot Task Transfer in Open-Ended Text Worlds

TL;DR

This paper presents a Mixture-of-Experts model with attention mechanisms for rapid adaptation in open-ended text worlds, enabling agents to reuse prior knowledge and learn new tasks efficiently without predefined goals.

Contribution

It introduces a novel mixture-of-experts approach with attention to combine frozen and learnable policies for few-shot task transfer in open-ended environments.

Findings

Improved zero-shot reward acquisition.

Enhanced sample efficiency in few-shot learning.

Effective reuse of prior task knowledge.

Abstract

Open-ended worlds are those in which there are no pre-specified goals or environmental reward signal. As a consequence, an agent must know how to perform a multitude of tasks. However, when a new task is presented to an agent, we expect it to be able to reuse some of what it knows from previous tasks to rapidly learn that new task. We introduce a novel technique whereby policies for different a priori known tasks are combined into a Mixture-of-Experts model with an attention mechanism across a mix of frozen and unfrozen experts. The model learns when to attend to frozen task-specific experts when appropriate and learns new experts to handle novel situations. We work in an open-ended text-based environment in which the agent is tasked with behaving like different types of character roles and must rapidly learn behaviors associated with new character role types. We show that our agent both obtains more rewards in the zero-shot setting, and discovers these rewards with greater sample efficiency in the few-shot learning settings.