Zero-shot Task Adaptation using Natural Language

TL;DR

This paper introduces a method for zero-shot task adaptation that combines demonstrations and natural language descriptions to enable agents to perform new tasks without additional training.

Contribution

The paper proposes LARVA, a novel approach that uses demonstrations and language to adapt reward and value functions for zero-shot task transfer.

Findings

Achieves over 95% success with template-based descriptions

Achieves over 70% success with free-form natural language

Demonstrates effective zero-shot adaptation across diverse tasks

Abstract

Imitation learning and instruction-following are two common approaches to communicate a user's intent to a learning agent. However, as the complexity of tasks grows, it could be beneficial to use both demonstrations and language to communicate with an agent. In this work, we propose a novel setting where an agent is given both a demonstration and a description, and must combine information from both the modalities. Specifically, given a demonstration for a task (the source task), and a natural language description of the differences between the demonstrated task and a related but different task (the target task), our goal is to train an agent to complete the target task in a zero-shot setting, that is, without any demonstrations for the target task. To this end, we introduce Language-Aided Reward and Value Adaptation (LARVA) which, given a source demonstration and a linguistic description of how the target task differs, learns to output a reward / value function that accurately describes the target task. Our experiments show that on a diverse set of adaptations, our approach is able to complete more than 95% of target tasks when using template-based descriptions, and more than 70% when using free-form natural language.