Dialog-based Language Learning

TL;DR

This paper explores dialog-based language learning where supervision is implicit in conversational responses, demonstrating that models can learn to answer questions effectively without explicit reward signals.

Contribution

It introduces a novel lookahead model for dialog learning and shows that effective question answering can be achieved without reward-based supervision.

Findings

The lookahead model improves learning efficiency.

Models can learn to answer without explicit rewards.

Effective learning from conversational responses demonstrated.

Abstract

A long-term goal of machine learning research is to build an intelligent dialog agent. Most research in natural language understanding has focused on learning from fixed training sets of labeled data, with supervision either at the word level (tagging, parsing tasks) or sentence level (question answering, machine translation). This kind of supervision is not realistic of how humans learn, where language is both learned by, and used for, communication. In this work, we study dialog-based language learning, where supervision is given naturally and implicitly in the response of the dialog partner during the conversation. We study this setup in two domains: the bAbI dataset of (Weston et al., 2015) and large-scale question answering from (Dodge et al., 2015). We evaluate a set of baseline learning strategies on these tasks, and show that a novel model incorporating predictive lookahead is a promising approach for learning from a teacher's response. In particular, a surprising result is that it can learn to answer questions correctly without any reward-based supervision at all.