Reinforced Natural Language Interfaces via Entropy Decomposition

TL;DR

This paper introduces a reinforcement learning approach that decomposes language uncertainty into structural and functional components, enabling conversational agents to adapt to new tasks and learn effective communication protocols efficiently.

Contribution

It proposes a novel entropy decomposition method combined with reinforcement learning for adaptive natural language interfaces, improving task-specific communication.

Findings

Effective adaptation to unseen tasks demonstrated in experiments

Agents learn succinct, helpful communication protocols

Method outperforms baseline approaches in test scenarios

Abstract

In this paper, we study the technical problem of developing conversational agents that can quickly adapt to unseen tasks, learn task-specific communication tactics, and help listeners finish complex, temporally extended tasks. We find that the uncertainty of language learning can be decomposed to an entropy term and a mutual information term, corresponding to the structural and functional aspect of language, respectively. Combined with reinforcement learning, our method automatically requests human samples for training when adapting to new tasks and learns communication protocols that are succinct and helpful for task completion. Human and simulation test results on a referential game and a 3D navigation game prove the effectiveness of the proposed method.