Answerer in Questioner's Mind: Information Theoretic Approach to Goal-Oriented Visual Dialog

TL;DR

This paper introduces AQM, an information theoretic algorithm for goal-oriented visual dialog, enabling questioners to infer answerer intentions and ask more effective questions, significantly improving performance on visual dialog tasks.

Contribution

The paper proposes the AQM algorithm, inspired by theory of mind, which explicitly models answerer intentions to enhance goal-oriented dialog performance.

Findings

AQM outperforms comparative algorithms by a large margin.

Effective inference of answerer intentions improves dialog efficiency.

Demonstrated success on MNIST Counting and GuessWhat?! tasks.

Abstract

Goal-oriented dialog has been given attention due to its numerous applications in artificial intelligence. Goal-oriented dialogue tasks occur when a questioner asks an action-oriented question and an answerer responds with the intent of letting the questioner know a correct action to take. To ask the adequate question, deep learning and reinforcement learning have been recently applied. However, these approaches struggle to find a competent recurrent neural questioner, owing to the complexity of learning a series of sentences. Motivated by theory of mind, we propose "Answerer in Questioner's Mind" (AQM), a novel information theoretic algorithm for goal-oriented dialog. With AQM, a questioner asks and infers based on an approximated probabilistic model of the answerer. The questioner figures out the answerer's intention via selecting a plausible question by explicitly calculating the information gain of the candidate intentions and possible answers to each question. We test our framework on two goal-oriented visual dialog tasks: "MNIST Counting Dialog" and "GuessWhat?!". In our experiments, AQM outperforms comparative algorithms by a large margin.