# Learning Goal-Oriented Visual Dialog Agents: Imitating and Surpassing Analytic Experts

## TL;DR

This paper introduces a hybrid approach combining analytic experts and reinforcement learning to train goal-oriented visual dialog agents, achieving state-of-the-art results on the GuessWhat?! dataset.

## Contribution

It develops two analytic experts as high-quality demonstration sources and integrates them with reinforcement learning for improved dialog agent training.

## Key findings

- Achieves state-of-the-art performance on GuessWhat?! dataset.
- Combines imitation learning from analytic experts with reinforcement learning.
- Outperforms previous methods relying solely on model-free reinforcement learning.

## Abstract

This paper tackles the problem of learning a questioner in the goal-oriented visual dialog task. Several previous works adopt model-free reinforcement learning. Most pretrain the model from a finite set of human-generated data. We argue that using limited demonstrations to kick-start the questioner is insufficient due to the large policy search space. Inspired by a recently proposed information theoretic approach, we develop two analytic experts to serve as a source of high-quality demonstrations for imitation learning. We then take advantage of reinforcement learning to refine the model towards the goal-oriented objective. Experimental results on the GuessWhat?! dataset show that our method has the combined merits of imitation and reinforcement learning, achieving the state-of-the-art performance.