Building Task-Oriented Visual Dialog Systems Through Alternative Optimization Between Dialog Policy and Language Generation

TL;DR

This paper introduces an alternative optimization framework combining reinforcement learning and supervised learning to enhance task-oriented visual dialog systems, achieving state-of-the-art results on the GuessWhich task.

Contribution

It proposes a novel alternating training approach for dialog policy learning and language generation, addressing reward design challenges in RL-based visual dialog systems.

Findings

Achieves state-of-the-art performance on GuessWhich.

Improves dialog quality and task completion.

Demonstrates effectiveness of alternating training.

Abstract

Reinforcement learning (RL) is an effective approach to learn an optimal dialog policy for task-oriented visual dialog systems. A common practice is to apply RL on a neural sequence-to-sequence (seq2seq) framework with the action space being the output vocabulary in the decoder. However, it is difficult to design a reward function that can achieve a balance between learning an effective policy and generating a natural dialog response. This paper proposes a novel framework that alternatively trains a RL policy for image guessing and a supervised seq2seq model to improve dialog generation quality. We evaluate our framework on the GuessWhich task and the framework achieves the state-of-the-art performance in both task completion and dialog quality.