Why Did the Chicken Cross the Road? Rephrasing and Analyzing Ambiguous Questions in VQA

TL;DR

This paper addresses ambiguity in visual questions by creating a dataset, analyzing linguistic causes, and developing a question-generation model that reduces ambiguity and integrates answer group information.

Contribution

It introduces a dataset of ambiguous visual questions, analyzes their linguistic causes, and proposes a question-generation model that reduces ambiguity without direct supervision.

Findings

The dataset reveals a linguistically-aligned ontology of ambiguity reasons.

The question-generation model produces less ambiguous questions.

The model effectively integrates answer group information without explicit supervision.

Abstract

Natural language is ambiguous. Resolving ambiguous questions is key to successfully answering them. Focusing on questions about images, we create a dataset of ambiguous examples. We annotate these, grouping answers by the underlying question they address and rephrasing the question for each group to reduce ambiguity. Our analysis reveals a linguistically-aligned ontology of reasons for ambiguity in visual questions. We then develop an English question-generation model which we demonstrate via automatic and human evaluation produces less ambiguous questions. We further show that the question generation objective we use allows the model to integrate answer group information without any direct supervision.