VizWiz Grand Challenge: Answering Visual Questions from Blind People

TL;DR

The paper introduces VizWiz, a challenging, real-world dataset of visual questions from blind users, aiming to foster development of more robust visual question answering algorithms that can assist visually impaired individuals.

Contribution

This work presents the first natural, goal-oriented VQA dataset from blind users, including images, spoken questions, and multiple answers, highlighting real-world challenges.

Findings

Modern algorithms struggle with VizWiz's poor-quality images.

Many visual questions in VizWiz are unanswerable.

The dataset reveals the need for more generalized VQA models.

Abstract

The study of algorithms to automatically answer visual questions currently is motivated by visual question answering (VQA) datasets constructed in artificial VQA settings. We propose VizWiz, the first goal-oriented VQA dataset arising from a natural VQA setting. VizWiz consists of over 31,000 visual questions originating from blind people who each took a picture using a mobile phone and recorded a spoken question about it, together with 10 crowdsourced answers per visual question. VizWiz differs from the many existing VQA datasets because (1) images are captured by blind photographers and so are often poor quality, (2) questions are spoken and so are more conversational, and (3) often visual questions cannot be answered. Evaluation of modern algorithms for answering visual questions and deciding if a visual question is answerable reveals that VizWiz is a challenging dataset. We introduce this dataset to encourage a larger community to develop more generalized algorithms that can assist blind people.