Visual Question Answering using Deep Learning: A Survey and Performance Analysis

TL;DR

This paper surveys recent datasets and deep learning models for Visual Question Answering, analyzing their performance, challenges, and future research directions in combining visual and linguistic data understanding.

Contribution

It provides a comprehensive review of recent VQA datasets, models, and performance analysis, including experimental results and future research insights.

Findings

Deep learning models show promising results on VQA datasets.

Analysis of model robustness and dataset challenges.

Discussion of future directions in VQA research.

Abstract

The Visual Question Answering (VQA) task combines challenges for processing data with both Visual and Linguistic processing, to answer basic `common sense' questions about given images. Given an image and a question in natural language, the VQA system tries to find the correct answer to it using visual elements of the image and inference gathered from textual questions. In this survey, we cover and discuss the recent datasets released in the VQA domain dealing with various types of question-formats and robustness of the machine-learning models. Next, we discuss about new deep learning models that have shown promising results over the VQA datasets. At the end, we present and discuss some of the results computed by us over the vanilla VQA model, Stacked Attention Network and the VQA Challenge 2017 winner model. We also provide the detailed analysis along with the challenges and future research directions.