Where To Look: Focus Regions for Visual Question Answering

TL;DR

This paper introduces a method for visual question answering that learns to select relevant image regions based on the question, significantly improving accuracy on the VQA dataset.

Contribution

It proposes a novel approach that dynamically focuses on image regions relevant to the question, enhancing VQA performance over previous methods.

Findings

Improved accuracy on VQA dataset

Effective region selection for specific question types

Demonstrated significance in question-specific visual reasoning

Abstract

We present a method that learns to answer visual questions by selecting image regions relevant to the text-based query. Our method exhibits significant improvements in answering questions such as "what color," where it is necessary to evaluate a specific location, and "what room," where it selectively identifies informative image regions. Our model is tested on the VQA dataset which is the largest human-annotated visual question answering dataset to our knowledge.