Video Question Answering for People with Visual Impairments Using an Egocentric 360-Degree Camera

TL;DR

This paper introduces a new 360-degree egocentric video dataset for question answering aimed at assisting visually impaired individuals, highlighting its unique features and evaluating current AI methods' performance.

Contribution

The paper presents a novel 360-degree egocentric video dataset for visual question answering, addressing multiple real-life challenges faced by visually impaired people.

Findings

State-of-the-art VideoQA methods show limited performance on the dataset.

The dataset captures ego-motion in diverse scenarios with 360-degree videos.

Progress in AI assistive services remains insufficient for practical use.

Abstract

This paper addresses the daily challenges encountered by visually impaired individuals, such as limited access to information, navigation difficulties, and barriers to social interaction. To alleviate these challenges, we introduce a novel visual question answering dataset. Our dataset offers two significant advancements over previous datasets: Firstly, it features videos captured using a 360-degree egocentric wearable camera, enabling observation of the entire surroundings, departing from the static image-centric nature of prior datasets. Secondly, unlike datasets centered on singular challenges, ours addresses multiple real-life obstacles simultaneously through an innovative visual-question answering framework. We validate our dataset using various state-of-the-art VideoQA methods and diverse metrics. Results indicate that while progress has been made, satisfactory performance levels for AI-powered assistive services remain elusive for visually impaired individuals. Additionally, our evaluation highlights the distinctive features of the proposed dataset, featuring ego-motion in videos captured via 360-degree cameras across varied scenarios.