Empowering Visually Impaired Individuals: A Novel Use of Apple Live Photos and Android Motion Photos

TL;DR

This paper demonstrates that using Apple Live Photos and Android Motion Photos significantly improves visual assistance for the visually impaired by outperforming traditional images in object classification and VideoQA tasks, validated through extensive experiments.

Contribution

It introduces a simple methodology to evaluate Live/Motion Photos versus traditional images, highlighting their superior performance in assistive visual tasks for the visually impaired.

Findings

Live/Motion Photos outperform single-frame images in object classification

Enhanced performance in VideoQA tasks with Live/Motion Photos

Ablation studies show deblurring and longer temporal crops improve results

Abstract

Numerous applications have been developed to assist visually impaired individuals that employ a machine learning unit to process visual input. However, a critical challenge with these applications is the sub-optimal quality of images captured by the users. Given the complexity of operating a camera for visually impaired individuals, we advocate for the use of Apple Live Photos and Android Motion Photos technologies. In this study, we introduce a straightforward methodology to evaluate and contrast the efficacy of Live/Motion Photos against traditional image-based approaches. Our findings reveal that both Live Photos and Motion Photos outperform single-frame images in common visual assisting tasks, specifically in object classification and VideoQA. We validate our results through extensive experiments on the ORBIT dataset, which consists of videos collected by visually impaired individuals. Furthermore, we conduct a series of ablation studies to delve deeper into the impact of deblurring and longer temporal crops.