Omnidirectional Image Quality Captioning: A Large-scale Database and A New Model

TL;DR

This paper introduces a large-scale database for omnidirectional image quality assessment, along with a novel model that generates descriptive quality captions, significantly advancing the field's ability to evaluate and interpret omnidirectional images.

Contribution

It presents the largest OIQA database with diverse distortions and a new adaptive captioning model, IQCaption360, for comprehensive image quality assessment.

Findings

IQCaption360 outperforms existing methods significantly.

The OIQ-10K database enables robust evaluation of OIQA models.

Human opinion data includes spatial distortion distribution and viewer movements.

Abstract

The fast growing application of omnidirectional images calls for effective approaches for omnidirectional image quality assessment (OIQA). Existing OIQA methods have been developed and tested on homogeneously distorted omnidirectional images, but it is hard to transfer their success directly to the heterogeneously distorted omnidirectional images. In this paper, we conduct the largest study so far on OIQA, where we establish a large-scale database called OIQ-10K containing 10,000 omnidirectional images with both homogeneous and heterogeneous distortions. A comprehensive psychophysical study is elaborated to collect human opinions for each omnidirectional image, together with the spatial distributions (within local regions or globally) of distortions, and the head and eye movements of the subjects. Furthermore, we propose a novel multitask-derived adaptive feature-tailoring OIQA model named IQCaption360, which is capable of generating a quality caption for an omnidirectional image in a manner of textual template. Extensive experiments demonstrate the effectiveness of IQCaption360, which outperforms state-of-the-art methods by a significant margin on the proposed OIQ-10K database. The OIQ-10K database and the related source codes are available at https://github.com/WenJuing/IQCaption360.