Fine-grained Image Quality Assessment for Perceptual Image Restoration

TL;DR

This paper introduces FGRestore, a detailed dataset for image restoration quality assessment, and proposes FGResQ, a new model that outperforms existing IQA metrics in fine-grained restoration evaluation.

Contribution

The paper presents the first fine-grained IQA dataset for image restoration and a novel IQA model tailored for restoration quality assessment.

Findings

FGResQ outperforms state-of-the-art IQA metrics.

FGRestore dataset contains 18,408 images and 30,886 pairwise preferences.

Existing IQA metrics show inconsistencies with fine-grained restoration quality.

Abstract

Recent years have witnessed remarkable achievements in perceptual image restoration (IR), creating an urgent demand for accurate image quality assessment (IQA), which is essential for both performance comparison and algorithm optimization. Unfortunately, the existing IQA metrics exhibit inherent weakness for IR task, particularly when distinguishing fine-grained quality differences among restored images. To address this dilemma, we contribute the first-of-its-kind fine-grained image quality assessment dataset for image restoration, termed FGRestore, comprising 18,408 restored images across six common IR tasks. Beyond conventional scalar quality scores, FGRestore was also annotated with 30,886 fine-grained pairwise preferences. Based on FGRestore, a comprehensive benchmark was conducted on the existing IQA metrics, which reveal significant inconsistencies between score-based IQA evaluations and the fine-grained restoration quality. Motivated by these findings, we further propose FGResQ, a new IQA model specifically designed for image restoration, which features both coarse-grained score regression and fine-grained quality ranking. Extensive experiments and comparisons demonstrate that FGResQ significantly outperforms state-of-the-art IQA metrics. Codes and model weights have been released in https://sxfly99.github.io/FGResQ-Home.