Region-Adaptive Deformable Network for Image Quality Assessment

TL;DR

This paper introduces RADN, a novel image quality assessment network that adaptively handles spatial misalignments in GAN-generated images, improving correlation with human perception.

Contribution

The paper proposes a reference-oriented deformable convolution and a patch-level attention module, enhancing IQA performance on GAN-based distortions and addressing spatial misalignment issues.

Findings

RADN outperforms existing IQA methods on NTIRE 2021 dataset

Ensemble of RADN models achieved fourth place in the challenge

Proposed modules improve robustness to GAN-induced distortions

Abstract

Image quality assessment (IQA) aims to assess the perceptual quality of images. The outputs of the IQA algorithms are expected to be consistent with human subjective perception. In image restoration and enhancement tasks, images generated by generative adversarial networks (GAN) can achieve better visual performance than traditional CNN-generated images, although they have spatial shift and texture noise. Unfortunately, the existing IQA methods have unsatisfactory performance on the GAN-based distortion partially because of their low tolerance to spatial misalignment. To this end, we propose the reference-oriented deformable convolution, which can improve the performance of an IQA network on GAN-based distortion by adaptively considering this misalignment. We further propose a patch-level attention module to enhance the interaction among different patch regions, which are processed independently in previous patch-based methods. The modified residual block is also proposed by applying modifications to the classic residual block to construct a patch-region-based baseline called WResNet. Equipping this baseline with the two proposed modules, we further propose Region-Adaptive Deformable Network (RADN). The experiment results on the NTIRE 2021 Perceptual Image Quality Assessment Challenge dataset show the superior performance of RADN, and the ensemble approach won fourth place in the final testing phase of the challenge. Code is available at https://github.com/IIGROUP/RADN.