DeepDC: Deep Distance Correlation as a Perceptual Image Quality Evaluator

TL;DR

DeepDC introduces a novel image quality assessment method leveraging deep neural network features and distance correlation to effectively evaluate image quality, texture similarity, and geometric transformations.

Contribution

The paper presents a new full-reference IQA model based on distance correlation in deep features, exploiting the texture-sensitive property of pre-trained DNNs, and applies it to texture synthesis and neural style transfer.

Findings

Outperforms existing IQA models on multiple datasets

Effective in texture and geometric transformation assessments

Achieves state-of-the-art results in neural style transfer

Abstract

ImageNet pre-trained deep neural networks (DNNs) show notable transferability for building effective image quality assessment (IQA) models. Such a remarkable byproduct has often been identified as an emergent property in previous studies. In this work, we attribute such capability to the intrinsic texture-sensitive characteristic that classifies images using texture features. We fully exploit this characteristic to develop a novel full-reference IQA (FR-IQA) model based exclusively on pre-trained DNN features. Specifically, we compute the distance correlation, a highly promising yet relatively under-investigated statistic, between reference and distorted images in the deep feature domain. In addition, the distance correlation quantifies both linear and nonlinear feature relationships, which is far beyond the widely used first-order and second-order statistics in the feature space. We conduct comprehensive experiments to demonstrate the superiority of the proposed quality model on five standard IQA datasets, one perceptual similarity dataset, two texture similarity datasets, and one geometric transformation dataset. Moreover, we optimize the proposed model to generate a broad spectrum of texture patterns, by treating the model as the style loss function for neural style transfer (NST). Extensive experiments demonstrate that the proposed texture synthesis and NST methods achieve the best quantitative and qualitative results. We release our code at https://github.com/h4nwei/DeepDC.