Distorted Representation Space Characterization Through Backpropagated Gradients

TL;DR

This paper explores how weight gradients from backpropagation can characterize learned representation spaces in deep learning, improving tasks like image quality assessment and out-of-distribution detection.

Contribution

It introduces a gradient-based feature method that outperforms activation features in key applications and analyzes the impact of regularization on gradients.

Findings

Gradient features outperform activation features in image quality assessment.

The proposed method achieves top performance on TID 2013 and MULTI-LIVE datasets.

Regularization influences gradient characteristics in out-of-distribution classification.

Abstract

In this paper, we utilize weight gradients from backpropagation to characterize the representation space learned by deep learning algorithms. We demonstrate the utility of such gradients in applications including perceptual image quality assessment and out-of-distribution classification. The applications are chosen to validate the effectiveness of gradients as features when the test image distribution is distorted from the train image distribution. In both applications, the proposed gradient based features outperform activation features. In image quality assessment, the proposed approach is compared with other state of the art approaches and is generally the top performing method on TID 2013 and MULTI-LIVE databases in terms of accuracy, consistency, linearity, and monotonic behavior. Finally, we analyze the effect of regularization on gradients using CURE-TSR dataset for out-of-distribution classification.