Causal Perception Inspired Representation Learning for Trustworthy Image Quality Assessment

TL;DR

This paper introduces a causal perception inspired representation learning framework for trustworthy image quality assessment, enhancing robustness against adversarial attacks and providing interpretability.

Contribution

It proposes a novel CPRL method that extracts causal perception representations to improve robustness and interpretability in IQA models.

Findings

Outperforms state-of-the-art adversarial defense methods

Provides explicit model interpretation

Achieves higher robustness on benchmark datasets

Abstract

Despite great success in modeling visual perception, deep neural network based image quality assessment (IQA) still remains unreliable in real-world applications due to its vulnerability to adversarial perturbations and the inexplicit black-box structure. In this paper, we propose to build a trustworthy IQA model via Causal Perception inspired Representation Learning (CPRL), and a score reflection attack method for IQA model. More specifically, we assume that each image is composed of Causal Perception Representation (CPR) and non-causal perception representation (N-CPR). CPR serves as the causation of the subjective quality label, which is invariant to the imperceptible adversarial perturbations. Inversely, N-CPR presents spurious associations with the subjective quality label, which may significantly change with the adversarial perturbations. To extract the CPR from each input image, we develop a soft ranking based channel-wise activation function to mediate the causally sufficient (beneficial for high prediction accuracy) and necessary (beneficial for high robustness) deep features, and based on intervention employ minimax game to optimize. Experiments on four benchmark databases show that the proposed CPRL method outperforms many state-of-the-art adversarial defense methods and provides explicit model interpretation.