Robustness as Architecture: Designing IQA Models to Withstand Adversarial Perturbations

TL;DR

This paper proposes a novel architectural approach to enhance the robustness of Image Quality Assessment models against adversarial attacks by designing networks with orthogonal information flow and norm-preserving operations, avoiding traditional data-driven defenses.

Contribution

It introduces the concept of robustness as an architectural prior, reshaping IQA models to inherently resist adversarial perturbations through structural design rather than training methods.

Findings

Architectural design improves adversarial robustness without retraining.

Orthogonal information flow reduces sensitivity to perturbations.

Norm-preserving operations stabilize IQA models against attacks.

Abstract

Image Quality Assessment (IQA) models are increasingly relied upon to evaluate image quality in real-world systems -- from compression and enhancement to generation and streaming. Yet their adoption brings a fundamental risk: these models are inherently unstable. Adversarial manipulations can easily fool them, inflating scores and undermining trust. Traditionally, such vulnerabilities are addressed through data-driven defenses -- adversarial retraining, regularization, or input purification. But what if this is the wrong lens? What if robustness in perceptual models is not something to learn but something to design? In this work, we propose a provocative idea: robustness as an architectural prior. Rather than training models to resist perturbations, we reshape their internal structure to suppress sensitivity from the ground up. We achieve this by enforcing orthogonal information flow, constraining the network to norm-preserving operations -- and further stabilizing the system through pruning and fine-tuning. The result is a robust IQA architecture that withstands adversarial attacks without requiring adversarial training or significant changes to the original model. This approach suggests a shift in perspective: from optimizing robustness through data to engineering it through design.