Fidelity Estimation Improves Noisy-Image Classification With Pretrained Networks

TL;DR

This paper introduces a fidelity estimation method that enhances the robustness of pretrained CNN classifiers to noisy images by integrating fidelity maps into their internal representations, significantly improving classification accuracy under noise.

Contribution

The proposed method improves noisy-image classification using pretrained networks without retraining, by incorporating fidelity maps into feature representations to guide attention.

Findings

Significant accuracy improvements at high noise levels.

Performance close to fully retrained models on noisy data.

Oracle fidelity maps can outperform retrained approaches.

Abstract

Image classification has significantly improved using deep learning. This is mainly due to convolutional neural networks (CNNs) that are capable of learning rich feature extractors from large datasets. However, most deep learning classification methods are trained on clean images and are not robust when handling noisy ones, even if a restoration preprocessing step is applied. While novel methods address this problem, they rely on modified feature extractors and thus necessitate retraining. We instead propose a method that can be applied on a $pretrained$ classifier. Our method exploits a fidelity map estimate that is fused into the internal representations of the feature extractor, thereby guiding the attention of the network and making it more robust to noisy data. We improve the noisy-image classification (NIC) results by significantly large margins, especially at high noise levels, and come close to the fully retrained approaches. Furthermore, as proof of concept, we show that when using our oracle fidelity map we even outperform the fully retrained methods, whether trained on noisy or restored images.