A Self-Supervised Feature Map Augmentation (FMA) Loss and Combined Augmentations Finetuning to Efficiently Improve the Robustness of CNNs

TL;DR

This paper introduces a novel feature-map augmentation loss and a combined augmentations finetuning strategy to enhance CNN robustness against various input distortions, achieving significant accuracy improvements on CIFAR-10 and ImageNet.

Contribution

It proposes a new regularization loss called FMA and a combined augmentations finetuning method to improve CNN robustness efficiently across multiple distortion types.

Findings

Achieved ~9% accuracy improvement on CIFAR-10 with FMA and ST.

Achieved ~8% accuracy improvement on ImageNet with FMA and ST.

Outperformed traditional data augmentation in robustness enhancement.

Abstract

Deep neural networks are often not robust to semantically-irrelevant changes in the input. In this work we address the issue of robustness of state-of-the-art deep convolutional neural networks (CNNs) against commonly occurring distortions in the input such as photometric changes, or the addition of blur and noise. These changes in the input are often accounted for during training in the form of data augmentation. We have two major contributions: First, we propose a new regularization loss called feature-map augmentation (FMA) loss which can be used during finetuning to make a model robust to several distortions in the input. Second, we propose a new combined augmentations (CA) finetuning strategy, that results in a single model that is robust to several augmentation types at the same time in a data-efficient manner. We use the CA strategy to improve an existing state-of-the-art method called stability training (ST). Using CA, on an image classification task with distorted images, we achieve an accuracy improvement of on average 8.94% with FMA and 8.86% with ST absolute on CIFAR-10 and 8.04% with FMA and 8.27% with ST absolute on ImageNet, compared to 1.98% and 2.12%, respectively, with the well known data augmentation method, while keeping the clean baseline performance.