Performance degradation of ImageNet trained models by simple image transformations

TL;DR

This paper evaluates how simple image transformations such as shifting, scaling, and noise affect the performance of ImageNet-trained models, revealing notable accuracy drops even with minor modifications.

Contribution

It systematically analyzes the robustness of popular ImageNet-trained models against common image transformations, highlighting their vulnerability.

Findings

Rotations of 10° reduce accuracy by over 1%.

Scaling by 20% causes significant performance degradation.

Simple transformations can notably impair model accuracy.

Abstract

ImageNet trained PyTorch models are generally preferred as the off-the-shelf models for direct use or for initialisation in most computer vision tasks. In this paper, we simply test a representative set of these convolution and transformer based models under many simple image transformations like horizontal shifting, vertical shifting, scaling, rotation, presence of Gaussian noise, cutout, horizontal flip and vertical flip and report the performance drop caused by such transformations. We find that even simple transformations like rotating the image by 10{\deg} or zooming in by 20% can reduce the top-1 accuracy of models like ResNet152 by 1%+. The code is available at https://github.com/harshm121/imagenet-transformation-degradation.