Bag of Tricks for Image Classification with Convolutional Neural Networks

TL;DR

This paper systematically evaluates various training refinements for CNNs in image classification, demonstrating significant accuracy improvements and better transfer learning performance.

Contribution

It provides an empirical analysis of training tricks for CNNs, showing their combined effect on improving accuracy and transfer learning.

Findings

ResNet-50 accuracy improved from 75.3% to 79.29% on ImageNet

Combining training tricks yields significant accuracy gains

Improved models enhance transfer learning in detection and segmentation

Abstract

Much of the recent progress made in image classification research can be credited to training procedure refinements, such as changes in data augmentations and optimization methods. In the literature, however, most refinements are either briefly mentioned as implementation details or only visible in source code. In this paper, we will examine a collection of such refinements and empirically evaluate their impact on the final model accuracy through ablation study. We will show that, by combining these refinements together, we are able to improve various CNN models significantly. For example, we raise ResNet-50's top-1 validation accuracy from 75.3% to 79.29% on ImageNet. We will also demonstrate that improvement on image classification accuracy leads to better transfer learning performance in other application domains such as object detection and semantic segmentation.