Scale Calibrated Training: Improving Generalization of Deep Networks via Scale-Specific Normalization

TL;DR

This paper introduces Scale Calibrated Training (SCT), a novel method enabling CNNs to generalize across multiple image scales during testing, significantly improving accuracy on low-resolution images.

Contribution

The paper proposes SCT and a new normalization scheme called Scale-Specific Batch Normalization, enhancing CNN robustness to varying input scales.

Findings

SCT improves ResNet-50 accuracy by 1.7% on 224-sized images.

SCT improves ResNet-50 accuracy by 11.5% on 128-sized images.

Scale-Specific Batch Normalization outperforms vanilla batch normalization in SCT.

Abstract

Standard convolutional neural networks(CNNs) require consistent image resolutions in both training and testing phase. However, in practice, testing with smaller image sizes is necessary for fast inference. We show that trivially evaluating low-resolution images on networks trained with high-resolution images results in a catastrophic accuracy drop in standard CNN architectures. We propose a novel training regime called Scale calibrated Training(SCT) which allows networks to learn from various scales of input simultaneously. By taking advantages of SCT, single network can provide decent accuracy at test time in response to multiple test scales. In our analysis, we surprisingly find that vanilla batch normalization can lead to sub-optimal performance in SCT. Therefore, a novel normalization scheme called Scale-Specific Batch Normalization is equipped to SCT in replacement of batch normalization. Experiment results show that SCT improves accuracy of single Resnet-50 on ImageNet by 1.7% and 11.5% accuracy when testing on image sizes of 224 and 128 respectively.