Resolution Switchable Networks for Runtime Efficient Image Recognition

TL;DR

This paper introduces Resolution Switchable Networks (RS-Nets), a method for training a single CNN that can switch image resolutions at inference to optimize speed and accuracy across different computational constraints.

Contribution

The paper presents a novel training framework for RS-Nets that shares parameters across resolutions with separate batch normalization, and introduces a multi-resolution ensemble distillation technique to improve accuracy.

Findings

RS-Nets achieve competitive accuracy across multiple resolutions.

Ensemble distillation improves performance over single-resolution models.

The method is validated on ImageNet and extends to quantization scenarios.

Abstract

We propose a general method to train a single convolutional neural network which is capable of switching image resolutions at inference. Thus the running speed can be selected to meet various computational resource limits. Networks trained with the proposed method are named Resolution Switchable Networks (RS-Nets). The basic training framework shares network parameters for handling images which differ in resolution, yet keeps separate batch normalization layers. Though it is parameter-efficient in design, it leads to inconsistent accuracy variations at different resolutions, for which we provide a detailed analysis from the aspect of the train-test recognition discrepancy. A multi-resolution ensemble distillation is further designed, where a teacher is learnt on the fly as a weighted ensemble over resolutions. Thanks to the ensemble and knowledge distillation, RS-Nets enjoy accuracy improvements at a wide range of resolutions compared with individually trained models. Extensive experiments on the ImageNet dataset are provided, and we additionally consider quantization problems. Code and models are available at https://github.com/yikaiw/RS-Nets.