Learning to Learn Parameterized Classification Networks for Scalable Input Images

TL;DR

This paper introduces a meta-learning approach that dynamically generates CNN weights for varying input resolutions, enabling scalable and efficient image classification with improved accuracy and adaptability across different resource settings.

Contribution

The proposed method uses meta learners and privatized Batch Normalization to adapt CNNs to arbitrary input sizes, enhancing flexibility and performance over traditional fixed-resolution models.

Findings

Achieves better accuracy-efficiency trade-off on ImageNet

Enables dynamic resolution adaptation for resource-constrained environments

Outperforms individually trained models across various input scales

Abstract

Convolutional Neural Networks (CNNs) do not have a predictable recognition behavior with respect to the input resolution change. This prevents the feasibility of deployment on different input image resolutions for a specific model. To achieve efficient and flexible image classification at runtime, we employ meta learners to generate convolutional weights of main networks for various input scales and maintain privatized Batch Normalization layers per scale. For improved training performance, we further utilize knowledge distillation on the fly over model predictions based on different input resolutions. The learned meta network could dynamically parameterize main networks to act on input images of arbitrary size with consistently better accuracy compared to individually trained models. Extensive experiments on the ImageNet demonstrate that our method achieves an improved accuracy-efficiency trade-off during the adaptive inference process. By switching executable input resolutions, our method could satisfy the requirement of fast adaption in different resource-constrained environments. Code and models are available at https://github.com/d-li14/SAN.