Glance and Focus: a Dynamic Approach to Reducing Spatial Redundancy in Image Classification

TL;DR

This paper introduces a dynamic, reinforcement learning-based framework that selectively processes image regions to reduce computational costs in CNN-based image classification, maintaining accuracy while improving efficiency.

Contribution

It presents a flexible, general approach that adapts inference by selecting relevant image regions, compatible with various lightweight CNN architectures.

Findings

Reduces MobileNet-V3 latency by 20% on iPhone XS Max

Improves computational efficiency across multiple CNN models

Maintains accuracy while decreasing redundant computation

Abstract

The accuracy of deep convolutional neural networks (CNNs) generally improves when fueled with high resolution images. However, this often comes at a high computational cost and high memory footprint. Inspired by the fact that not all regions in an image are task-relevant, we propose a novel framework that performs efficient image classification by processing a sequence of relatively small inputs, which are strategically selected from the original image with reinforcement learning. Such a dynamic decision process naturally facilitates adaptive inference at test time, i.e., it can be terminated once the model is sufficiently confident about its prediction and thus avoids further redundant computation. Notably, our framework is general and flexible as it is compatible with most of the state-of-the-art light-weighted CNNs (such as MobileNets, EfficientNets and RegNets), which can be conveniently deployed as the backbone feature extractor. Experiments on ImageNet show that our method consistently improves the computational efficiency of a wide variety of deep models. For example, it further reduces the average latency of the highly efficient MobileNet-V3 on an iPhone XS Max by 20% without sacrificing accuracy. Code and pre-trained models are available at https://github.com/blackfeather-wang/GFNet-Pytorch.