Localizing Semantic Patches for Accelerating Image Classification

TL;DR

This paper introduces AnchorNet, a lightweight patch proposal network that localizes task-aware regions in images, enabling more efficient classification by reducing spatial redundancy and improving inference speed.

Contribution

The paper presents a novel method for localizing semantic patches using AnchorNet, which enhances image classification efficiency and interpretability without modifying existing architectures.

Findings

Outperforms state-of-the-art dynamic inference methods on ImageNet

Reduces inference costs while maintaining high accuracy

Provides interpretable contribution of each localized patch

Abstract

Existing works often focus on reducing the architecture redundancy for accelerating image classification but ignore the spatial redundancy of the input image. This paper proposes an efficient image classification pipeline to solve this problem. We first pinpoint task-aware regions over the input image by a lightweight patch proposal network called AnchorNet. We then feed these localized semantic patches with much smaller spatial redundancy into a general classification network. Unlike the popular design of deep CNN, we aim to carefully design the Receptive Field of AnchorNet without intermediate convolutional paddings. This ensures the exact mapping from a high-level spatial location to the specific input image patch. The contribution of each patch is interpretable. Moreover, AnchorNet is compatible with any downstream architecture. Experimental results on ImageNet show that our method outperforms SOTA dynamic inference methods with fewer inference costs. Our code is available at https://github.com/winycg/AnchorNet.