Resolution Adaptive Networks for Efficient Inference

TL;DR

RANet adaptively processes images at different resolutions based on difficulty, enabling efficient inference by early exiting for easy samples and detailed processing for hard ones, reducing computational redundancy.

Contribution

This paper introduces RANet, a novel resolution adaptive network that exploits spatial redundancy for efficient inference, allowing early exits for easy inputs and detailed processing for hard ones.

Findings

RANet achieves significant computational savings on CIFAR-10, CIFAR-100, and ImageNet.

It maintains high accuracy while reducing inference cost for easy samples.

Effective in both anytime prediction and budgeted batch classification settings.

Abstract

Adaptive inference is an effective mechanism to achieve a dynamic tradeoff between accuracy and computational cost in deep networks. Existing works mainly exploit architecture redundancy in network depth or width. In this paper, we focus on spatial redundancy of input samples and propose a novel Resolution Adaptive Network (RANet), which is inspired by the intuition that low-resolution representations are sufficient for classifying "easy" inputs containing large objects with prototypical features, while only some "hard" samples need spatially detailed information. In RANet, the input images are first routed to a lightweight sub-network that efficiently extracts low-resolution representations, and those samples with high prediction confidence will exit early from the network without being further processed. Meanwhile, high-resolution paths in the network maintain the capability to recognize the "hard" samples. Therefore, RANet can effectively reduce the spatial redundancy involved in inferring high-resolution inputs. Empirically, we demonstrate the effectiveness of the proposed RANet on the CIFAR-10, CIFAR-100 and ImageNet datasets in both the anytime prediction setting and the budgeted batch classification setting.