SSBNet: Improving Visual Recognition Efficiency by Adaptive Sampling

TL;DR

SSBNet introduces adaptive sampling layers into neural networks like ResNet, enhancing visual recognition efficiency by better preserving important information and improving accuracy with minimal added complexity.

Contribution

The paper demonstrates that integrating adaptive sampling into deep neural network blocks improves efficiency and accuracy in image classification and object detection tasks.

Findings

SSBNet achieves higher accuracy than baseline ResNet models.

Adaptive sampling allows different layers to focus on relevant regions.

Visualization confirms adaptive sampling's advantage over uniform methods.

Abstract

Downsampling is widely adopted to achieve a good trade-off between accuracy and latency for visual recognition. Unfortunately, the commonly used pooling layers are not learned, and thus cannot preserve important information. As another dimension reduction method, adaptive sampling weights and processes regions that are relevant to the task, and is thus able to better preserve useful information. However, the use of adaptive sampling has been limited to certain layers. In this paper, we show that using adaptive sampling in the building blocks of a deep neural network can improve its efficiency. In particular, we propose SSBNet which is built by inserting sampling layers repeatedly into existing networks like ResNet. Experiment results show that the proposed SSBNet can achieve competitive image classification and object detection performance on ImageNet and COCO datasets. For example, the SSB-ResNet-RS-200 achieved 82.6% accuracy on ImageNet dataset, which is 0.6% higher than the baseline ResNet-RS-152 with a similar complexity. Visualization shows the advantage of SSBNet in allowing different layers to focus on different positions, and ablation studies further validate the advantage of adaptive sampling over uniform methods.