BABO: Background Activation Black-Out for Efficient Object Detection

TL;DR

This paper introduces BABO, a method that sparsifies background activations in object detection models to improve inference efficiency on resource-limited devices, maintaining accuracy while reducing computation.

Contribution

BABO incorporates a lightweight objectness mask network to zero out background regions, enabling faster inference through sparse convolution techniques without sacrificing detection accuracy.

Findings

Background activation zeros increased from 36% to 68% on MobileNetV2-SSDLite.

Total MAC operations reduced to 62% of original with background sparsification.

Effective across different networks and datasets, including VGG, RetinaNet, MS-COCO, and PASCAL VOC.

Abstract

Recent advances in deep learning have enabled complex real-world use cases comprised of multiple vision tasks and detection tasks are being shifted to the edge side as a pre-processing step of the entire workload. Since running a deep model on resource-constraint devices is challenging, techniques for efficient inference methods are demanded. In this paper, we present an objectness-aware object detection method to reduce computational cost by sparsifying activation values on background regions where target objects don't exist. Sparsified activation can be exploited to increase inference speed by software or hardware accelerated sparse convolution techniques. To accomplish this goal, we incorporate a light-weight objectness mask generation (OMG) network in front of an object detection (OD) network so that it can zero out unnecessary background areas of an input image before being fed into the OD network. In experiments, by switching background activation values to zero, the average number of zero values increases further from 36% to 68% on MobileNetV2-SSDLite even with ReLU activation while maintaining accuracy on MS-COCO. This result indicates that the total MAC including both OMG and OD networks can be reduced to 62% of the original OD model when only non-zero multiply-accumulate operations are considered. Moreover, we show a similar tendency in heavy networks (VGG and RetinaNet) and an additional dataset (PASCAL VOC).