CPUBone: Efficient Vision Backbone Design for Devices with Low Parallelization Capabilities

TL;DR

CPUBone is a new vision backbone designed specifically for CPU inference, balancing operations and hardware efficiency, achieving state-of-the-art speed-accuracy trade-offs on diverse CPU devices.

Contribution

The paper introduces CPUBone, a novel CPU-optimized vision backbone that maintains high hardware efficiency while reducing computational cost through modified convolution techniques.

Findings

CPUBone achieves superior speed-accuracy trade-offs on various CPU devices.

Modified convolutions reduce MACs while preserving hardware efficiency.

CPUBone transfers efficiency gains effectively to object detection and segmentation tasks.

Abstract

Recent research on vision backbone architectures has predominantly focused on optimizing efficiency for hardware platforms with high parallel processing capabilities. This category increasingly includes embedded systems such as mobile phones and embedded AI accelerator modules. In contrast, CPUs do not have the possibility to parallelize operations in the same manner, wherefore models benefit from a specific design philosophy that balances amount of operations (MACs) and hardware-efficient execution by having high MACs per second (MACpS). In pursuit of this, we investigate two modifications to standard convolutions, aimed at reducing computational cost: grouping convolutions and reducing kernel sizes. While both adaptations substantially decrease the total number of MACs required for inference, sustaining low latency necessitates preserving hardware-efficiency. Our experiments across diverse CPU devices confirm that these adaptations successfully retain high hardware-efficiency on CPUs. Based on these insights, we introduce CPUBone, a new family of vision backbone models optimized for CPU-based inference. CPUBone achieves state-of-the-art Speed-Accuracy Trade-offs (SATs) across a wide range of CPU devices and effectively transfers its efficiency to downstream tasks such as object detection and semantic segmentation. Models and code are available at https://github.com/altair199797/CPUBone.