YOLObile: Real-Time Object Detection on Mobile Devices via Compression-Compilation Co-Design

TL;DR

YOLObile introduces a compression-compilation co-design framework enabling real-time object detection on mobile devices, achieving high accuracy and speed through novel pruning and GPU-CPU collaboration.

Contribution

The paper presents a novel co-design framework with a block-punched pruning scheme and GPU-CPU collaboration for efficient mobile object detection.

Findings

14× compression of YOLOv4 with 49.0 mAP

17 FPS inference on Samsung Galaxy S20 with GPU

Speedup to 19.1 FPS with GPU-CPU collaboration

Abstract

The rapid development and wide utilization of object detection techniques have aroused attention on both accuracy and speed of object detectors. However, the current state-of-the-art object detection works are either accuracy-oriented using a large model but leading to high latency or speed-oriented using a lightweight model but sacrificing accuracy. In this work, we propose YOLObile framework, a real-time object detection on mobile devices via compression-compilation co-design. A novel block-punched pruning scheme is proposed for any kernel size. To improve computational efficiency on mobile devices, a GPU-CPU collaborative scheme is adopted along with advanced compiler-assisted optimizations. Experimental results indicate that our pruning scheme achieves 14$\times$ compression rate of YOLOv4 with 49.0 mAP. Under our YOLObile framework, we achieve 17 FPS inference speed using GPU on Samsung Galaxy S20. By incorporating our proposed GPU-CPU collaborative scheme, the inference speed is increased to 19.1 FPS, and outperforms the original YOLOv4 by 5$\times$ speedup. Source code is at: \url{https://github.com/nightsnack/YOLObile}.