High performance ultra-low-precision convolutions on mobile devices

TL;DR

This paper presents an open-source implementation of ultra-low-precision convolutions (<4 bits) optimized for mobile devices, achieving significant speedups over traditional methods, especially on older ARMv7 hardware.

Contribution

It introduces a novel ultra-low-precision convolution implementation for ARMv7 devices, with comprehensive analysis and open-source code, enabling faster mobile deep learning inference.

Findings

Achieved 4x-20x speedup over float32 and int8 baselines.

Demonstrated effectiveness on older ARMv7 mobile devices.

Provided open-source implementation for broader adoption.

Abstract

Many applications of mobile deep learning, especially real-time computer vision workloads, are constrained by computation power. This is particularly true for workloads running on older consumer phones, where a typical device might be powered by a single- or dual-core ARMv7 CPU. We provide an open-source implementation and a comprehensive analysis of (to our knowledge) the state of the art ultra-low-precision (<4 bit precision) implementation of the core primitives required for modern deep learning workloads on ARMv7 devices, and demonstrate speedups of 4x-20x over our additional state-of-the-art float32 and int8 baselines.