Efficient Computer Vision on Edge Devices with Pipeline-Parallel Hierarchical Neural Networks

TL;DR

This paper introduces a pipeline-parallel hierarchical neural network approach for efficient computer vision inference on multiple low-power edge devices, significantly improving throughput and reducing energy and memory usage.

Contribution

It presents a novel parallel inference pipeline for hierarchical DNNs on edge devices, balancing workloads and reducing communication overhead.

Findings

3.21X higher throughput on Raspberry Pi 4B clusters

68% less energy consumption per device per frame

58% decrease in memory usage

Abstract

Computer vision on low-power edge devices enables applications including search-and-rescue and security. State-of-the-art computer vision algorithms, such as Deep Neural Networks (DNNs), are too large for inference on low-power edge devices. To improve efficiency, some existing approaches parallelize DNN inference across multiple edge devices. However, these techniques introduce significant communication and synchronization overheads or are unable to balance workloads across devices. This paper demonstrates that the hierarchical DNN architecture is well suited for parallel processing on multiple edge devices. We design a novel method that creates a parallel inference pipeline for computer vision problems that use hierarchical DNNs. The method balances loads across the collaborating devices and reduces communication costs to facilitate the processing of multiple video frames simultaneously with higher throughput. Our experiments consider a representative computer vision problem where image recognition is performed on each video frame, running on multiple Raspberry Pi 4Bs. With four collaborating low-power edge devices, our approach achieves 3.21X higher throughput, 68% less energy consumption per device per frame, and 58% decrease in memory when compared with existing single-device hierarchical DNNs.