SplitNets: Designing Neural Architectures for Efficient Distributed Computing on Head-Mounted Systems

TL;DR

SplitNets is a neural architecture search framework that optimizes the distribution of neural network workloads across camera sensors and central processors in head-mounted devices, balancing accuracy, latency, and resource use.

Contribution

The paper introduces SplitNets, a novel framework for designing split neural networks that optimize computation, communication, and accuracy for head-mounted systems, including multi-view setups.

Findings

Achieves state-of-the-art performance on ImageNet with optimized latency.

Effectively balances computation, communication, and accuracy in head-mounted systems.

Extends to multi-view systems for improved 3D classification.

Abstract

We design deep neural networks (DNNs) and corresponding networks' splittings to distribute DNNs' workload to camera sensors and a centralized aggregator on head mounted devices to meet system performance targets in inference accuracy and latency under the given hardware resource constraints. To achieve an optimal balance among computation, communication, and performance, a split-aware neural architecture search framework, SplitNets, is introduced to conduct model designing, splitting, and communication reduction simultaneously. We further extend the framework to multi-view systems for learning to fuse inputs from multiple camera sensors with optimal performance and systemic efficiency. We validate SplitNets for single-view system on ImageNet as well as multi-view system on 3D classification, and show that the SplitNets framework achieves state-of-the-art (SOTA) performance and system latency compared with existing approaches.