HarDNet: A Low Memory Traffic Network

TL;DR

HarDNet introduces a neural network architecture optimized for low memory traffic, significantly reducing inference time in high-resolution tasks by focusing on memory efficiency alongside MACs.

Contribution

The paper proposes HarDNet, a novel network design that reduces memory traffic and inference latency, emphasizing the importance of memory considerations in high-resolution neural network applications.

Findings

HarDNet reduces inference time by up to 45% compared to existing models.

Memory traffic correlates strongly with inference latency.

HarDNet consumes less memory traffic, verified by profiling tools.

Abstract

State-of-the-art neural network architectures such as ResNet, MobileNet, and DenseNet have achieved outstanding accuracy over low MACs and small model size counterparts. However, these metrics might not be accurate for predicting the inference time. We suggest that memory traffic for accessing intermediate feature maps can be a factor dominating the inference latency, especially in such tasks as real-time object detection and semantic segmentation of high-resolution video. We propose a Harmonic Densely Connected Network to achieve high efficiency in terms of both low MACs and memory traffic. The new network achieves 35%, 36%, 30%, 32%, and 45% inference time reduction compared with FC-DenseNet-103, DenseNet-264, ResNet-50, ResNet-152, and SSD-VGG, respectively. We use tools including Nvidia profiler and ARM Scale-Sim to measure the memory traffic and verify that the inference latency is indeed proportional to the memory traffic consumption and the proposed network consumes low memory traffic. We conclude that one should take memory traffic into consideration when designing neural network architectures for high-resolution applications at the edge.