ISyNet: Convolutional Neural Networks design for AI accelerator

TL;DR

This paper introduces ISyNet, a set of neural network architectures optimized for AI accelerators, emphasizing hardware efficiency and real-time performance without sacrificing accuracy, validated on ImageNet and downstream tasks.

Contribution

The paper proposes a new hardware efficiency measure, a tailored search space, and a latency-aware scaling method for designing neural networks optimized for NPUs.

Findings

ISyNet architectures outperform existing models on NPU hardware.

Designed architectures achieve high accuracy with low latency.

Demonstrated generalization on classification and detection tasks.

Abstract

In recent years Deep Learning reached significant results in many practical problems, such as computer vision, natural language processing, speech recognition and many others. For many years the main goal of the research was to improve the quality of models, even if the complexity was impractically high. However, for the production solutions, which often require real-time work, the latency of the model plays a very important role. Current state-of-the-art architectures are found with neural architecture search (NAS) taking model complexity into account. However, designing of the search space suitable for specific hardware is still a challenging task. To address this problem we propose a measure of hardware efficiency of neural architecture search space - matrix efficiency measure (MEM); a search space comprising of hardware-efficient operations; a latency-aware scaling method; and ISyNet - a set of architectures designed to be fast on the specialized neural processing unit (NPU) hardware and accurate at the same time. We show the advantage of the designed architectures for the NPU devices on ImageNet and the generalization ability for the downstream classification and detection tasks.