NAIS: Neural Architecture and Implementation Search and its Applications in Autonomous Driving

TL;DR

This paper advocates for a co-design approach called NAIS that simultaneously searches for optimal neural architectures and implementations, aiming to improve efficiency and quality in AI applications like autonomous driving.

Contribution

It introduces the NAIS methodology for joint neural architecture and implementation search, reducing search costs while maintaining solution quality, especially for demanding applications.

Findings

NAIS can significantly cut down search costs.

Application in autonomous driving shows potential for industry impact.

Discussion of FPGA and GPU contexts for DNN/implementation co-design.

Abstract

The rapidly growing demands for powerful AI algorithms in many application domains have motivated massive investment in both high-quality deep neural network (DNN) models and high-efficiency implementations. In this position paper, we argue that a simultaneous DNN/implementation co-design methodology, named Neural Architecture and Implementation Search (NAIS), deserves more research attention to boost the development productivity and efficiency of both DNN models and implementation optimization. We propose a stylized design methodology that can drastically cut down the search cost while preserving the quality of the end solution.As an illustration, we discuss this DNN/implementation methodology in the context of both FPGAs and GPUs. We take autonomous driving as a key use case as it is one of the most demanding areas for high quality AI algorithms and accelerators. We discuss how such a co-design methodology can impact the autonomous driving industry significantly. We identify several research opportunities in this exciting domain.