From Algorithm to Hardware: A Survey on Efficient and Safe Deployment of Deep Neural Networks

TL;DR

This survey comprehensively reviews recent advances in efficient, secure, and hardware-accelerated deployment of deep neural networks, covering model compression, hardware design, and security techniques.

Contribution

It provides an integrated overview of recent methods in model compression, hardware acceleration, and security for DNN deployment, highlighting their interconnections and challenges.

Findings

Model compression techniques significantly reduce DNN resource requirements.

Hardware accelerators are increasingly tailored to compressed models.

Homomorphic encryption enhances DNN security during deployment.

Abstract

Deep neural networks (DNNs) have been widely used in many artificial intelligence (AI) tasks. However, deploying them brings significant challenges due to the huge cost of memory, energy, and computation. To address these challenges, researchers have developed various model compression techniques such as model quantization and model pruning. Recently, there has been a surge in research of compression methods to achieve model efficiency while retaining the performance. Furthermore, more and more works focus on customizing the DNN hardware accelerators to better leverage the model compression techniques. In addition to efficiency, preserving security and privacy is critical for deploying DNNs. However, the vast and diverse body of related works can be overwhelming. This inspires us to conduct a comprehensive survey on recent research toward the goal of high-performance, cost-efficient, and safe deployment of DNNs. Our survey first covers the mainstream model compression techniques such as model quantization, model pruning, knowledge distillation, and optimizations of non-linear operations. We then introduce recent advances in designing hardware accelerators that can adapt to efficient model compression approaches. Additionally, we discuss how homomorphic encryption can be integrated to secure DNN deployment. Finally, we discuss several issues, such as hardware evaluation, generalization, and integration of various compression approaches. Overall, we aim to provide a big picture of efficient DNNs, from algorithm to hardware accelerators and security perspectives.