Lightweight Design and Optimization methods for DCNNs: Progress and Futures

TL;DR

This paper reviews recent strategies for designing and compressing lightweight Deep Convolutional Neural Networks (DCNNs) to enable their effective deployment on resource-limited devices, highlighting progress, limitations, and future directions.

Contribution

It provides a comprehensive overview of lightweight architectural design and model compression techniques for DCNNs, along with insights into current challenges and future research prospects.

Findings

Summarizes recent advances in lightweight DCNN architectures.

Analyzes current limitations in model compression methods.

Proposes future research directions for lightweight neural networks.

Abstract

Lightweight design, as a key approach to mitigate disparity between computational requirements of deep learning models and hardware performance, plays a pivotal role in advancing application of deep learning technologies on mobile and embedded devices, alongside rapid development of smart home, telemedicine, and autonomous driving. With its outstanding feature extracting capabilities, Deep Convolutional Neural Networks (DCNNs) have demonstrated superior performance in computer vision tasks. However, high computational costs and large network architectures severely limit the widespread application of DCNNs on resource-constrained hardware platforms such as smartphones, robots, and IoT devices. This paper reviews lightweight design strategies for DCNNs and examines recent research progress in both lightweight architectural design and model compression. Additionally, this paper discusses current limitations in this field of research and propose prospects for future directions, aiming to provide valuable guidance and reflection for lightweight design philosophy on deep neural networks in the field of computer vision.