Dynamic Multi-path Neural Network

TL;DR

The paper introduces Dynamic Multi-path Neural Networks (DMNN), which adaptively select network paths during inference to improve efficiency and accuracy, especially for resource-constrained applications.

Contribution

It proposes a novel dynamic multi-path approach that adjusts both width and depth during inference, enhancing flexibility and performance over traditional static networks.

Findings

DMNN-101 reduces FLOPs by 45.1% while outperforming ResNet-101.

DMNN-50 achieves similar accuracy to ResNet-101 with 42.1% fewer parameters.

Experimental results on ImageNet and CIFAR-100 validate the effectiveness of DMNN.

Abstract

Although deeper and larger neural networks have achieved better performance, the complex network structure and increasing computational cost cannot meet the demands of many resource-constrained applications. Existing methods usually choose to execute or skip an entire specific layer, which can only alter the depth of the network. In this paper, we propose a novel method called Dynamic Multi-path Neural Network (DMNN), which provides more path selection choices in terms of network width and depth during inference. The inference path of the network is determined by a controller, which takes into account both previous state and object category information. The proposed method can be easily incorporated into most modern network architectures. Experimental results on ImageNet and CIFAR-100 demonstrate the superiority of our method on both efficiency and overall classification accuracy. To be specific, DMNN-101 significantly outperforms ResNet-101 with an encouraging 45.1% FLOPs reduction, and DMNN-50 performs comparably to ResNet-101 while saving 42.1% parameters.