CNN Mixture-of-Depths

TL;DR

The paper presents Mixture-of-Depths, a CNN method that improves computational efficiency by dynamically selecting relevant channels for processing, leading to faster inference without sacrificing accuracy.

Contribution

Introduces a static-graph, channel-selective approach called Mixture-of-Depths for CNNs that enhances efficiency without complex dynamic computation.

Findings

ResNet86-MoD outperforms ResNet50 by 0.45% on ImageNet.

ResNet75-MoD matches ResNet50 performance with 25% CPU speedup.

Method speeds up training and inference without specialized hardware or loss functions.

Abstract

We introduce Mixture-of-Depths (MoD) for Convolutional Neural Networks (CNNs), a novel approach that enhances the computational efficiency of CNNs by selectively processing channels based on their relevance to the current prediction. This method optimizes computational resources by dynamically selecting key channels in feature maps for focused processing within the convolutional blocks (Conv-Blocks), while skipping less relevant channels. Unlike conditional computation methods that require dynamic computation graphs, CNN MoD uses a static computation graph with fixed tensor sizes which improve hardware efficiency. It speeds up the training and inference processes without the need for customized CUDA kernels, unique loss functions, or finetuning. CNN MoD either matches the performance of traditional CNNs with reduced inference times, GMACs, and parameters, or exceeds their performance while maintaining similar inference times, GMACs, and parameters. For example, on ImageNet, ResNet86-MoD exceeds the performance of the standard ResNet50 by 0.45% with a 6% speedup on CPU and 5% on GPU. Moreover, ResNet75-MoD achieves the same performance as ResNet50 with a 25% speedup on CPU and 15% on GPU.