Partial Channel Network: Compute Fewer, Perform Better

TL;DR

This paper introduces PartialNet, a network architecture utilizing partial channel mechanisms and attention to reduce parameters and FLOPs while maintaining or improving accuracy across vision tasks.

Contribution

It proposes the partial channel mechanism, novel partial attention convolution, and dynamic partial convolution, enabling efficient models that outperform state-of-the-art methods in accuracy and speed.

Findings

PartialNet achieves superior accuracy and inference speed on ImageNet-1K.

PATConv can replace regular convolution and attention with fewer parameters.

The dynamic partial convolution adapts channel splits for better trade-offs.

Abstract

Designing a module or mechanism that enables a network to maintain low parameters and FLOPs without sacrificing accuracy and throughput remains a challenge. To address this challenge and exploit the redundancy within feature map channels, we propose a new solution: partial channel mechanism (PCM). Specifically, through the split operation, the feature map channels are divided into different parts, with each part corresponding to different operations, such as convolution, attention, pooling, and identity mapping. Based on this assumption, we introduce a novel partial attention convolution (PATConv) that can efficiently combine convolution with visual attention. Our exploration indicates that the PATConv can completely replace both the regular convolution and the regular visual attention while reducing model parameters and FLOPs. Moreover, PATConv can derive three new types of blocks: Partial Channel-Attention block (PAT_ch), Partial Spatial-Attention block (PAT_sp), and Partial Self-Attention block (PAT_sf). In addition, we propose a novel dynamic partial convolution (DPConv) that can adaptively learn the proportion of split channels in different layers to achieve better trade-offs. Building on PATConv and DPConv, we propose a new hybrid network family, named PartialNet, which achieves superior top-1 accuracy and inference speed compared to some SOTA models on ImageNet-1K classification and excels in both detection and segmentation on the COCO dataset. Our code is available at https://github.com/haiduo/PartialNet.