Scaling Up Your Kernels to 31x31: Revisiting Large Kernel Design in CNNs

TL;DR

This paper introduces RepLKNet, a CNN architecture with large 31x31 kernels inspired by vision transformers, achieving competitive performance and scalability, and challenging the dominance of small kernels in CNN design.

Contribution

The paper proposes a new large-kernel CNN design guideline and introduces RepLKNet, demonstrating that large kernels can outperform small ones in modern CNNs.

Findings

RepLKNet achieves 87.8% top-1 accuracy on ImageNet.

Large-kernel CNNs have larger receptive fields and higher shape bias.

RepLKNet outperforms comparable models on downstream tasks.

Abstract

We revisit large kernel design in modern convolutional neural networks (CNNs). Inspired by recent advances in vision transformers (ViTs), in this paper, we demonstrate that using a few large convolutional kernels instead of a stack of small kernels could be a more powerful paradigm. We suggested five guidelines, e.g., applying re-parameterized large depth-wise convolutions, to design efficient high-performance large-kernel CNNs. Following the guidelines, we propose RepLKNet, a pure CNN architecture whose kernel size is as large as 31x31, in contrast to commonly used 3x3. RepLKNet greatly closes the performance gap between CNNs and ViTs, e.g., achieving comparable or superior results than Swin Transformer on ImageNet and a few typical downstream tasks, with lower latency. RepLKNet also shows nice scalability to big data and large models, obtaining 87.8% top-1 accuracy on ImageNet and 56.0% mIoU on ADE20K, which is very competitive among the state-of-the-arts with similar model sizes. Our study further reveals that, in contrast to small-kernel CNNs, large-kernel CNNs have much larger effective receptive fields and higher shape bias rather than texture bias. Code & models at https://github.com/megvii-research/RepLKNet.