RepNeXt: A Fast Multi-Scale CNN using Structural Reparameterization

TL;DR

RepNeXt is a novel, efficient multi-scale CNN that uses structural reparameterization to improve speed and accuracy for resource-limited vision tasks, outperforming existing lightweight models.

Contribution

It introduces RepNeXt, a new CNN architecture combining multi-scale features and structural reparameterization to enhance performance without sacrificing inference speed.

Findings

RepNeXt-M4 achieves 82.3% accuracy on ImageNet within 1.5ms on an iPhone 12.

RepNeXt outperforms current lightweight CNNs and ViTs in latency and accuracy.

Parameters are reduced by 0.7 million compared to similar models.

Abstract

In the realm of resource-constrained mobile vision tasks, the pursuit of efficiency and performance consistently drives innovation in lightweight Convolutional Neural Networks (CNNs) and Vision Transformers (ViTs). While ViTs excel at capturing global context through self-attention mechanisms, their deployment in resource-limited environments is hindered by computational complexity and latency. Conversely, lightweight CNNs are favored for their parameter efficiency and low latency. This study investigates the complementary advantages of CNNs and ViTs to develop a versatile vision backbone tailored for resource-constrained applications. We introduce RepNeXt, a novel model series integrates multi-scale feature representations and incorporates both serial and parallel structural reparameterization (SRP) to enhance network depth and width without compromising inference speed. Extensive experiments demonstrate RepNeXt's superiority over current leading lightweight CNNs and ViTs, providing advantageous latency across various vision benchmarks. RepNeXt-M4 matches RepViT-M1.5's 82.3\% accuracy on ImageNet within 1.5ms on an iPhone 12, outperforms its AP$^{box}$ by 1.3 on MS-COCO, and reduces parameters by 0.7M. Codes and models are available at https://github.com/suous/RepNeXt.