RapidNet: Multi-Level Dilated Convolution Based Mobile Backbone

TL;DR

RapidNet introduces multi-level dilated convolutions to create a purely CNN-based mobile backbone that surpasses state-of-the-art models in accuracy and speed for various vision tasks on mobile devices.

Contribution

This work proposes a novel multi-level dilated convolution approach for CNNs, enabling larger receptive fields and better feature interaction, leading to superior mobile vision model performance.

Findings

Outperforms SOTA mobile CNN, ViT, ViG, and hybrid models in accuracy and speed.

RapidNet-Ti achieves 76.3% top-1 accuracy on ImageNet-1K with 0.9 ms latency.

Pure CNN architectures can surpass hybrid and ViT models when properly designed.

Abstract

Vision transformers (ViTs) have dominated computer vision in recent years. However, ViTs are computationally expensive and not well suited for mobile devices; this led to the prevalence of convolutional neural network (CNN) and ViT-based hybrid models for mobile vision applications. Recently, Vision GNN (ViG) and CNN hybrid models have also been proposed for mobile vision tasks. However, all of these methods remain slower compared to pure CNN-based models. In this work, we propose Multi-Level Dilated Convolutions to devise a purely CNN-based mobile backbone. Using Multi-Level Dilated Convolutions allows for a larger theoretical receptive field than standard convolutions. Different levels of dilation also allow for interactions between the short-range and long-range features in an image. Experiments show that our proposed model outperforms state-of-the-art (SOTA) mobile CNN, ViT, ViG, and hybrid architectures in terms of accuracy and/or speed on image classification, object detection, instance segmentation, and semantic segmentation. Our fastest model, RapidNet-Ti, achieves 76.3\% top-1 accuracy on ImageNet-1K with 0.9 ms inference latency on an iPhone 13 mini NPU, which is faster and more accurate than MobileNetV2x1.4 (74.7\% top-1 with 1.0 ms latency). Our work shows that pure CNN architectures can beat SOTA hybrid and ViT models in terms of accuracy and speed when designed properly.