SBCFormer: Lightweight Network Capable of Full-size ImageNet Classification at 1 FPS on Single Board Computers

TL;DR

SBCFormer is a novel lightweight CNN-ViT hybrid network designed for full-size ImageNet classification on low-end single board computers, achieving high accuracy at 1 FPS on Raspberry Pi 4.

Contribution

It introduces an architectural design that balances attention mechanism efficiency and local detail preservation for SBCs, enabling high-accuracy ImageNet classification.

Findings

Achieves around 80% top-1 accuracy on ImageNet-1K.

Runs at 1 frame/sec on Raspberry Pi 4.

Outperforms existing lightweight models in accuracy-speed trade-off.

Abstract

Computer vision has become increasingly prevalent in solving real-world problems across diverse domains, including smart agriculture, fishery, and livestock management. These applications may not require processing many image frames per second, leading practitioners to use single board computers (SBCs). Although many lightweight networks have been developed for mobile/edge devices, they primarily target smartphones with more powerful processors and not SBCs with the low-end CPUs. This paper introduces a CNN-ViT hybrid network called SBCFormer, which achieves high accuracy and fast computation on such low-end CPUs. The hardware constraints of these CPUs make the Transformer's attention mechanism preferable to convolution. However, using attention on low-end CPUs presents a challenge: high-resolution internal feature maps demand excessive computational resources, but reducing their resolution results in the loss of local image details. SBCFormer introduces an architectural design to address this issue. As a result, SBCFormer achieves the highest trade-off between accuracy and speed on a Raspberry Pi 4 Model B with an ARM-Cortex A72 CPU. For the first time, it achieves an ImageNet-1K top-1 accuracy of around 80% at a speed of 1.0 frame/sec on the SBC. Code is available at https://github.com/xyongLu/SBCFormer.