Big-Little Net: An Efficient Multi-Scale Feature Representation for Visual and Speech Recognition

TL;DR

Big-Little Net introduces a multi-scale CNN architecture that efficiently balances speed and accuracy, improving performance in visual and speech recognition tasks by merging features at different scales.

Contribution

It presents a novel multi-branch CNN design with feature merging for enhanced efficiency and accuracy, outperforming existing CNN acceleration methods.

Findings

Reduces object recognition computation by 33% with 0.9% accuracy gain.

Saves 30% FLOPs in speech recognition with improved word error rates.

Surpasses state-of-the-art CNN acceleration approaches in accuracy and efficiency.

Abstract

In this paper, we propose a novel Convolutional Neural Network (CNN) architecture for learning multi-scale feature representations with good tradeoffs between speed and accuracy. This is achieved by using a multi-branch network, which has different computational complexity at different branches. Through frequent merging of features from branches at distinct scales, our model obtains multi-scale features while using less computation. The proposed approach demonstrates improvement of model efficiency and performance on both object recognition and speech recognition tasks,using popular architectures including ResNet and ResNeXt. For object recognition, our approach reduces computation by 33% on object recognition while improving accuracy with 0.9%. Furthermore, our model surpasses state-of-the-art CNN acceleration approaches by a large margin in accuracy and FLOPs reduction. On the task of speech recognition, our proposed multi-scale CNNs save 30% FLOPs with slightly better word error rates, showing good generalization across domains. The codes are available at https://github.com/IBM/BigLittleNet