Efficient Fusion of Sparse and Complementary Convolutions

TL;DR

This paper introduces a method for creating compact CNNs by using hand-crafted sparse kernels combined with an efficient module, achieving significant reductions in parameters and computation while maintaining or improving performance across vision tasks.

Contribution

The paper presents a novel approach to CNN compression using regular sparse kernels and an efficient fusion module, enabling practical speedups without specialized hardware.

Findings

Achieves 2-4x reduction in parameters and computation for classification and localization.

Produces a VGG-16-based Faster R-CNN detector 12.4x smaller and 3x faster.

Maintains or improves accuracy compared to baseline models.

Abstract

We propose a new method to create compact convolutional neural networks (CNNs) by exploiting sparse convolutions. Different from previous works that learn sparsity in models, we directly employ hand-crafted kernels with regular sparse patterns, which result in the computational gain in practice without sophisticated and dedicated software or hardware. The core of our approach is an efficient network module that linearly combines sparse kernels to yield feature representations as strong as those from regular kernels. We integrate this module into various network architectures and demonstrate its effectiveness on three vision tasks, object classification, localization and detection. For object classification and localization, our approach achieves comparable or better performance than several baselines and related works while providing lower computational costs with fewer parameters (on average, a $2-4\times$ reduction of convolutional parameters and computation). For object detection, our approach leads to a VGG-16-based Faster RCNN detector that is 12.4$\times$ smaller and about 3$\times$ faster than the baseline.