A Channel-Pruned and Weight-Binarized Convolutional Neural Network for Keyword Spotting

TL;DR

This paper proposes a method combining channel pruning and weight binarization to create a compact, efficient CNN for keyword spotting, achieving over 50% channel sparsity with minimal accuracy loss.

Contribution

It introduces a group-wise splitting method with group Lasso penalty and a three-stage training procedure for effective network compression.

Findings

Achieved over 50% channel sparsity without significant accuracy loss

Effective combination of channel pruning and weight binarization

Maintained high performance with a highly compressed model

Abstract

We study channel number reduction in combination with weight binarization (1-bit weight precision) to trim a convolutional neural network for a keyword spotting (classification) task. We adopt a group-wise splitting method based on the group Lasso penalty to achieve over 50% channel sparsity while maintaining the network performance within 0.25% accuracy loss. We show an effective three-stage procedure to balance accuracy and sparsity in network training.