Deep Network Pruning: A Comparative Study on CNNs in Face Recognition

TL;DR

This paper compares various deep network pruning methods for CNNs used in face recognition, demonstrating that significant compression is achievable with minimal accuracy loss, especially by removing less important filters based on Taylor scores.

Contribution

It provides a comparative analysis of pruning techniques on different CNN architectures for face recognition, highlighting the effectiveness of Taylor score-based pruning in reducing model size.

Findings

Substantial filter removal possible with minimal performance loss

High-dimensional output channels are often over-dimensioned

Pruning effectiveness varies across different CNN architectures

Abstract

The widespread use of mobile devices for all kinds of transactions makes necessary reliable and real-time identity authentication, leading to the adoption of face recognition (FR) via the cameras embedded in such devices. Progress of deep Convolutional Neural Networks (CNNs) has provided substantial advances in FR. Nonetheless, the size of state-of-the-art architectures is unsuitable for mobile deployment, since they often encompass hundreds of megabytes and millions of parameters. We address this by studying methods for deep network compression applied to FR. In particular, we apply network pruning based on Taylor scores, where less important filters are removed iteratively. The method is tested on three networks based on the small SqueezeNet (1.24M parameters) and the popular MobileNetv2 (3.5M) and ResNet50 (23.5M) architectures. These have been selected to showcase the method on CNNs with different complexities and sizes. We observe that a substantial percentage of filters can be removed with minimal performance loss. Also, filters with the highest amount of output channels tend to be removed first, suggesting that high-dimensional spaces within popular CNNs are over-dimensioned.