Model Compression Techniques in Biometrics Applications: A Survey

TL;DR

This survey reviews various model compression techniques like quantization, distillation, and pruning in biometric applications, emphasizing their benefits, limitations, and future research directions including fairness considerations.

Contribution

It systematically analyzes existing compression methods in biometrics, compares their effectiveness, and highlights the importance of addressing bias and fairness in future research.

Findings

Quantization reduces model size with minimal accuracy loss.

Knowledge distillation transfers knowledge effectively between models.

Pruning simplifies models while maintaining performance.

Abstract

The development of deep learning algorithms has extensively empowered humanity's task automatization capacity. However, the huge improvement in the performance of these models is highly correlated with their increasing level of complexity, limiting their usefulness in human-oriented applications, which are usually deployed in resource-constrained devices. This led to the development of compression techniques that drastically reduce the computational and memory costs of deep learning models without significant performance degradation. This paper aims to systematize the current literature on this topic by presenting a comprehensive survey of model compression techniques in biometrics applications, namely quantization, knowledge distillation and pruning. We conduct a critical analysis of the comparative value of these techniques, focusing on their advantages and disadvantages and presenting suggestions for future work directions that can potentially improve the current methods. Additionally, we discuss and analyze the link between model bias and model compression, highlighting the need to direct compression research toward model fairness in future works.