Engineering Resource-constrained Software Systems with DNN Components: a Concept-based Pruning Approach

TL;DR

This paper introduces a concept-based pruning method for DNNs that uses human-interpretable features to produce smaller, efficient models suitable for resource-constrained software systems.

Contribution

A novel concept-guided pruning approach that leverages human-interpretable concepts to optimize DNNs for practical resource limitations in software engineering.

Findings

Concept-based pruning produces significantly smaller, effective DNNs.

Pruned DNNs show improved computational efficiency and performance.

Alternative configurations allow trade-offs for different practical needs.

Abstract

Deep Neural Networks (DNNs) are widely used by engineers to solve difficult problems that require predictive modeling from data. However, these models are often massive, with millions or billions of parameters, and require substantial computational power, RAM, and storage. This becomes a limitation in practical scenarios where strict size and resource constraints must be respected. In this paper, we present a novel concept-based pruning technique for DNNs that guides pruning decisions using human-interpretable concepts, such as features, colors, and classes. This is particularly important in a software engineering context, as DNNs are integrated into systems and must be pruned according to specific system requirements. Our concept-based pruning solution analyzes neuron activations to identify important neurons from a system requirements viewpoint and uses this information to guide the DNN pruning. We assess our solution using the VGG-19 network and a dataset of 26'384 RGB images, focusing on its ability to produce small, effective pruned DNNs and on the computational complexity and performance of these pruned DNNs. We also analyzed the pruning efficiency of our solution and compared alternative configurations. Our results show that concept-based pruning efficiently generates much smaller, effective pruned DNNs. Pruning greatly improves the computational efficiency and performance of DNNs, properties that are particularly useful for practical applications with stringent memory and computational time constraints. Finally, alternative configuration options enable engineers to identify trade-offs adapted to different practical situations.