Unified Framework for Pre-trained Neural Network Compression via Decomposition and Optimized Rank Selection

TL;DR

This paper introduces a unified framework for neural network compression that combines tensor decomposition with automatic rank selection, enabling efficient model size reduction while preserving accuracy.

Contribution

It proposes an integrated method for decomposition and rank optimization with automatic search, reducing computational costs and eliminating the need for extra training data.

Findings

Effective model compression with maintained accuracy

Automatic rank selection improves efficiency

Validated on multiple benchmarks

Abstract

Despite their high accuracy, complex neural networks demand significant computational resources, posing challenges for deployment on resource constrained devices such as mobile phones and embedded systems. Compression algorithms have been developed to address these challenges by reducing model size and computational demands while maintaining accuracy. Among these approaches, factorization methods based on tensor decomposition are theoretically sound and effective. However, they face difficulties in selecting the appropriate rank for decomposition. This paper tackles this issue by presenting a unified framework that simultaneously applies decomposition and rank selection, employing a composite compression loss within defined rank constraints. Our method includes an automatic rank search in a continuous space, efficiently identifying optimal rank configurations for the pre-trained model by eliminating the need for additional training data and reducing computational overhead in the search step. Combined with a subsequent fine-tuning step, our approach maintains the performance of highly compressed models on par with their original counterparts. Using various benchmark datasets and models, we demonstrate the efficacy of our method through a comprehensive analysis.