# Cascaded Projection: End-to-End Network Compression and Acceleration

## TL;DR

This paper introduces Cascaded Projection, a novel data-driven method for deep neural network compression that achieves high accuracy, low memory usage, and fast processing by projecting layer channels into a low-dimensional space.

## Contribution

The paper presents a new end-to-end network compression technique using cascaded low-rank projections optimized via backpropagation and SGD, outperforming existing methods.

## Key findings

- Over 4x reduction in computations for VGG16 and ResNet.
- State-of-the-art accuracy on ImageNet with compressed networks.
- Effective in maintaining top-5 accuracy after compression.

## Abstract

We propose a data-driven approach for deep convolutional neural network compression that achieves high accuracy with high throughput and low memory requirements. Current network compression methods either find a low-rank factorization of the features that requires more memory, or select only a subset of features by pruning entire filter channels. We propose the Cascaded Projection (CaP) compression method that projects the output and input filter channels of successive layers to a unified low dimensional space based on a low-rank projection. We optimize the projection to minimize classification loss and the difference between the next layer's features in the compressed and uncompressed networks. To solve this non-convex optimization problem we propose a new optimization method of a proxy matrix using backpropagation and Stochastic Gradient Descent (SGD) with geometric constraints. Our cascaded projection approach leads to improvements in all critical areas of network compression: high accuracy, low memory consumption, low parameter count and high processing speed. The proposed CaP method demonstrates state-of-the-art results compressing VGG16 and ResNet networks with over 4x reduction in the number of computations and excellent performance in top-5 accuracy on the ImageNet dataset before and after fine-tuning.