Exploiting Linear Structure Within Convolutional Networks for Efficient Evaluation

TL;DR

This paper introduces methods to accelerate large convolutional networks for object recognition by exploiting linear structures in filters, achieving 2x speedups with minimal accuracy loss on CPU and GPU.

Contribution

It presents novel approximation techniques that leverage linear structures in convolutional filters to significantly reduce computation during evaluation.

Findings

2x speedup in convolutional layer evaluation

Accuracy within 1% of original models

Effective on both CPU and GPU

Abstract

We present techniques for speeding up the test-time evaluation of large convolutional networks, designed for object recognition tasks. These models deliver impressive accuracy but each image evaluation requires millions of floating point operations, making their deployment on smartphones and Internet-scale clusters problematic. The computation is dominated by the convolution operations in the lower layers of the model. We exploit the linear structure present within the convolutional filters to derive approximations that significantly reduce the required computation. Using large state-of-the-art models, we demonstrate we demonstrate speedups of convolutional layers on both CPU and GPU by a factor of 2x, while keeping the accuracy within 1% of the original model.