Fixed-point Factorized Networks

TL;DR

This paper introduces Fixed-point Factorized Networks (FFN), a method to significantly reduce the computational and storage demands of deep neural networks by constraining weights to -1, 0, and 1, enabling efficient deployment on resource-limited devices.

Contribution

The paper proposes FFN, a novel network quantization approach that drastically cuts down multiply-accumulate operations while maintaining accuracy on large-scale image classification.

Findings

FFN reduces multiply operations to one-thousandth of original.

FFN achieves comparable accuracy with significantly fewer resources.

Extensive experiments validate the efficiency of FFN on ImageNet.

Abstract

In recent years, Deep Neural Networks (DNN) based methods have achieved remarkable performance in a wide range of tasks and have been among the most powerful and widely used techniques in computer vision. However, DNN-based methods are both computational-intensive and resource-consuming, which hinders the application of these methods on embedded systems like smart phones. To alleviate this problem, we introduce a novel Fixed-point Factorized Networks (FFN) for pretrained models to reduce the computational complexity as well as the storage requirement of networks. The resulting networks have only weights of -1, 0 and 1, which significantly eliminates the most resource-consuming multiply-accumulate operations (MACs). Extensive experiments on large-scale ImageNet classification task show the proposed FFN only requires one-thousandth of multiply operations with comparable accuracy.