Integer-Only Operations on Extreme Learning Machine Test Time Classification

TL;DR

This paper introduces techniques enabling test time classification using only integer operations in extreme learning machines, reducing computational costs without sacrificing accuracy, especially beneficial for embedded and data center applications.

Contribution

The paper provides a theoretical and empirical framework for integer-only test time classification in ELMs, including weight binarization and accuracy-preserving normalization.

Findings

Input weights can be ternary with minimal accuracy loss.

Normalized and non-normalized signals have equal accuracy.

Integer output weights maintain accuracy with limited reduction.

Abstract

We present a theoretical analysis and empirical evaluations of a novel set of techniques for computational cost reduction of test time operations of network classifiers based on extreme learning machine (ELM). By exploring some characteristics we derived from these models, we show that the classification at test time can be performed using solely integer operations without compromising the classification accuracy. Our contributions are as follows: (i) We show empirical evidence that the input weights values can be drawn from the ternary set with limited reduction of the classification accuracy. This has the computational advantage of dismissing multiplications; (ii) We prove the classification accuracy of normalized and non-normalized test signals are the same; (iii) We show how to create an integer version of the output weights that results in a limited reduction of the classification accuracy. We tested our techniques on 5 computer vision datasets commonly used in the literature and the results indicate that our techniques can allow the reduction of the computational cost of the operations necessary for the classification at test time in FPGAs. This is important in embedded applications, where power consumption is limited, and crucial in data centers of large corporations, where power consumption is expensive.