Multi-Activation Hidden Units for Neural Networks with Random Weights

TL;DR

This paper introduces multi-activation hidden units in single-layer neural networks with random weights, enhancing their ability to form complex decision boundaries without increasing hidden units, thus improving accuracy or reducing computation.

Contribution

It proposes multi-activation hidden units to increase model flexibility and efficiency in random weight neural networks, a novel approach compared to traditional single-activation units.

Findings

Improved classification accuracy with multi-activation units

Reduced computational complexity in neural networks

Enhanced decision surface formation without more hidden units

Abstract

Single layer feedforward networks with random weights are successful in a variety of classification and regression problems. These networks are known for their non-iterative and fast training algorithms. A major drawback of these networks is that they require a large number of hidden units. In this paper, we propose the use of multi-activation hidden units. Such units increase the number of tunable parameters and enable formation of complex decision surfaces, without increasing the number of hidden units. We experimentally show that multi-activation hidden units can be used either to improve the classification accuracy, or to reduce computations.