Generalized Learning Vector Quantization for Classification in Randomized Neural Networks and Hyperdimensional Computing

TL;DR

This paper introduces a modified RVFL network that replaces the least-squares classifier with GLVQ, reducing computational costs and achieving state-of-the-art accuracy on various datasets, suitable for resource-constrained edge devices.

Contribution

The paper presents a novel RVFL variant using GLVQ, eliminating expensive matrix operations and improving efficiency without sacrificing accuracy.

Findings

Achieved state-of-the-art accuracy on UCI datasets.

Reduced training computational costs by approximately 79%.

Maintained high accuracy with limited training iterations.

Abstract

Machine learning algorithms deployed on edge devices must meet certain resource constraints and efficiency requirements. Random Vector Functional Link (RVFL) networks are favored for such applications due to their simple design and training efficiency. We propose a modified RVFL network that avoids computationally expensive matrix operations during training, thus expanding the network's range of potential applications. Our modification replaces the least-squares classifier with the Generalized Learning Vector Quantization (GLVQ) classifier, which only employs simple vector and distance calculations. The GLVQ classifier can also be considered an improvement upon certain classification algorithms popularly used in the area of Hyperdimensional Computing. The proposed approach achieved state-of-the-art accuracy on a collection of datasets from the UCI Machine Learning Repository - higher than previously proposed RVFL networks. We further demonstrate that our approach still achieves high accuracy while severely limited in training iterations (using on average only 21% of the least-squares classifier computational costs).