A Random Projection k Nearest Neighbours Ensemble for Classification via Extended Neighbourhood Rule

TL;DR

This paper introduces RPExNRule, a novel ensemble method that combines random projections and an extended neighborhood rule to improve kNN classification by enhancing diversity and feature preservation.

Contribution

It proposes a new ensemble framework using random projections and an extended neighborhood rule for kNN, offering increased randomness and feature retention.

Findings

Enhanced classification accuracy demonstrated over traditional kNN ensembles.

Improved diversity of base learners through random projections.

Effective feature preservation via the extended neighborhood rule.

Abstract

Ensembles based on k nearest neighbours (kNN) combine a large number of base learners, each constructed on a sample taken from a given training data. Typical kNN based ensembles determine the k closest observations in the training data bounded to a test sample point by a spherical region to predict its class. In this paper, a novel random projection extended neighbourhood rule (RPExNRule) ensemble is proposed where bootstrap samples from the given training data are randomly projected into lower dimensions for additional randomness in the base models and to preserve features information. It uses the extended neighbourhood rule (ExNRule) to fit kNN as base learners on randomly projected bootstrap samples.