Confusion-based rank similarity filters for computationally-efficient machine learning on high dimensional data

TL;DR

This paper introduces the rank similarity filter (RSF), a computationally efficient neural network component that transforms and classifies high-dimensional, nonlinearly separable data using rank-based similarity measures, enabling faster and effective analysis.

Contribution

The paper presents a novel rank similarity filter (RSF) for neural networks, along with the rank similarity transform (RST), and classifiers RSC and RSPC, demonstrating improved computational efficiency for high-dimensional data.

Findings

RSC classifier is competitive with existing methods.

RSF-based methods are faster and scalable.

Open-source Python implementation available.

Abstract

We introduce a novel type of computationally efficient artificial neural network (ANN) called the rank similarity filter (RSF). RSFs can be used to both transform and classify nonlinearly separable datasets with many data points and dimensions. The weights of RSF are set using the rank orders of features in a data point, or optionally the 'confusion' adjusted ranks between features (determined from their distributions in the dataset). The activation strength of a filter determines its similarity to other points in the dataset, a measure related to cosine similarity. The activation of many RSFs maps samples into a new nonlinear space suitable for linear classification (the rank similarity transform (RST)). We additionally used this method to create the nonlinear rank similarity classifier (RSC), which is a fast and accurate multiclass classifier, and the nonlinear rank similarity probabilistic classifier (RSPC), which is an extension to the multilabel case. We evaluated the classifiers on multiple datasets and RSC was competitive with existing classifiers but with superior computational efficiency. Open-source code for RST, RSC and RSPC was written in Python using the popular scikit-learn framework to make it easily accessible. In future extensions the algorithm can be applied to specialised hardware suitable for the parallelization of an ANN (GPU) and a Spiking Neural Network (neuromorphic computing) with corresponding performance gains. This makes RSF a promising solution to the problem of efficient analysis of nonlinearly separable data.