On Supervised Classification of Feature Vectors with Independent and Non-Identically Distributed Elements

TL;DR

This paper addresses the classification of independent but non-identically distributed feature vectors, proposing an asymptotically optimal classifier with improved performance in low-data, high-dimensional scenarios.

Contribution

It introduces a new classifier for independent, non-i.i.d. feature vectors and derives an error bound showing its effectiveness with limited training data.

Findings

Error probability approaches zero with increasing feature vector length

Proposed classifier outperforms conventional algorithms in small training data regimes

Performance improves as feature vector length increases

Abstract

In this paper, we investigate the problem of classifying feature vectors with mutually independent but non-identically distributed elements. First, we show the importance of this problem. Next, we propose a classifier and derive an analytical upper bound on its error probability. We show that the error probability goes to zero as the length of the feature vectors grows, even when there is only one training feature vector per label available. Thereby, we show that for this important problem at least one asymptotically optimal classifier exists. Finally, we provide numerical examples where we show that the performance of the proposed classifier outperforms conventional classification algorithms when the number of training data is small and the length of the feature vectors is sufficiently high.