Fast Multi-Class Probabilistic Classifier by Sparse Non-parametric Density Estimation

TL;DR

This paper introduces a novel sparse non-parametric density estimation method for multi-class classification that enhances model interpretability by identifying impactful variables, supported by theoretical analysis and empirical validation.

Contribution

It proposes a new variable selection technique for nonparametric multi-class classifiers based on sparse density estimation, filling a gap in existing research.

Findings

The method effectively identifies important variables for each class.

Theoretical properties and asymptotic behavior are established.

Numerical experiments demonstrate improved interpretability and performance.

Abstract

The model interpretation is essential in many application scenarios and to build a classification model with a ease of model interpretation may provide useful information for further studies and improvement. It is common to encounter with a lengthy set of variables in modern data analysis, especially when data are collected in some automatic ways. This kinds of datasets may not collected with a specific analysis target and usually contains redundant features, which have no contribution to a the current analysis task of interest. Variable selection is a common way to increase the ability of model interpretation and is popularly used with some parametric classification models. There is a lack of studies about variable selection in nonparametric classification models such as the density estimation-based methods and this is especially the case for multiple-class classification situations. In this study we study multiple-class classification problems using the thought of sparse non-parametric density estimation and propose a method for identifying high impacts variables for each class. We present the asymptotic properties and the computation procedure for the proposed method together with some suggested sample size. We also repost the numerical results using both synthesized and some real data sets.