Classification of Functional Data with k-Nearest-Neighbor Ensembles by Fitting Constrained Multinomial Logit Models

TL;DR

This paper introduces a novel functional data classification method combining ensemble learning with a penalized constrained multinomial logit model, enabling automatic feature selection and interpretability, and demonstrating superior performance in real-world applications.

Contribution

It proposes a new classification approach that integrates functional ensembles with penalized multinomial logit models, allowing for feature selection and interpretability in functional data analysis.

Findings

Effective feature selection with sparsity-inducing penalties

Improved classification accuracy over existing methods

Enhanced interpretability through feature importance measures

Abstract

During the last decades, many methods for the analysis of functional data including classification methods have been developed. Nonetheless, there are issues that have not been adressed satisfactorily by currently available methods, as, for example, feature selection combined with variable selection when using multiple functional covariates. In this paper, a functional ensemble is combined with a penalized and constrained multinomial logit model. It is shown that this synthesis yields a powerful classification tool for functional data (possibly mixed with non-functional predictors), which also provides automatic variable selection. The choice of an appropriate, sparsity-inducing penalty allows to estimate most model coefficients to exactly zero, and permits class-specific coefficients in multiclass problems, such that feature selection is obtained. An additional constraint within the multinomial logit model ensures that the model coefficients can be considered as weights. Thus, the estimation results become interpretable with respect to the discriminative importance of the selected features, which is rated by a feature importance measure. In two application examples, data of a cell chip used for water quality monitoring experiments and phoneme data used for speech recognition, the interpretability as well as the selection results are examined. The classification performance is compared to various other classification approaches which are in common use.