Sparse Feature Selection in Kernel Discriminant Analysis via Optimal Scoring

TL;DR

This paper introduces a kernel-based classification method that incorporates structured sparsity for feature selection, with theoretical guarantees and automated tuning, demonstrating superior performance over existing classifiers.

Contribution

It presents a novel kernel classifier with structured sparsity and risk consistency guarantees, along with automated parameter tuning methods.

Findings

Demonstrates superior classification accuracy in numerical studies.

Provides theoretical guarantees on risk consistency.

Introduces automated tuning procedures for kernel parameters.

Abstract

We consider the two-group classification problem and propose a kernel classifier based on the optimal scoring framework. Unlike previous approaches, we provide theoretical guarantees on the expected risk consistency of the method. We also allow for feature selection by imposing structured sparsity using weighted kernels. We propose fully-automated methods for selection of all tuning parameters, and in particular adapt kernel shrinkage ideas for ridge parameter selection. Numerical studies demonstrate the superior classification performance of the proposed approach compared to existing nonparametric classifiers.