An Extension of Fisher's Criterion: Theoretical Results with a Neural Network Realization

TL;DR

This paper extends Fisher's criterion to better handle classes with similar means by leveraging heteroscedasticity, and demonstrates its neural network implementation through a proof-of-concept experiment.

Contribution

It introduces a novel extension of Fisher's criterion that accounts for heteroscedasticity and shows how to implement it within a neural network framework.

Findings

Extension improves class separation when means are close

Neural network implementation is feasible

Proof-of-concept demonstrates potential for classification tasks

Abstract

Fisher's criterion is a widely used tool in machine learning for feature selection. For large search spaces, Fisher's criterion can provide a scalable solution to select features. A challenging limitation of Fisher's criterion, however, is that it performs poorly when mean values of class-conditional distributions are close to each other. Motivated by this challenge, we propose an extension of Fisher's criterion to overcome this limitation. The proposed extension utilizes the available heteroscedasticity of class-conditional distributions to distinguish one class from another. Additionally, we describe how our theoretical results can be casted into a neural network framework, and conduct a proof-of-concept experiment to demonstrate the viability of our approach to solve classification problems.