Supervised Quadratic Feature Analysis: Information Geometry Approach for Dimensionality Reduction

TL;DR

This paper introduces SQFA, a supervised linear dimensionality reduction method based on information geometry, which maximizes Fisher-Rao distances between class distributions to improve classification performance.

Contribution

SQFA is the first method to leverage Fisher-Rao distance for supervised feature extraction under Gaussian assumptions, offering a new geometric perspective.

Findings

Supports good classification with QDA on real datasets

Provides a novel information geometry framework for dimensionality reduction

Motivates future research in machine learning and neuroscience

Abstract

Supervised dimensionality reduction maps labeled data into a low-dimensional feature space while preserving class discriminability. A common approach is to maximize a statistical measure of dissimilarity between classes in the feature space. Information geometry provides an alternative framework for measuring class dissimilarity, with the potential for improved insights and novel applications. Information geometry, which is grounded in Riemannian geometry, uses the Fisher information metric, a local measure of discriminability that induces the Fisher-Rao distance. Here, we present Supervised Quadratic Feature Analysis (SQFA), a linear dimensionality reduction method that maximizes Fisher-Rao distances between class-conditional distributions, under Gaussian assumptions. We motivate the Fisher-Rao distance as a good proxy for discriminability. We show that SQFA features support good classification performance with Quadratic Discriminant Analysis (QDA) on three real-world datasets. SQFA provides a novel framework for supervised dimensionality reduction, motivating future research in applying information geometry to machine learning and neuroscience.