A Bi-level Nonlinear Eigenvector Algorithm for Wasserstein Discriminant Analysis

TL;DR

This paper introduces WDA-nepv, a novel bi-level nonlinear eigenvector algorithm for Wasserstein discriminant analysis that effectively captures data class structures using optimal transport principles, with proven convergence and improved efficiency.

Contribution

The paper presents a new eigenvector-based algorithm for WDA that fully exploits the bi-level optimization structure, offering a derivative-free and scalable solution.

Findings

Demonstrates high classification accuracy on synthetic and real datasets.

Shows improved scalability and efficiency over existing methods.

Provides convergence analysis supporting the algorithm's robustness.

Abstract

Much like the classical Fisher linear discriminant analysis (LDA), the recently proposed Wasserstein discriminant analysis (WDA) is a linear dimensionality reduction method that seeks a projection matrix to maximize the dispersion of different data classes and minimize the dispersion of same data classes via a bi-level optimization. In contrast to LDA, WDA can account for both global and local interconnections between data classes by using the underlying principles of optimal transport. In this paper, a bi-level nonlinear eigenvector algorithm (WDA-nepv) is presented to fully exploit the structures of the bi-level optimization of WDA. The inner level of WDA-nepv for computing the optimal transport matrices is formulated as an eigenvector-dependent nonlinear eigenvalue problem (NEPv), and meanwhile, the outer level for trace ratio optimizations is formulated as another NEPv. Both NEPvs can be computed efficiently under the self-consistent field (SCF) framework. WDA-nepv is derivative-free and surrogate-model-free when compared with existing algorithms. Convergence analysis of the proposed WDA-nepv justifies the utilization of the SCF for solving the bi-level optimization of WDA. Numerical experiments with synthetic and real-life datasets demonstrate the classification accuracy and scalability of WDA-nepv.