Invariant Coordinate Selection and Fisher discriminant subspace beyond the case of two groups

TL;DR

This paper extends the theoretical understanding of Invariant Coordinate Selection (ICS), demonstrating its effectiveness in recovering Fisher discriminant subspaces across multiple clusters and scatter combinations, beyond the two-group case.

Contribution

It generalizes the theoretical connection between ICS and Fisher discriminant subspace to multiple clusters and various scatter matrices, including cases where the group centers matrix is not full rank.

Findings

ICS effectively recovers Fisher discriminant subspace in multi-cluster scenarios.

Theoretical links between ICS and FDS are strengthened for full-rank group centers.

Cases of ICS failure are rare under general conditions.

Abstract

Invariant Coordinate Selection (ICS) is a multivariate technique that relies on the simultaneous diagonalization of two scatter matrices. It serves various purposes, including its use as a dimension reduction tool prior to clustering or outlier detection. ICS's theoretical foundation establishes why and when the identified subspace should contain relevant information by demonstrating its connection with the Fisher discriminant subspace (FDS). These general results have been examined in detail primarily for specific scatter combinations within a two-cluster framework. In this study, we expand these investigations to include more clusters and scatter combinations. Our analysis reveals the importance of distinguishing whether the group centers matrix has full rank. In the full-rank case, we establish deeper connections between ICS and FDS. We provide a detailed study of these relationships for three clusters when the group centers matrix has full rank and when it does not. Based on these expanded theoretical insights and supported by numerical studies, we conclude that ICS is indeed suitable for recovering the FDS under very general settings and cases of failure seem rare.