Trace Ratio Optimization with an Application to Multi-view Learning

TL;DR

This paper studies trace ratio optimization on the Stiefel manifold, develops a convergent numerical method, and applies it to multi-view learning, demonstrating improved efficiency and effectiveness on real data.

Contribution

It introduces a novel numerical method for trace ratio optimization and applies it to develop new multi-view subspace learning models.

Findings

Numerical method based on SCF iteration is always convergent.

New multi-view subspace learning models outperform existing methods.

Proposed models are effective on real-world datasets.

Abstract

A trace ratio optimization problem over the Stiefel manifold is investigated from the perspectives of both theory and numerical computations. At least three special cases of the problem have arisen from Fisher linear discriminant analysis, canonical correlation analysis, and unbalanced Procrustes problem, respectively. Necessary conditions in the form of nonlinear eigenvalue problem with eigenvector dependency are established and a numerical method based on the self-consistent field (SCF) iteration is designed and proved to be always convergent. As an application to multi-view subspace learning, a new framework and its instantiated concrete models are proposed and demonstrated on real world data sets. Numerical results show that the efficiency of the proposed numerical methods and effectiveness of the new multi-view subspace learning models.