Eigenvalue and Generalized Eigenvalue Problems: Tutorial

TL;DR

This tutorial explains eigenvalue and generalized eigenvalue problems, their optimization origins, and applications in machine learning like PCA and Fisher analysis, providing solution methods.

Contribution

It offers a comprehensive overview of eigenvalue problems, connecting theory with practical machine learning applications and solution techniques.

Findings

Eigenvalue problems are fundamental in spectral analysis.

Applications include PCA, kernel PCA, and Fisher discriminant analysis.

Provides solution methods for eigenvalue and generalized eigenvalue problems.

Abstract

This paper is a tutorial for eigenvalue and generalized eigenvalue problems. We first introduce eigenvalue problem, eigen-decomposition (spectral decomposition), and generalized eigenvalue problem. Then, we mention the optimization problems which yield to the eigenvalue and generalized eigenvalue problems. We also provide examples from machine learning, including principal component analysis, kernel supervised principal component analysis, and Fisher discriminant analysis, which result in eigenvalue and generalized eigenvalue problems. Finally, we introduce the solutions to both eigenvalue and generalized eigenvalue problems.