Improved Convergence Speed of Fully Symmetric Learning Rules for Principal Component Analysis

TL;DR

This paper introduces a modified objective function for symmetric PCA learning rules that significantly enhances convergence speed, especially when eigenvalues are close, by adding a term that increases the steepness of the objective.

Contribution

The authors propose a new modified objective function for symmetric PCA learning rules that improves convergence speed without altering fixed points or stability.

Findings

Convergence speed is improved with the modified objective.

The fixed points of the original rule are preserved.

Simulation results confirm faster convergence depending on the added term's weight.

Abstract

Fully symmetric learning rules for principal component analysis can be derived from a novel objective function suggested in our previous work. We observed that these learning rules suffer from slow convergence for covariance matrices where some principal eigenvalues are close to each other. Here we describe a modified objective function with an additional term which mitigates this convergence problem. We show that the learning rule derived from the modified objective function inherits all fixed points from the original learning rule (but may introduce additional ones). Also the stability of the inherited fixed points remains unchanged. Only the steepness of the objective function is increased in some directions. Simulations confirm that the convergence speed can be noticeably improved, depending on the weight factor of the additional term.