Optimisation of on-line principal component analysis

TL;DR

This paper investigates various optimization techniques for online principal component analysis using statistical mechanics, introducing methods that significantly accelerate learning and improve training efficiency, validated through simulations.

Contribution

It introduces new optimization strategies for online PCA, focusing on local and global learning-rate adjustments to enhance learning speed and training performance.

Findings

Optimized learning rates significantly speed up PCA training.

New methods improve training efficiency demonstrated by simulations.

Insights into learning rate dynamics inform practical optimization techniques.

Abstract

Different techniques, used to optimise on-line principal component analysis, are investigated by methods of statistical mechanics. These include local and global optimisation of node-dependent learning-rates which are shown to be very efficient in speeding up the learning process. They are investigated further for gaining insight into the learning rates' time-dependence, which is then employed for devising simple practical methods to improve training performance. Simulations demonstrate the benefit gained from using the new methods.