A Consistently Oriented Basis for Eigenanalysis

TL;DR

This paper introduces an algorithm that creates a consistently oriented basis for eigenvectors, improving stability and interpretability in eigenanalysis for evolving datasets and machine learning models.

Contribution

The work provides a postprocessing algorithm for eigenvectors that ensures consistent orientation, agnostic to implementation, enabling better analysis of eigenvector dynamics.

Findings

Eigenvector orientation can vary in standard implementations.

The algorithm achieves stable, consistent eigenvector orientations.

Consistent eigenvectors improve interpretability in machine learning models.

Abstract

Repeated application of machine-learning, eigen-centric methods to an evolving dataset reveals that eigenvectors calculated by well-established computer implementations are not stable along an evolving sequence. This is because the sign of any one eigenvector may point along either the positive or negative direction of its associated eigenaxis, and for any one eigen call the sign does not matter when calculating a solution. This work reports an algorithm that creates a consistently oriented basis of eigenvectors. The algorithm postprocesses any well-established eigen call and is therefore agnostic to the particular implementation of the latter. Once consistently oriented, directional statistics can be applied to the eigenvectors in order to track their motion and summarize their dispersion. When a consistently oriented eigensystem is applied to methods of machine-learning, the time series of training weights becomes interpretable in the context of the machine-learning model. Ordinary linear regression is used to demonstrate such interpretability. A reference implementation of the algorithm reported herein has been written in Python and is freely available, both as source code and through the thucyd Python package.