Local Loss Optimization in Operator Models: A New Insight into Spectral Learning

TL;DR

This paper introduces a local loss optimization approach for spectral learning of latent variable models, providing a new perspective that improves accuracy and flexibility through regularization and probabilistic strategies.

Contribution

It presents a novel local loss formulation, a regularized convex relaxation, and probabilistic guarantees for spectral operator learning methods.

Findings

Regularization improves accuracy and model complexity trade-off.

A randomized strategy for local loss selection succeeds with high probability.

The method offers a new perspective on spectral learning of latent variable models.

Abstract

This paper re-visits the spectral method for learning latent variable models defined in terms of observable operators. We give a new perspective on the method, showing that operators can be recovered by minimizing a loss defined on a finite subset of the domain. A non-convex optimization similar to the spectral method is derived. We also propose a regularized convex relaxation of this optimization. We show that in practice the availabilty of a continuous regularization parameter (in contrast with the discrete number of states in the original method) allows a better trade-off between accuracy and model complexity. We also prove that in general, a randomized strategy for choosing the local loss will succeed with high probability.