Multi-Relevance: Coexisting but Distinct Notions of Scale in Large Systems

TL;DR

This paper explores how mixture models can exhibit multiple independent notions of scale, called multi-relevance, which challenges traditional RG and PCA methods in high-dimensional systems.

Contribution

It introduces the concept of multi-relevance in mixture models, demonstrating its implications for RG analysis and coarse-graining techniques like PCA.

Findings

Mixture models can have multiple coexisting RG flows with distinct scales.

Linear classifiers can infer latent states in some regimes, affecting RG analysis.

PCA may be inadequate for systems with multi-relevance, leading to oversimplification.

Abstract

Renormalization group (RG) methods are emerging as tools in biology and computer science to support the search for simplifying structure in distributions over high-dimensional spaces. We show that mixture models can be thought of as having multiple coexisting, exactly independent RG flows, each with its own notion of scale. We define this property as ``multi-relevance''. As an example, we construct a model that has two distinct notions of scale, each corresponding to the state of an unobserved categorical variable. In the regime where this latent variable can be inferred using a linear classifier, the vertex expansion approach in non-perturbative RG can be applied successfully but will give different answers depending the choice of expansion point in state space. In the regime where linear estimation of the latent state fails, we show that the vertex expansion predicts a decrease in the total number of relevant couplings from four to three and does not admit a good polynomial truncation scheme. This indicates oversimplification. One consequence of this is that principal component analysis (PCA) may be a poor choice of coarse-graining scheme in multi-relevant systems, since it imposes a notion of scale which is incorrect from the RG perspective. Taken together, our results indicate that RG and PCA can lead to oversimplification when multi-relevance is present and not accounted for.