Renormalization Group-Motivated Learning

TL;DR

This paper presents an RG-inspired coarse-graining method for data analysis that effectively reduces dimensionality while preserving essential features, applicable to Gaussian, Ising, and glass systems.

Contribution

It introduces a novel coarse-graining technique based on renormalization group principles, incorporating correlation and mutual information for non-spatial data.

Findings

Effective dimensionality reduction demonstrated on Gaussian and Ising data

Preserves key features while minimizing information loss

Applicable to complex systems like glasses

Abstract

We introduce an RG-inspired coarse-graining for extracting the collective features of data. The key to successful coarse-graining lies in finding appropriate pairs of data sets. We coarse-grain the two closest data in a regular real-space RG in a lattice while considers the overall information loss in momentum-space RG. Here we compromise the two measures for the non-spatial data set. For weakly correlated data close to Gaussian, we use the correlation of data as a metric for the proximity of data points, but minimize an overall projection error for optimal coarse-graining steps. It compresses the data to maximize the correlation between the two data points to be compressed while minimizing the correlation between the paired data and other data points. We show that this approach can effectively reduce the dimensionality of the data while preserving the essential features. We extend our method to incorporate non-linear features by replacing correlation measures with mutual information. This results in an information-bottleneck-like trade-off: maximally compress the data while preserving the information among the compressed data and the rest. Indeed, our approach can be interpreted as an exact form of information-bottleneck-like trade off near linear data. We examine our method with random Gaussian data and the Ising model to demonstrate its validity and apply glass systems. Our approach has potential applications in various fields, including machine learning and statistical physics.