Criticality & Deep Learning II: Momentum Renormalisation Group

TL;DR

This paper introduces a novel regularisation mechanism inspired by critical systems and the renormalisation group, enabling scale invariance in deep learning models through a field theory mapping and momentum space analysis.

Contribution

It develops a new inhomogeneous polynomial regularisation method based on RG flow equations, providing a theoretical framework for inducing scale invariance in deep learning.

Findings

Derivation of flow equations for deep learning couplings

Establishment of critical regularisation conditions

Demonstration of scale invariance induction in DL models

Abstract

Guided by critical systems found in nature we develop a novel mechanism consisting of inhomogeneous polynomial regularisation via which we can induce scale invariance in deep learning systems. Technically, we map our deep learning (DL) setup to a genuine field theory, on which we act with the Renormalisation Group (RG) in momentum space and produce the flow equations of the couplings; those are translated to constraints and consequently interpreted as "critical regularisation" conditions in the optimiser; the resulting equations hence prove to be sufficient conditions for - and serve as an elegant and simple mechanism to induce scale invariance in any deep learning setup.