A renormalization-group inspired lattice-based framework for piecewise generalized linear models

TL;DR

This paper introduces a novel, interpretable lattice-based framework inspired by renormalization group theory for piecewise generalized linear models, with theoretical analysis and competitive empirical results.

Contribution

It presents a new RG-inspired hierarchical model class with explicit partition structure, interpretability, and scalable regularization, analyzed using replica methods.

Findings

The models are almost everywhere locally linear, like ReLU networks.

The framework provides guidance on lattice design based on dataset size.

A scaling law for regularization prior improves generalization without increasing complexity.

Abstract

We formally introduce a class of models inspired by renormalization group (RG) theory, built on additive hierarchical expansions analogous to those appearing in functional ANOVA and mixed-effects models. Like ReLU convolutional neural networks, they are almost everywhere locally linear; unlike ReLU networks, their partition structure is explicit, interpretable, and easy to modify or constrain. In these models, one defines a multidimensional lattice partition of the input space and uses it to scaffold variations in regression parameters. Each dimension of the lattice corresponds to an attribute by which the statistics of the problem may vary. The parameters are themselves expressed in the form of an expansion, where each term captures variations relative to a lower (coarser) interaction scale. These models admit multiple equivalent interpretations: as piecewise GLMs, as hierarchical mixed-effects regressions, or as regression trees with structured parameter sharing. Since RG motivates the design of these models, we use techniques from statistical physics -- specifically replica analysis -- to study their generalization properties. Specifically, we analyze the behavior of the Watanabe-Akaike Information Criterion (WAIC) as a proxy for generalization loss. This analysis yields two practical results: (i) guidance on the lattice design as a function of dataset size and predictor dimensionality; and (ii) a principled scaling law for the regularization prior when adding higher-order terms to the expansion so that one can increase model complexity without an expected increase in generalization loss. We evaluate the methodology on public datasets and find performance competitive against both blackbox methods and other intrinsically interpretable approaches.