Field Theoretical Analysis of On-line Learning of Probability Distributions

TL;DR

This paper introduces a field theoretical approach to online learning of probability distributions, enabling optimal algorithms that adapt complexity based on data without prior model assumptions.

Contribution

It presents a novel method using renormalization group techniques to control model complexity in online distribution learning without prior parameter knowledge.

Findings

Develops an optimal online learning algorithm

Uses renormalization group to manage degrees of freedom

Does not require prior model parameters

Abstract

On-line learning of probability distributions is analyzed from the field theoretical point of view. We can obtain an optimal on-line learning algorithm, since renormalization group enables us to control the number of degrees of freedom of a system according to the number of examples. We do not learn parameters of a model, but probability distributions themselves. Therefore, the algorithm requires no a priori knowledge of a model.