Statistical physics, Bayesian inference and neural information processing

TL;DR

This paper explores neural information processing using statistical physics, focusing on Bayesian inference, generalized linear models, and dimensionality reduction techniques to understand learning and generalization.

Contribution

It provides a comprehensive overview connecting statistical physics principles with neural network learning methods and Bayesian inference.

Findings

Bayesian inference linked to Gibbs learning description

Generalized Linear Models as alternatives to backpropagation

Techniques for linear and non-linear dimensionality reduction

Abstract

Lecture notes from the course given by Professor Sara A. Solla at the Les Houches summer school on "Statistical physics of Machine Learning". The notes discuss neural information processing through the lens of Statistical Physics. Contents include Bayesian inference and its connection to a Gibbs description of learning and generalization, Generalized Linear Models as a controlled alternative to backpropagation through time, and linear and non-linear techniques for dimensionality reduction.