Learning with hidden variables

TL;DR

This paper reviews recent advances in neural network algorithms for learning features from complex, dynamical inputs, highlighting the role of hidden variables and single neuron models in understanding cortical learning.

Contribution

It provides a comprehensive overview of recent developments in learning with hidden variables, connecting machine learning methods to cortical neural processes.

Findings

Advances in algorithms for learning features from natural signals.

Insights into processing dynamical inputs with hidden nodes.

Discussion of single neuron models in cortical learning.

Abstract

Learning and inferring features that generate sensory input is a task continuously performed by cortex. In recent years, novel algorithms and learning rules have been proposed that allow neural network models to learn such features from natural images, written text, audio signals, etc. These networks usually involve deep architectures with many layers of hidden neurons. Here we review recent advancements in this area emphasizing, amongst other things, the processing of dynamical inputs by networks with hidden nodes and the role of single neuron models. These points and the questions they arise can provide conceptual advancements in understanding of learning in the cortex and the relationship between machine learning approaches to learning with hidden nodes and those in cortical circuits.