Representation Learning: A Statistical Perspective

TL;DR

This paper reviews recent advances in statistical approaches to learning data representations, focusing on unsupervised vector and matrix representation learning, highlighting their importance in deep learning and related fields.

Contribution

It provides a comprehensive overview of the statistical foundations and recent developments in unsupervised and joint vector-matrix representation learning.

Findings

Highlights the connection between classical statistical methods and modern deep learning

Summarizes recent techniques in unsupervised vector representation learning

Discusses methods for learning matrix and joint representations

Abstract

Learning representations of data is an important problem in statistics and machine learning. While the origin of learning representations can be traced back to factor analysis and multidimensional scaling in statistics, it has become a central theme in deep learning with important applications in computer vision and computational neuroscience. In this article, we review recent advances in learning representations from a statistical perspective. In particular, we review the following two themes: (a) unsupervised learning of vector representations and (b) learning of both vector and matrix representations.