An Overview on Data Representation Learning: From Traditional Feature Learning to Recent Deep Learning

TL;DR

This paper reviews the evolution of data representation learning from traditional methods to modern deep learning, highlighting key developments, resources, and future research directions in the field.

Contribution

It provides a comprehensive overview of both classical and deep learning approaches to data representation, including historical context and resource compilation.

Findings

Deep learning models have achieved top results in image, object detection, and speech tasks.

Traditional feature learning methods laid the groundwork for modern deep architectures.

The paper discusses available resources and future research directions in data representation learning.

Abstract

Since about 100 years ago, to learn the intrinsic structure of data, many representation learning approaches have been proposed, including both linear ones and nonlinear ones, supervised ones and unsupervised ones. Particularly, deep architectures are widely applied for representation learning in recent years, and have delivered top results in many tasks, such as image classification, object detection and speech recognition. In this paper, we review the development of data representation learning methods. Specifically, we investigate both traditional feature learning algorithms and state-of-the-art deep learning models. The history of data representation learning is introduced, while available resources (e.g. online course, tutorial and book information) and toolboxes are provided. Finally, we conclude this paper with remarks and some interesting research directions on data representation learning.