Scoring and Classifying with Gated Auto-encoders

TL;DR

This paper introduces a scoring function for Gated Auto-encoders (GAEs) using a dynamical systems perspective, demonstrating their effectiveness in classification tasks and connecting them to Restricted Boltzmann Machines.

Contribution

It derives a novel scoring function for GAEs and explores their theoretical connections to RBMs, along with empirical validation on deep learning benchmarks.

Findings

GAEs can be effectively scored using the proposed dynamical systems approach.

GAEs demonstrate strong performance in single and multi-label classification tasks.

Theoretical links between GAEs and RBMs are established.

Abstract

Auto-encoders are perhaps the best-known non-probabilistic methods for representation learning. They are conceptually simple and easy to train. Recent theoretical work has shed light on their ability to capture manifold structure, and drawn connections to density modelling. This has motivated researchers to seek ways of auto-encoder scoring, which has furthered their use in classification. Gated auto-encoders (GAEs) are an interesting and flexible extension of auto-encoders which can learn transformations among different images or pixel covariances within images. However, they have been much less studied, theoretically or empirically. In this work, we apply a dynamical systems view to GAEs, deriving a scoring function, and drawing connections to Restricted Boltzmann Machines. On a set of deep learning benchmarks, we also demonstrate their effectiveness for single and multi-label classification.