Deconstructing the Ladder Network Architecture
Mohammad Pezeshki, Linxi Fan, Philemon Brakel, Aaron Courville, Yoshua, Bengio

TL;DR
This paper dissects the Ladder Network architecture, identifying the importance of its components through extensive experiments, and introduces a new combinator function that improves performance in semi-supervised and supervised tasks.
Contribution
It provides a detailed analysis of the Ladder Network's components, revealing their relative importance and proposing a novel combinator function that enhances accuracy.
Findings
All components are necessary but contribute unequally.
Lateral connections and noise are most critical for semi-supervised learning.
New combinator function reduces test error rates significantly.
Abstract
The Manual labeling of data is and will remain a costly endeavor. For this reason, semi-supervised learning remains a topic of practical importance. The recently proposed Ladder Network is one such approach that has proven to be very successful. In addition to the supervised objective, the Ladder Network also adds an unsupervised objective corresponding to the reconstruction costs of a stack of denoising autoencoders. Although the empirical results are impressive, the Ladder Network has many components intertwined, whose contributions are not obvious in such a complex architecture. In order to help elucidate and disentangle the different ingredients in the Ladder Network recipe, this paper presents an extensive experimental investigation of variants of the Ladder Network in which we replace or remove individual components to gain more insight into their relative importance. We find that…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsDomain Adaptation and Few-Shot Learning · Machine Learning and Data Classification · Advanced Image and Video Retrieval Techniques
