Laplacian Denoising Autoencoder

TL;DR

This paper introduces a novel Laplacian denoising autoencoder that corrupts data in the gradient domain across multiple scales, leading to more robust representations useful for various visual tasks.

Contribution

It proposes a new multi-scale denoising autoencoder framework using Laplacian pyramid representations for improved unsupervised learning.

Findings

Outperforms single-scale corruption autoencoders on visual benchmarks

Learns representations that transfer effectively to downstream tasks

Exploits multi-scale data structures for robustness

Abstract

While deep neural networks have been shown to perform remarkably well in many machine learning tasks, labeling a large amount of ground truth data for supervised training is usually very costly to scale. Therefore, learning robust representations with unlabeled data is critical in relieving human effort and vital for many downstream tasks. Recent advances in unsupervised and self-supervised learning approaches for visual data have benefited greatly from domain knowledge. Here we are interested in a more generic unsupervised learning framework that can be easily generalized to other domains. In this paper, we propose to learn data representations with a novel type of denoising autoencoder, where the noisy input data is generated by corrupting latent clean data in the gradient domain. This can be naturally generalized to span multiple scales with a Laplacian pyramid representation of the input data. In this way, the agent learns more robust representations that exploit the underlying data structures across multiple scales. Experiments on several visual benchmarks demonstrate that better representations can be learned with the proposed approach, compared to its counterpart with single-scale corruption and other approaches. Furthermore, we also demonstrate that the learned representations perform well when transferring to other downstream vision tasks.