Soft Autoencoder and Its Wavelet Adaptation Interpretation

TL;DR

This paper introduces Soft Autoencoder, a wavelet-inspired, interpretable deep learning model with adaptable soft-thresholding, demonstrating competitive denoising performance and enhanced adaptability through a new generalized linear unit.

Contribution

It proposes Soft Autoencoder with soft-thresholding activation functions, providing a wavelet-based interpretability and a generalized linear unit for improved data filtering.

Findings

Soft-AE is interpretable as a learned wavelet shrinkage system.

Soft-AE achieves competitive denoising performance.

The generalized linear unit enhances adaptability in image processing.

Abstract

Recently, deep learning becomes the main focus of machine learning research and has greatly impacted many important fields. However, deep learning is criticized for lack of interpretability. As a successful unsupervised model in deep learning, the autoencoder embraces a wide spectrum of applications, yet it suffers from the model opaqueness as well. In this paper, we propose a new type of convolutional autoencoders, termed as Soft Autoencoder (Soft-AE), in which the activation functions of encoding layers are implemented with adaptable soft-thresholding units while decoding layers are realized with linear units. Consequently, Soft-AE can be naturally interpreted as a learned cascaded wavelet shrinkage system. Our denoising experiments demonstrate that Soft-AE not only is interpretable but also offers a competitive performance relative to its counterparts. Furthermore, we propose a generalized linear unit (GenLU) to make an autoencoder more adaptive in nonlinearly filtering images and data, such as denoising and deblurring.