Lookahead optimizer improves the performance of Convolutional Autoencoders for reconstruction of natural images

TL;DR

This paper demonstrates that integrating the lookahead optimizer with Adam enhances the reconstruction performance of convolutional autoencoders on natural image datasets.

Contribution

It introduces the use of lookahead optimizer with convolutional autoencoders and variational autoencoders, showing performance improvements over standard Adam optimization.

Findings

Lookahead optimizer improves CAE reconstruction quality.

Enhanced performance of CVAE with lookahead on CIFAR100.

Comparison shows significant gains over Adam alone.

Abstract

Autoencoders are a class of artificial neural networks which have gained a lot of attention in the recent past. Using the encoder block of an autoencoder the input image can be compressed into a meaningful representation. Then a decoder is employed to reconstruct the compressed representation back to a version which looks like the input image. It has plenty of applications in the field of data compression and denoising. Another version of Autoencoders (AE) exist, called Variational AE (VAE) which acts as a generative model like GAN. Recently, an optimizer was introduced which is known as lookahead optimizer which significantly enhances the performances of Adam as well as SGD. In this paper, we implement Convolutional Autoencoders (CAE) and Convolutional Variational Autoencoders (CVAE) with lookahead optimizer (with Adam) and compare them with the Adam (only) optimizer counterparts. For this purpose, we have used a movie dataset comprising of natural images for the former case and CIFAR100 for the latter case. We show that lookahead optimizer (with Adam) improves the performance of CAEs for reconstruction of natural images.