Recurrent autoencoder with sequence-aware encoding

TL;DR

This paper introduces a sequence-aware recurrent autoencoder architecture using 1D convolutional layers, significantly enhancing training speed and outperforming standard RAEs in efficiency.

Contribution

The paper proposes a novel autoencoder architecture with sequence-aware encoding that accelerates training compared to traditional recurrent autoencoders.

Findings

Training time is reduced by an order of magnitude.

Sequence-aware encoding outperforms standard RAE in most cases.

Proposed method maintains comparable or better performance.

Abstract

Recurrent Neural Networks (RNN) received a vast amount of attention last decade. Recently, the architectures of Recurrent AutoEncoders (RAE) found many applications in practice. RAE can extract the semantically valuable information, called context that represents a latent space useful for further processing. Nevertheless, recurrent autoencoders are hard to train, and the training process takes much time. In this paper, we propose an autoencoder architecture with sequence-aware encoding, which employs 1D convolutional layer to improve its performance in terms of model training time. We prove that the recurrent autoencoder with sequence-aware encoding outperforms a standard RAE in terms of training speed in most cases. The preliminary results show that the proposed solution dominates over the standard RAE, and the training process is order of magnitude faster.