Extended Deep Adaptive Input Normalization for Preprocessing Time Series Data for Neural Networks

TL;DR

The paper introduces EDAIN, an adaptive neural layer that learns to normalize irregular time series data end-to-end, improving neural network performance in prediction and classification tasks.

Contribution

It proposes the EDAIN layer, a novel adaptive normalization method that optimizes parameters during training for better handling of irregular time series data.

Findings

EDAIN outperforms traditional normalization methods.

EDAIN improves model accuracy on real-world datasets.

The approach is effective across synthetic and large-scale datasets.

Abstract

Data preprocessing is a crucial part of any machine learning pipeline, and it can have a significant impact on both performance and training efficiency. This is especially evident when using deep neural networks for time series prediction and classification: real-world time series data often exhibit irregularities such as multi-modality, skewness and outliers, and the model performance can degrade rapidly if these characteristics are not adequately addressed. In this work, we propose the EDAIN (Extended Deep Adaptive Input Normalization) layer, a novel adaptive neural layer that learns how to appropriately normalize irregular time series data for a given task in an end-to-end fashion, instead of using a fixed normalization scheme. This is achieved by optimizing its unknown parameters simultaneously with the deep neural network using back-propagation. Our experiments, conducted using synthetic data, a credit default prediction dataset, and a large-scale limit order book benchmark dataset, demonstrate the superior performance of the EDAIN layer when compared to conventional normalization methods and existing adaptive time series preprocessing layers.