Uncertainty-Aware Deep Attention Recurrent Neural Network for Heterogeneous Time Series Imputation

TL;DR

This paper introduces DEARI, a deep attention-based recurrent neural network that jointly imputes missing values and estimates their uncertainty in heterogeneous multivariate time series, improving accuracy and confidence measurement.

Contribution

The paper presents DEARI, a novel deep attention recurrent model with a Bayesian extension for uncertainty-aware imputation in complex multivariate time series.

Findings

DEARI outperforms state-of-the-art methods on real-world datasets.

The Bayesian version of DEARI provides reliable uncertainty estimates.

DEARI achieves stable convergence and high imputation accuracy.

Abstract

Missingness is ubiquitous in multivariate time series and poses an obstacle to reliable downstream analysis. Although recurrent network imputation achieved the SOTA, existing models do not scale to deep architectures that can potentially alleviate issues arising in complex data. Moreover, imputation carries the risk of biased estimations of the ground truth. Yet, confidence in the imputed values is always unmeasured or computed post hoc from model output. We propose DEep Attention Recurrent Imputation (DEARI), which jointly estimates missing values and their associated uncertainty in heterogeneous multivariate time series. By jointly representing feature-wise correlations and temporal dynamics, we adopt a self attention mechanism, along with an effective residual component, to achieve a deep recurrent neural network with good imputation performance and stable convergence. We also leverage self-supervised metric learning to boost performance by optimizing sample similarity. Finally, we transform DEARI into a Bayesian neural network through a novel Bayesian marginalization strategy to produce stochastic DEARI, which outperforms its deterministic equivalent. Experiments show that DEARI surpasses the SOTA in diverse imputation tasks using real-world datasets, namely air quality control, healthcare and traffic.