Memory-Gated Recurrent Networks

TL;DR

This paper introduces Memory-Gated Recurrent Networks (mGRN), a novel architecture that explicitly manages marginal and joint memories in multivariate sequential data, demonstrating superior performance over existing models.

Contribution

The paper proposes a new recurrent network architecture with gates for separate marginal and joint memories, addressing multivariate dependencies more effectively.

Findings

mGRN outperforms state-of-the-art models in experiments

Effective in capturing complex multivariate dependencies

Consistent improvements across diverse datasets

Abstract

The essence of multivariate sequential learning is all about how to extract dependencies in data. These data sets, such as hourly medical records in intensive care units and multi-frequency phonetic time series, often time exhibit not only strong serial dependencies in the individual components (the "marginal" memory) but also non-negligible memories in the cross-sectional dependencies (the "joint" memory). Because of the multivariate complexity in the evolution of the joint distribution that underlies the data generating process, we take a data-driven approach and construct a novel recurrent network architecture, termed Memory-Gated Recurrent Networks (mGRN), with gates explicitly regulating two distinct types of memories: the marginal memory and the joint memory. Through a combination of comprehensive simulation studies and empirical experiments on a range of public datasets, we show that our proposed mGRN architecture consistently outperforms state-of-the-art architectures targeting multivariate time series.