Grouped Convolutional Neural Networks for Multivariate Time Series

TL;DR

This paper introduces two algorithms for learning structured convolutional neural networks that exploit covariance among multivariate time series, improving regression performance on real-world datasets.

Contribution

It proposes novel structure learning algorithms for CNNs that automatically partition input features based on covariance, tailored for multivariate time series analysis.

Findings

Group CNNs outperform existing CNN regression methods.

Algorithms effectively learn input groupings from data.

Improved modeling of multivariate dependencies.

Abstract

Analyzing multivariate time series data is important for many applications such as automated control, fault diagnosis and anomaly detection. One of the key challenges is to learn latent features automatically from dynamically changing multivariate input. In visual recognition tasks, convolutional neural networks (CNNs) have been successful to learn generalized feature extractors with shared parameters over the spatial domain. However, when high-dimensional multivariate time series is given, designing an appropriate CNN model structure becomes challenging because the kernels may need to be extended through the full dimension of the input volume. To address this issue, we present two structure learning algorithms for deep CNN models. Our algorithms exploit the covariance structure over multiple time series to partition input volume into groups. The first algorithm learns the group CNN structures explicitly by clustering individual input sequences. The second algorithm learns the group CNN structures implicitly from the error backpropagation. In experiments with two real-world datasets, we demonstrate that our group CNNs outperform existing CNN based regression methods.