Deep Efficient Continuous Manifold Learning for Time Series Modeling

TL;DR

This paper introduces a novel deep learning framework for efficient continuous manifold learning on time series data, leveraging Riemannian geometry and Cholesky space to improve optimization and computational efficiency.

Contribution

It proposes a new method combining diffeomorphism mapping and a continuous manifold learning approach with RNNs for time series modeling.

Findings

Outperforms existing manifold and state-of-the-art methods

Efficient training with Riemannian metrics

Effective on regular and irregular time-series datasets

Abstract

Modeling non-Euclidean data is drawing extensive attention along with the unprecedented successes of deep neural networks in diverse fields. Particularly, a symmetric positive definite matrix is being actively studied in computer vision, signal processing, and medical image analysis, due to its ability to learn beneficial statistical representations. However, owing to its rigid constraints, it remains challenging to optimization problems and inefficient computational costs, especially, when incorporating it with a deep learning framework. In this paper, we propose a framework to exploit a diffeomorphism mapping between Riemannian manifolds and a Cholesky space, by which it becomes feasible not only to efficiently solve optimization problems but also to greatly reduce computation costs. Further, for dynamic modeling of time-series data, we devise a continuous manifold learning method by systematically integrating a manifold ordinary differential equation and a gated recurrent neural network. It is worth noting that due to the nice parameterization of matrices in a Cholesky space, training our proposed network equipped with Riemannian geometric metrics is straightforward. We demonstrate through experiments over regular and irregular time-series datasets that our proposed model can be efficiently and reliably trained and outperforms existing manifold methods and state-of-the-art methods in various time-series tasks.