Estimation of Correlation Matrices from Limited time series Data using Machine Learning

TL;DR

This paper introduces a machine learning approach to predict entire correlation matrices of dynamical systems from limited time series data of only a few nodes, enabling efficient analysis of complex networks.

Contribution

It presents a novel supervised learning method for predicting full correlation matrices from partial data, validated on real-world datasets, and provides insights through unsupervised learning.

Findings

High accuracy in correlation matrix prediction from limited data

Supervised learning effectively captures system-wide correlations

Unsupervised analysis explains prediction success

Abstract

Correlation matrices contain a wide variety of spatio-temporal information about a dynamical system. Predicting correlation matrices from partial time series information of a few nodes characterizes the spatio-temporal dynamics of the entire underlying system. This information can help to predict the underlying network structure, e.g., inferring neuronal connections from spiking data, deducing causal dependencies between genes from expression data, and discovering long spatial range influences in climate variations. Traditional methods of predicting correlation matrices utilize time series data of all the nodes of the underlying networks. Here, we use a supervised machine learning technique to predict the correlation matrix of entire systems from finite time series information of a few randomly selected nodes. The accuracy of the prediction validates that only a limited time series of a subset of the entire system is enough to make good correlation matrix predictions. Furthermore, using an unsupervised learning algorithm, we furnish insights into the success of the predictions from our model. Finally, we employ the machine learning model developed here to real-world data sets.