Learning Vector Autoregressive Models with Latent Processes

TL;DR

This paper addresses the challenge of identifying the influence structure in VAR models with latent processes, proposing methods to recover the support of transition matrices and latent subgraphs under certain conditions.

Contribution

It introduces a novel approach to recover the support of transition matrices and latent subgraphs in VAR models with latent variables, including algorithms for minimal latent graph identification.

Findings

Support of transition matrix can be identified under certain conditions.

Latent subgraph reconstruction is possible if its topology is a directed tree.

Experimental results validate the theoretical guarantees on synthetic and real data.

Abstract

We study the problem of learning the support of transition matrix between random processes in a Vector Autoregressive (VAR) model from samples when a subset of the processes are latent. It is well known that ignoring the effect of the latent processes may lead to very different estimates of the influences among observed processes, and we are concerned with identifying the influences among the observed processes, those between the latent ones, and those from the latent to the observed ones. We show that the support of transition matrix among the observed processes and lengths of all latent paths between any two observed processes can be identified successfully under some conditions on the VAR model. From the lengths of latent paths, we reconstruct the latent subgraph (representing the influences among the latent processes) with a minimum number of variables uniquely if its topology is a directed tree. Furthermore, we propose an algorithm that finds all possible minimal latent graphs under some conditions on the lengths of latent paths. Our results apply to both non-Gaussian and Gaussian cases, and experimental results on various synthetic and real-world datasets validate our theoretical results.