Scalable Matrix-valued Kernel Learning for High-dimensional Nonlinear Multivariate Regression and Granger Causality

TL;DR

This paper introduces a scalable matrix-valued kernel learning framework for high-dimensional nonlinear multivariate regression and causal inference, enabling sparse and nonlinear extensions of Granger causality with theoretical guarantees.

Contribution

It presents a novel scalable algorithm for matrix-valued kernel learning with mixed norm regularizers, extending Granger causality to nonlinear high-dimensional settings.

Findings

Algorithm is highly scalable and eigendecomposition-free.

Framework enables nonlinear causal inference with sparse solutions.

Theoretical Rademacher bounds support generalization capabilities.

Abstract

We propose a general matrix-valued multiple kernel learning framework for high-dimensional nonlinear multivariate regression problems. This framework allows a broad class of mixed norm regularizers, including those that induce sparsity, to be imposed on a dictionary of vector-valued Reproducing Kernel Hilbert Spaces. We develop a highly scalable and eigendecomposition-free algorithm that orchestrates two inexact solvers for simultaneously learning both the input and output components of separable matrix-valued kernels. As a key application enabled by our framework, we show how high-dimensional causal inference tasks can be naturally cast as sparse function estimation problems, leading to novel nonlinear extensions of a class of Graphical Granger Causality techniques. Our algorithmic developments and extensive empirical studies are complemented by theoretical analyses in terms of Rademacher generalization bounds.